build a model with fuzzy prior knowledge and train it

Question

Given that there are variables x1, x2, x3, and y, I know that x1, x2 are positively correlated with y, while x3 is negatively correlated with y, but the specific quantitative relationships are unknown. Furthermore, I have a dataset (X, y) pertaining to these variables, which is of limited size. To train an accurate model using this limited data, I wish to incorporate the prior knowledge about the correlations between x1, x2, x3, and y into the model, so as to utilize automatic differentiation frameworks like JAX to train a reliable model with a small amount of data.

Is there any method to build such a model and train it?

Answer 1

Pragmatically, the most realistic path is probably to build a model to predict y from x1,x2,x3, without trying to make any special use of your knowledge about the correlation structure. Most likely any reasonable classifier/regression model will automatically learn that correlation structure.

If it is critical to use that prior knowledge and you know that there is a strict monotonicity relationship, then you can use a monotonic model. e.g., https://en.wikipedia.org/wiki/Isotonic_regression, XGBoost with monotonicity constraints, monotonic classifiers, or linear regression with a constraint that forces the coefficients for x1,x2 to be positive and the coefficient for x3 to be negative (e.g., you can learn such a linear regression model using projected gradient descent).