Suppose I have only three independent features (x, y, z) as the input to some machine learning routine (e.g. neural network). From some domain knowledge, I know that the output o(x, y, z) must have the specific functional form

o(x, y, z) = f(x)*g(y)*g(z)

where the g(.) are the same function. The details of f(.) and g(.) are not known beforehand (except that f(.) is a decaying function with respect to x). Given that there is no upper limit on the sample size, is it possible to incorporate such a functional form (or in general, any specific functional form) into the machine learning routine?


Yup. You define two neural nets (or other models) $f,g$ with weights $\theta,\psi$, define

$$o_{\theta,\psi}(x,y,z) = f_\theta(x) g_\psi(y) g_\psi(z),$$

define a loss function (which compares the predicted output $o_{\theta,\psi}(x,y,z)$ to the desired output), and then use stochastic gradient descent to find weights $\theta,\psi$ that minimize the aggregate loss on the training set.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.