# What is a logical approach to developing an algorithm which can find the optimal parameters for a function which make it best fit a given data set?

Consider the highlighted columns in the following table:

Starting with 100,000 newborns, $l_{x+2}$ denotes the number of individuals in the sample still alive at age $x+2$. If we consider Makeham's law of mortality, defined by the function $$S_0(x) = \exp \left( -Ax - \frac{Bc^x - 1}{\log (c)} \right)$$ we find that for the correct parameters $A, B$ and $c$ this formula very closely models the actual number of deaths.

So my question is this, given a data set and a model with unknown parameters, what computational method may be used to approximate the optimal parameters for the model (which most closely fit the given data set)?

As a more simple example, suppose we are given the data set $$X = \left\{ 1, 3, 5.1, 7, 9 \right\}$$ and are required to find the optimal parameters $A$ and $B$ such that the model $$f(x) = Ax + B$$ most closely fits this data set. As humans, it is clear to see that the optimal parameters are $A = 2$ and $B = 1$. How might a computer deduce these values?

• This question may be too broad. Maybe you can take a look at the field of estimation theory. – xskxzr Feb 9 '18 at 14:04

Suppose your model is $f(x,\overline{p})$, where $p$ is the parameters vector, and you wish to find the vector $\overline{p}$ which minimizes $\sum\limits_{i=1}^n\left( y_i-f(x_i,\overline{p})\right)^2$. This problem is known as least squares fitting, and if your model $f(x,\overline{p})$ is linear, then the optimal value of $\overline{p}$ is given by ${(x^Tx)^{-1}x^Ty}$. For more complicated models, the solution might not be so easy to find analytically, and one might need to settle for approximations. One option is to use iterative methods, such as the Gauss-Newton algorithm.
There is no general way to deal with any type of error function. If for example your cost function does not allow any mistake, and your model $f$ is polynomial in the parameters, then the task reduces to solving a system of polynomial equations. Different types of error functions can take you to different areas in optimization, or even in mathematics in general.