7 deleted 664 characters in body edited Aug 16 '17 at 6:26 D.W.♦ 108k1414 gold badges143143 silver badges318318 bronze badges I have this dataset, and I am using y = (a * x^n) / (b + x^n) Hill function as the model, where a is the limit of the Hill curve, b is the point at which a/2 is reached (for n = 1) and n is the cooperativity or steepness of the curve. Currently, I am storing all Xx,y values, and computing the parameters from scipy.optimize.curve_fita,b,n, and plotting using black-box optimization to minimize the curveleast-squares error. The method (I currently use scipy.optimize.curve_fit is, a standard optimizer that given a set of X,y data-points (tuples of X-coordinates and associated y-coordinates) finds the co-efficient of the Hillcan minimize an arbitrary objective function described above such the least-squared error of the final curve to the data-points is minimized. To re-iterate In my case, scipy.optimize.curve_fitthe objective function is a (wrapper around)the total least-squaredsquares error minimizer (more specifically, scipy.optimize.least_squares.) You can use/suggest any black-box optimization algorithm in place of scipy.optimize.curve_fit, I do not care.. However, whenWhen new data points come along, I re-calculate the parameters with the old+new data, i.e. I append the new data to the old tuple of Xx,y values and get the new parameters. Is there a way to update the parameters of the model without storing all of the previous old data points, once the initial parameters are obtained from the previous data points? I am looking for some kind of on-line algorithm to tackle the problem, but have no clue how to go about it. I read up a blog-post that shows to generalize any given (fixed-)polynomialhow do this for polynomial regression function to its equivalent vector update form, but as the power of x which is n is a parameter to the model, I have no idea how to extend the approach to my specific function. I am looking for a solution like so: I fit the curve to the first 1000 data points and have my parameters. Next, I discard some or all of the old data. Then, when I see the 1001st point I simply update my parameters and plot the curve again and so on for every new data point. EDIT M̶y̶ ̶e̶x̶i̶s̶t̶i̶n̶g̶ ̶c̶o̶d̶e̶ ̶i̶s̶ ̶a̶s̶ ̶f̶o̶l̶l̶o̶w̶s̶ ̶(̶n̶o̶t̶ ̶s̶u̶p̶e̶r̶ ̶e̶l̶e̶g̶a̶n̶t̶)̶.̶ I had to remove my code as someone in the comments suggested that the problem statement is better off without the code. I have this dataset, and I am using y = (a * x^n) / (b + x^n) Hill function as the model, where a is the limit of the Hill curve, b is the point at which a/2 is reached (for n = 1) and n is the cooperativity or steepness of the curve. Currently, I am storing all X,y values, computing the parameters from scipy.optimize.curve_fit, and plotting the curve. The method scipy.optimize.curve_fit is a standard optimizer that given a set of X,y data-points (tuples of X-coordinates and associated y-coordinates) finds the co-efficient of the Hill function described above such the least-squared error of the final curve to the data-points is minimized. To re-iterate, scipy.optimize.curve_fit is a (wrapper around) least-squared error minimizer (more specifically, scipy.optimize.least_squares) You can use/suggest any black-box optimization algorithm in place of scipy.optimize.curve_fit, I do not care. However, when new data points come along, I re-calculate the parameters with the old+new data, i.e. I append the new data to the old tuple of X,y values and get the new parameters. Is there a way to update the parameters of the model without storing all of the previous old data points, once the initial parameters are obtained from the previous data points? I am looking for some kind of on-line algorithm to tackle the problem, but have no clue how to go about it. I read up a blog-post that shows to generalize any given (fixed-)polynomial regression function to its equivalent vector update form, but as the power of x which is n is a parameter to the model, I have no idea how to extend the approach to my specific function. I am looking for a solution like so: I fit the curve to the first 1000 data points and have my parameters. Next, I discard some or all of the old data. Then, when I see the 1001st point I simply update my parameters and plot the curve again and so on for every new data point. EDIT M̶y̶ ̶e̶x̶i̶s̶t̶i̶n̶g̶ ̶c̶o̶d̶e̶ ̶i̶s̶ ̶a̶s̶ ̶f̶o̶l̶l̶o̶w̶s̶ ̶(̶n̶o̶t̶ ̶s̶u̶p̶e̶r̶ ̶e̶l̶e̶g̶a̶n̶t̶)̶.̶ I had to remove my code as someone in the comments suggested that the problem statement is better off without the code. I have this dataset, and I am using y = (a * x^n) / (b + x^n) Hill function as the model, where a is the limit of the Hill curve, b is the point at which a/2 is reached (for n = 1) and n is the cooperativity or steepness of the curve. Currently, I am storing all x,y values and computing the parameters a,b,n using black-box optimization to minimize the least-squares error. (I currently use scipy.optimize.curve_fit, a standard optimizer that can minimize an arbitrary objective function. In my case, the objective function is the total least-squares error.). When new data points come along, I re-calculate the parameters with the old+new data, i.e. I append the new data to the old tuple of x,y values and get the new parameters. Is there a way to update the parameters of the model without storing all of the previous old data points, once the initial parameters are obtained from the previous data points? I am looking for some kind of on-line algorithm to tackle the problem, but have no clue how to go about it. I read a blog-post that shows how do this for polynomial regression, but as the power of x which is n is a parameter to the model, I have no idea how to extend the approach to my specific function. I am looking for a solution like so: I fit the curve to the first 1000 data points and have my parameters. Next, I discard some or all of the old data. Then, when I see the 1001st point I simply update my parameters and plot the curve again and so on for every new data point. 6 added 15 characters in body edited Aug 16 '17 at 1:41 neo4k 633 bronze badges I have this dataset, and I am using y = (a * x^n) / (b + x^n) Hill function as the model, where a is the limit of the Hill curve, b is the point of inflectionat which a/2 is reached (for n = 1) and n is the cooperativity or steepness of the curve. Currently, I am storing all X,y values, computing the parameters from scipy.optimize.curve_fit, and plotting the curve. The method scipy.optimize.curve_fit is a standard optimizer that given a set of X,y data-points (tuples of X-coordinates and associated y-coordinates) finds the co-efficient of the Hill function described above such the least-squared error of the final curve to the data-points is minimized. To re-iterate, scipy.optimize.curve_fit is a (wrapper around) least-squared error minimizer (more specifically, scipy.optimize.least_squares) You can use/suggest any black-box optimization algorithm in place of scipy.optimize.curve_fit, I do not care. However, when new data points come along, I re-calculate the parameters with the old+new data, i.e. I append the new data to the old tuple of X,y values and get the new parameters. Is there a way to update the parameters of the model without storing all of the previous old data points, once the initial parameters are obtained from the previous data points? I am looking for some kind of on-line algorithm to tackle the problem, but have no clue how to go about it. I read up a blog-post that shows to generalize any given (fixed-)polynomial regression function to its equivalent vector update form, but as the power of x which is n is a parameter to the model, I have no idea how to extend the approach to my specific function. I am looking for a solution like so: I fit the curve to the first 1000 data points and have my parameters. Next, I discard some or all of the old data. Then, when I see the 1001st point I simply update my parameters and plot the curve again and so on for every new data point. EDIT M̶y̶ ̶e̶x̶i̶s̶t̶i̶n̶g̶ ̶c̶o̶d̶e̶ ̶i̶s̶ ̶a̶s̶ ̶f̶o̶l̶l̶o̶w̶s̶ ̶(̶n̶o̶t̶ ̶s̶u̶p̶e̶r̶ ̶e̶l̶e̶g̶a̶n̶t̶)̶.̶ I had to remove my code as someone in the comments suggested that the problem statement is better off without the code. I have this dataset, and I am using y = (a * x^n) / (b + x^n) Hill function as the model, where a is the limit of the Hill curve, b is the point of inflection and n is the cooperativity or steepness of the curve. Currently, I am storing all X,y values, computing the parameters from scipy.optimize.curve_fit, and plotting the curve. The method scipy.optimize.curve_fit is a standard optimizer that given a set of X,y data-points (tuples of X-coordinates and associated y-coordinates) finds the co-efficient of the Hill function described above such the least-squared error of the final curve to the data-points is minimized. To re-iterate, scipy.optimize.curve_fit is a (wrapper around) least-squared error minimizer (more specifically, scipy.optimize.least_squares) You can use/suggest any black-box optimization algorithm in place of scipy.optimize.curve_fit, I do not care. However, when new data points come along, I re-calculate the parameters with the old+new data, i.e. I append the new data to the old tuple of X,y values and get the new parameters. Is there a way to update the parameters of the model without storing all of the previous old data points, once the initial parameters are obtained from the previous data points? I am looking for some kind of on-line algorithm to tackle the problem, but have no clue how to go about it. I read up a blog-post that shows to generalize any given (fixed-)polynomial regression function to its equivalent vector update form, but as the power of x which is n is a parameter to the model, I have no idea how to extend the approach to my specific function. I am looking for a solution like so: I fit the curve to the first 1000 data points and have my parameters. Next, I discard some or all of the old data. Then, when I see the 1001st point I simply update my parameters and plot the curve again and so on for every new data point. EDIT M̶y̶ ̶e̶x̶i̶s̶t̶i̶n̶g̶ ̶c̶o̶d̶e̶ ̶i̶s̶ ̶a̶s̶ ̶f̶o̶l̶l̶o̶w̶s̶ ̶(̶n̶o̶t̶ ̶s̶u̶p̶e̶r̶ ̶e̶l̶e̶g̶a̶n̶t̶)̶.̶ I had to remove my code as someone in the comments suggested that the problem statement is better off without the code. I have this dataset, and I am using y = (a * x^n) / (b + x^n) Hill function as the model, where a is the limit of the Hill curve, b is the point at which a/2 is reached (for n = 1) and n is the cooperativity or steepness of the curve. Currently, I am storing all X,y values, computing the parameters from scipy.optimize.curve_fit, and plotting the curve. The method scipy.optimize.curve_fit is a standard optimizer that given a set of X,y data-points (tuples of X-coordinates and associated y-coordinates) finds the co-efficient of the Hill function described above such the least-squared error of the final curve to the data-points is minimized. To re-iterate, scipy.optimize.curve_fit is a (wrapper around) least-squared error minimizer (more specifically, scipy.optimize.least_squares) You can use/suggest any black-box optimization algorithm in place of scipy.optimize.curve_fit, I do not care. However, when new data points come along, I re-calculate the parameters with the old+new data, i.e. I append the new data to the old tuple of X,y values and get the new parameters. Is there a way to update the parameters of the model without storing all of the previous old data points, once the initial parameters are obtained from the previous data points? I am looking for some kind of on-line algorithm to tackle the problem, but have no clue how to go about it. I read up a blog-post that shows to generalize any given (fixed-)polynomial regression function to its equivalent vector update form, but as the power of x which is n is a parameter to the model, I have no idea how to extend the approach to my specific function. I am looking for a solution like so: I fit the curve to the first 1000 data points and have my parameters. Next, I discard some or all of the old data. Then, when I see the 1001st point I simply update my parameters and plot the curve again and so on for every new data point. EDIT M̶y̶ ̶e̶x̶i̶s̶t̶i̶n̶g̶ ̶c̶o̶d̶e̶ ̶i̶s̶ ̶a̶s̶ ̶f̶o̶l̶l̶o̶w̶s̶ ̶(̶n̶o̶t̶ ̶s̶u̶p̶e̶r̶ ̶e̶l̶e̶g̶a̶n̶t̶)̶.̶ I had to remove my code as someone in the comments suggested that the problem statement is better off without the code. 5 added 8 characters in body edited Aug 16 '17 at 0:41 neo4k 633 bronze badges I have this dataset, and I am using y = (a * x^n) / (b + x^n) Hill function as the model, where a is the limit of the Hill curve, b is the point of inflection and n is the cooperativity or steepness of the curve. Currently, I am storing all X,y values, computing the parameters from scipy.optimize.curve_fit, and plotting the curve. The method scipy.optimize.curve_fit is a standard optimizer that given a set of X,y data-points (tuples of X-coordinates and associated y-coordinates) finds the co-efficient of the Hill function described above such the least-squared error of the final curve to the data-points is minimized. To re-iterate, scipy.optimize.curve_fit is a (wrapper around) least-squared error minimizer (more specifically, scipy.optimize.least_squares) You can use/suggest any black-box optimization algorithm in place of scipy.optimize.curve_fit, I do not care. However, when new data points come along, I re-calculate the parameters with the old+new data, i.e. I append the new data to the old tuple of X,y values and get the new parameters. Is there a way to update the parameters of the model without storing all of the previous old data points, once the initial parameters are obtained from the previous data points? I am looking for some kind of on-line algorithm to tackle the problem, but have no clue how to go about it. I read up a blog-post that shows to generalize any given polynomial(fixed-)polynomial regression function to its equivalent vector update form, but as the power of x which is n is a parameter to the model, I have no idea how to extend the approach to my specific function. I am looking for a solution like so: I fit the curve to the first 1000 data points and have my parameters. Next, I discard some or all of the old data. Then, when I see the 1001st point I simply update my parameters and plot the curve again and so on for every new data point. EDIT M̶y̶ ̶e̶x̶i̶s̶t̶i̶n̶g̶ ̶c̶o̶d̶e̶ ̶i̶s̶ ̶a̶s̶ ̶f̶o̶l̶l̶o̶w̶s̶ ̶(̶n̶o̶t̶ ̶s̶u̶p̶e̶r̶ ̶e̶l̶e̶g̶a̶n̶t̶)̶.̶ I had to remove my code as someone in the comments suggested that the problem statement is better off without the code. I have this dataset, and I am using y = (a * x^n) / (b + x^n) Hill function as the model, where a is the limit of the Hill curve, b is the point of inflection and n is the cooperativity or steepness of the curve. Currently, I am storing all X,y values, computing the parameters from scipy.optimize.curve_fit, and plotting the curve. The method scipy.optimize.curve_fit is a standard optimizer that given a set of X,y data-points (tuples of X-coordinates and associated y-coordinates) finds the co-efficient of the Hill function described above such the least-squared error of the final curve to the data-points is minimized. To re-iterate, scipy.optimize.curve_fit is a (wrapper around) least-squared error minimizer (more specifically, scipy.optimize.least_squares) You can use/suggest any black-box optimization algorithm in place of scipy.optimize.curve_fit, I do not care. However, when new data points come along, I re-calculate the parameters with the old+new data, i.e. I append the new data to the old tuple of X,y values and get the new parameters. Is there a way to update the parameters of the model without storing all of the previous old data points, once the initial parameters are obtained from the previous data points? I am looking for some kind of on-line algorithm to tackle the problem, but have no clue how to go about it. I read up a blog-post that shows to generalize any given polynomial regression function to its equivalent vector update form, but as the power of x which is n is a parameter to the model, I have no idea how to extend the approach to my specific function. I am looking for a solution like so: I fit the curve to the first 1000 data points and have my parameters. Next, I discard some or all of the old data. Then, when I see the 1001st point I simply update my parameters and plot the curve again and so on for every new data point. EDIT M̶y̶ ̶e̶x̶i̶s̶t̶i̶n̶g̶ ̶c̶o̶d̶e̶ ̶i̶s̶ ̶a̶s̶ ̶f̶o̶l̶l̶o̶w̶s̶ ̶(̶n̶o̶t̶ ̶s̶u̶p̶e̶r̶ ̶e̶l̶e̶g̶a̶n̶t̶)̶.̶ I had to remove my code as someone in the comments suggested that the problem statement is better off without the code. I have this dataset, and I am using y = (a * x^n) / (b + x^n) Hill function as the model, where a is the limit of the Hill curve, b is the point of inflection and n is the cooperativity or steepness of the curve. Currently, I am storing all X,y values, computing the parameters from scipy.optimize.curve_fit, and plotting the curve. The method scipy.optimize.curve_fit is a standard optimizer that given a set of X,y data-points (tuples of X-coordinates and associated y-coordinates) finds the co-efficient of the Hill function described above such the least-squared error of the final curve to the data-points is minimized. To re-iterate, scipy.optimize.curve_fit is a (wrapper around) least-squared error minimizer (more specifically, scipy.optimize.least_squares) You can use/suggest any black-box optimization algorithm in place of scipy.optimize.curve_fit, I do not care. However, when new data points come along, I re-calculate the parameters with the old+new data, i.e. I append the new data to the old tuple of X,y values and get the new parameters. Is there a way to update the parameters of the model without storing all of the previous old data points, once the initial parameters are obtained from the previous data points? I am looking for some kind of on-line algorithm to tackle the problem, but have no clue how to go about it. I read up a blog-post that shows to generalize any given (fixed-)polynomial regression function to its equivalent vector update form, but as the power of x which is n is a parameter to the model, I have no idea how to extend the approach to my specific function. I am looking for a solution like so: I fit the curve to the first 1000 data points and have my parameters. Next, I discard some or all of the old data. Then, when I see the 1001st point I simply update my parameters and plot the curve again and so on for every new data point. EDIT M̶y̶ ̶e̶x̶i̶s̶t̶i̶n̶g̶ ̶c̶o̶d̶e̶ ̶i̶s̶ ̶a̶s̶ ̶f̶o̶l̶l̶o̶w̶s̶ ̶(̶n̶o̶t̶ ̶s̶u̶p̶e̶r̶ ̶e̶l̶e̶g̶a̶n̶t̶)̶.̶ I had to remove my code as someone in the comments suggested that the problem statement is better off without the code. 4 added 148 characters in body edited Aug 16 '17 at 0:31 neo4k 633 bronze badges 3 incorporated moderator suggestion. edited Aug 16 '17 at 0:20 neo4k 633 bronze badges 2 incorporated moderator suggestion. edited Aug 16 '17 at 0:14 neo4k 633 bronze badges 1 asked Aug 14 '17 at 5:32 neo4k 633 bronze badges