I have been given a task to dynamically learn the optimum value of a parameter in a Heuristics Filtering Algorithm used in a tool.

The accuracy of the tool increase as the value of K in the heuristics filtering algorithm increases. How ever a lower K leads the tool towards a low resource consumption (memory, time and temporary storage).

How can I dynamically learning the optimum value of K using Machine Learning techniques so that the tool can dynamically generate an optimal value of K.

I have been looking into Reinforcement Learning and learning from online resources. First, the problem has to be modeled as a Markov Decision Process (s, a, r, s'). I got stuck at what my State Space s should be. My set of actions a (they represent the value that k should be) are {10,20,30...,100}. Is Reinforcement Learning the way to go in the first place? If not, what other machine learning techniques do you suggest? I'm not into AI but I enjoy solving problems

  • $\begingroup$ By `optimum', I assume you mean some tradeoff between accuracy and resource consumption? If so, this can be framed as a multi-objective problem. $\endgroup$ Dec 20, 2015 at 12:37
  • $\begingroup$ @user217281728 yes you are correct. Thanks for your response. It's really enlightening $\endgroup$
    – aig
    Dec 22, 2015 at 10:38

1 Answer 1


First some questions:

  • Can $k$ only take on 10 possible values?
  • From what you've said above, the measure of the 'quality' of a given value for $k$ would be an n-tuple e.g. $(accuracy,memory, time, storage)$, right?

So 'dynamically learning a value for $k$' involves learning a good choice based on $L$, the list of n-tuples for previous values of $k$.

Here are two very simple schemes:

If your problem is genuinely multi-objective, then you can always just pick the associated $k$ at random from the Pareto-front of $L$.

If your multiple objectives can be combined into a single quality value, then fit a differentiable curve (e.g. a polynomial) to these values, then symbolically determine the optimum of that curve and the associated value of $k$. Most computer algebra packages support the functionality for this, as does the Apache Commons Math library for Java.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.