I've recently viewed the paper of AlphaGo, which has done gradient-based reinforcement learning to get stronger. The learning was successful enough to beat a human master, but in this case, supervised learning with a large database of master level human games was preceded the reinforcement learning. For a complex enough game as go, one can expect that the search space for the policy function would not be smooth at all. So supposedly supervised learning was necessary to guide the policy function to a good starting point before reinforcement. Without such, applying reinforcement learning directly to a random policy can easily make the policy stuck at a bad local optimum. I could have a miunderstanding at this point; correct me if so, but to continue on: if it is hard to have "the good starting point" such as a trained policy from human expert game records, what is a way to devise one. I've had a look on NEAT and HyperNEAT, which are evolutionary methods. Do these evolutionary algorithms scale well on complex strategic decision processes and not just on simple linear decisions such as food gathering and danger avoidance? In case not, what alternatives are known? Is there any success case of a chess, go, or any kind of complex strategic game playing algorithm, where it gained expert strength without domain knowledge such as expert game examples?

  • $\begingroup$ The title of your question seem to have little to do with what you're actually asking. Please clarify your question. Indeed, being stuck in local-optima is a problem, which can be approached with e.g. simulated annealing. However, the real problem you get when you lack domain knowledge or training is not finding the optimum, but the value of a game position, which is hard for any non-trivial combinatorial game. (The value of certain opening positions in Chess is debatable even among experts, for example) $\endgroup$ – Discrete lizard Feb 25 '17 at 17:51
  • $\begingroup$ @Discretelizard Please suggest me a good title, or you can edit this post. I was also confused when writing the title. $\endgroup$ – MinjaeKim Feb 26 '17 at 22:40
  • $\begingroup$ Apart from the title being uninformative, you are discussing quite a few things in your question, to the point that your question becomes unclear to me. If you could clarify what you're asking precisely, we might be able to find a proper title (and perhaps an answer!). $\endgroup$ – Discrete lizard Feb 26 '17 at 22:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.