I have already taken a college course at my uni on machine learning where we implemented all the basic ML programs: linear regression, logistic regression, basic neural network with logistic regression (not perceptron, but we learned the theory of perceptron as a history lesson), k-means, and naive Bayes classifier. The class also had a high focus on the theory behind these algorithms so i know a lot of the relates maths.

But all of our projects were based on simple numbers. What I mean by that is all of the projects had features which were simple numbers such as miles per gallon, year, horsepower, weight, frequency, etc. We never made anything that could understand more abstract things like text, or color, etc.

I recently stumbled upon this article about a recurrent neural network that makes up its own Magic: The Gathering cards and my interest in ML was piqued again. I want to learn to implement something which can learn about things besides basic numbers, I want to make something that can learn to put sentences together like the one in this article. Hell it even makes up its own words (fuseback) that don't exist in Magic and added rules text to them (like for Tromple).

What resources are there to learn how to make a system which can learn these more abstract ideas like words and colors? I don't understand how the neural network can come up with it's own words. All the machine learning stuff i did only classified test data into existing sets (or predicted a number on a feature), but it never created a new feature.

  • $\begingroup$ A reference request like yours is too broad for Stack Exchange -- you ask for a survey of a whole research area! You need to narrow your focus considerably before a question of reasonable scope appears. Try talking to your advisor(s), search with Google Scholar and check out this guide to better (re)searches on Academia. $\endgroup$ – Raphael Jul 14 '15 at 5:39
  • $\begingroup$ well i was hoping for a general introduction for creating novel samples from a training/test set input into a neural network, is that too broad? $\endgroup$ – user2237160 Jul 14 '15 at 5:49