Are neural networks a type of reinforcement learning or are they different?

Question

Can neural networks be considered a form of reinforcement learning or is there some essential difference between the two?

By the same token could we consider neural networks a sub-class of genetic algorithms?

According to my current understanding the taxonomy is kind of like this:

   Reinforcement Learning
      Evolutionary Algorithms
         Genetic Algorithms
            Neural Networks

Is this right?

Answer 1

What is a neural network?

Neural networks are algorithms for function approximation. I like to call them a construction kit for functions. Their basic building block is a neuron, commonly visualized like this:

You can see the $n$ inputs $x_1, \dots, x_n$ ($x_0$ is typically constant 1), each multiplied with a weight $w_i \in \mathbb{R}$. This gets summed up and an activation function $\varphi$ (e.g. sigmoid $\varphi(x) = \frac{1}{1+e^{-x}}$) gets applied. So a neuron is a function

$$f(x_0, \dots, x_n) = \varphi(\sum_{i=0}^n x_i w_i)$$

The learning is just adjusting the weights $w_i$ automatically to something that makes sense in your context.

Now we can have arbitrary numbers of inputs $x_i$, but to have an arbitrary number of outputs we need more than one neuron. To get our model much more flexible, we stack them to a so called multilayer Perceptron (MLP):

So you can see, the output of one neuron (perceptron) can be the input of another! With backpropagation (a good implementation of gradient descent for this type of model) you can automatically adjust the weights.

What is reinforcement learning?

In machine learning you can distinguish 5 types of problems:

• Regression: Predicting a continuous variable
  • You have a photo of a person and you want to tell how old the person is (e.g. howhot.io)
• Classification: Predicting a variable with finite possible values
  • MNIST: You get a 28px x 28px image. You know it is either 0, or 1, or 2, ..., or 9. So a digit. You have to say which one.
  • ImageNet: You get a bigger image. You have to decide which one of a 1000 classes it is (dog, cat, car, house, ...). You can be sure it is one (and exactly one) of those.
  • http://write-math.com: Classify a handwritten symbol into about 380 classes
  • https://howhot.io/: Classify gender
• Clustering: Grouping data
  • You have a lot of portrait images (or crops of images). You know there is exactly one persons face on it. You don't know how many people there are in total. Now you would like to group it so that each group is one person.
  • Species: You properties of animals and you want to group them. So you want a hierarchical clustering which puts closely related animals (dogs, cats; fly, moscito) closer together than not related ones (dogs, flys)
• Collaborative Filtering: Filling gaps
  • Netflix: You have a lot of movies, a lot of ratings. However, not every person rated every movie. In fact, probably no person did. But you want to fill the gaps so that you can tell the users which movies they should watch because they will probably like it.
  • Amazon: pretty much the same as with Netflix, but for all kinds of products
• Reinforcement learning (RL): Learning with environment
  • Self-driving cars
  • Playing games (e.g. Backgammon, Go, Atari (video))

What makes RL very different from the others is that you typically don't have a lot of data to start with, but you can generate a lot of data by playing. You have to deal with the problem that you have to make decisions, but it is not clear what is good (delayed reward). For example, it might take several moves until you know in Go if a move was smart.

What is different between neural networks and RL?

Neural networks are algorithms, RL is a problem type. You can approach RL with neural networks.

What is the relationship between neural networks and genetic algorithms?

Search for "the five tribes of machine learning", e.g. the image on http://www.welchlabs.com/blog/2016/2/16/whats-next-for-welch-labs

Answer 2

I would suggest a different categorization. Note that it is hard to put those terms into a strict hierarchy. But there are the three large concepts that your terms can be categorized into:

Problem

A mathematical formalization can defines an objective object, also known as loss, error or cost function. The goal is to minimize or maximize the objective which can be done by defining a model and applying an optimization algorithm. Examples:

• Markov decision processes are the problems studied in the field of reinforcement learning.
• Supervised learning where the model output should be close to an existing target or label. Subcategories are classification or regression where the output is a probability distribution or a scalar value, respectively.
• Unsupervised learning is a class of problem settings where no labels are available. One problem in this class is to reconstruct data examples from small representations.
• If you want, supervised learning can be seen as a special form of reinforcement learning with the environment being fully observable, sequences of length one, and the cost function as the reward. Both are problem settings.

Optimization

An algorithm at tweaks parameters of a model in order to minimize or maximize some objective function. The model we know about the model, the more effective algorithm we can use. Examples:

• Some simple models can be solved analytically. For example, one can calculate the optimal parameters of a linear regression model by performing inverting a matrix.
• Some problem settings and models are fully derivable. For example, neural networks can be trained for classification problems by computing the derivatives using the chain rule and applying gradient descent.
• When the problem setting is completely unknown, one can only guess parameters of the model and see if they work better or worse. Evolutionary algorithms are a class of algorithms that aim to do this efficiently.

Model

Models approximate functions and have parameters that can be adjusted. Usually, a model is optimized using an algorithm to better fit observations, or training data. Coming up with powerful models usually is not the problem. Optimizing (training) the models is. Examples:

• Neural network
• Generalized linear model
• Support vector machine
• Stochastic policy