What does non-linearity mean in Neural Networks? Why is it necessary?

ReLU units are said to be necessary in CNNs to introduce non-linearity which convolution does not involve. This is needed, because many real-world forms of data are non-linear.

What does non-linearity refer to?

A neuron $$x$$ on layer $$i+1$$ is connected to several neurons $$y_1,\ldots,y_d$$ on layer $$i$$, and it computes the following function: $$x = f(\alpha_1 y_1 + \cdots + \alpha_d y_d + \beta),$$ where $$f$$ is a fixed function (nowadays often ReLU), and $$\alpha_1,\ldots,\alpha_d,\beta$$ are parameters which are learned in the training period. (The output layer often uses a different $$f$$.)
If $$f$$ is the identity function $$f(z) = z$$, then each neuron in the output is just an affine function of the input. This means that no matter how many layers the neural network has, all it can do is compute linear functions.
In order to allow the network to compute more complicated functions, we introduce nonlinearity, in the form of a non-linear function $$f$$ (that is, a function which is not of the form $$f(z) = az + b$$), such as ReLU, which is $$f(x) = \max(x,0)$$. This makes the neural network more expressive.