For simplicity lets consider a simple single hidden layer feed forward neural net for binary prediction. At test time the neural network predicts
p(Y = 1 \mid X = x) = \sigma(w \cdot \varphi(Ax)),
where $w$ is the vector of hidden to output connections, $A$ is the matrix of input to hidden connections, $\sigma$ is the logistic sigmoid function, $\varphi$ is an element wise function (normally something like a sigmoid, tanh, rectified linear, etc), and I've ignored biases for simplicity.
Now skim through the section Nonlinear Classification in the SVM Wikipedia article. Notice how essentially we have the same thing above, but with everything happening in the neural network before the final dot product serving as the transform function $\varphi$, and that's it. The advantage of neural networks is they allow you to easily learn this transform, which can be very beneficial.