I've been learning about neural networks and SVMs. The tutorials I've read have emphasized how important kernelization is, for SVMs. Without a kernel function, SVMs are just a linear classifier. With kernelization, SVMs can also incorporate non-linear features, which makes them a more powerful classifier.
It looks to me like one could also apply kernelization to neural networks, but none of the tutorials on neural networks I've seen have mentioned this. Do people commonly use the kernel trick with neural networks? I presume someone must have experimented with it to see if it makes a big difference. Does kernelization help neural networks as much as it helps SVMs? Why or why not?
(I can imagine several ways to incorporate the kernel trick into neural networks. One way would be to use a suitable kernel function to preprocess the input, a vector in $\mathbb{R}^n$, into a higher-dimensional input, a vector in $\mathbb{R}^{m}$ for $m\ge n$. For multiple-layer neural nets, another alternative would be to apply a kernel function at each level of the neural network.)