3
$\begingroup$

I was looking at Support Vector machines (SVM) kernels. Looking at Polynomial Kernel and Kernel Perceptron I was curious how they differ?

Work Done

Polynomial Kernel:

$d_{k+1}(x)=d_{k}(\bar{x})+\rho k(\bar{x}_{k}, \bar{x})\; \mbox{if}\; \bar{x}_k\epsilon\,C_1$

$d_{k+1}(x)=d_{k}(\bar{x})-\rho k(\bar{x}_{k}, \bar{x})\; \mbox{if}\; \bar{x}_k\epsilon\,C_2$

where $k(\bar{x}_{k}, \bar{x}) = (\bar{x}\cdot \bar{x}_{k}+1)^{q}$

Kernel Perceptron:

This is given by

$g(x)=\sum_{j=1}^{N}a_{j} K(\bar{x},\bar{x}_j)$

So as per my understanding a bias constant is added in former case when compared to later. So how does that impact and what difference does it make? Or am I missing something?

Any insights are appreciated.

$\endgroup$
1
$\begingroup$

A SVM with a polynomial kernel is a SVM classifier.

A kernel perceptron is a perceptron classifier, or in other words, a neural net.

A SVM is quite different from a neural net. So, that's one way that they differ.

However, Wikipedia says that SVMs are in some respects a generalization of a kernel perceptron, generalized with regularization. Regularization is basically a form of Occam's razor: it says that, all else being equal, simpler models are preferred over more complex models. Regularization helps avoid overfitting and thus is very useful in practice.

(Related: See also Support Vector Machines as Neural Nets? -- a SVM with a linear kernel is similar to a single-layer perceptron classifier, in case that's what you were thinking of.)

$\endgroup$
  • $\begingroup$ I am not sure what a Neural Net is... I am more interested in the bias constant that is added in Polynomial Kernel and how is it good/bad when compared to Kernel Perceptron (where there is no bias constant) $\endgroup$ – priyanka Mar 31 '15 at 5:06
  • $\begingroup$ @priyanka, if you don't know what a neural net is, this is a great time to learn! P.S. On this site it's important to make sure that you think through exactly what question you want answered and make sure to ask the question that you want answered. People tend not to like "chameleon questions" that change after they're answered. I suggest you edit the question to be a bit more precise about what you know and what specifically your question is. $\endgroup$ – D.W. Mar 31 '15 at 6:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.