# Tag Info

6

The SVM problem (and other related problems) can be described as a minimization \ maximization of a quadratic function. This can be easily solved with the gradient descent algorithm, however I recommend using the SMO algorithm since it is a direct solution (to the dual of the SVM problem), and can be also used for kernelized SVMs

3

A SVM classifier requires a fixed-length feature vector, i.e., all feature vectors must have the same length. There are multiple solutions: Pad out the strings to fixed length. Choose a different set of features, so that there is a fixed number of features. Pick a fixed number $k$, and look at windows of length $k$ (i.e., substrings of length $k$). ...

2

Yes, they can overfit too. Overfitting is especially a risk when the number of features is much larger than the number of samples in the training set.

2

To train your SVM inevitably you need some baseline, which can be accquired from FERA 2011 contest. The whole procedure using this data is described in Emotion Recognition Using PHOG and LPQ features by Abhinav Dhall, Akshay Asthana, Roland Goecke and Tom Gedeon. After reading that there is a follow up FERA 2015.

2

This is just the explicit form of the feature map for a homogenous, degree 2 polynomial kernel. Here your input space is $\mathbb{R}^2$ and the feature space can be identified with $\mathbb{R}^3$. The point the author is making is that instead of computing the feature map explicitly, and then taking the Euclidean inner product $\langle \Phi(p),\Phi(q)\... 2 I think the answer to this deals with convex functions and duality. $$L(w, b, \alpha) = \frac{1}{2}\|w\|^2 + \sum_i α_i(y_i(w \bullet x+b)-1).$$ When you minimize this, you are minimizing it over$w,b$. As you stated, this will produce the Lagrangian dual function:$\Theta (\alpha) = inf L(w, b, \alpha) = \sum_iα_i-\frac{1}{2}\sum_{i,j}y_iy_jα_iα_j(x_i\...

2

I think you can create a "half-hard" SVM problem. It will be like the hard SVM for positive labels (without the error term) but for negative example it will be the like the soft SVM (with the error term). This wil guarantee you get the line that labels correctly all positive examples, but still tries to label as many of the negative examples as it ...

1

Minimizing $\|\theta\|$ is equivalent to minimizing $\|\theta\|^2/2$, in the sense that the minimum is achieved for the same value of $\theta$. Since our goal is primarily to find $\theta$, this substitution is not unreasonable. Why was the substitution done? I don't know. Perhaps because it makes the resulting optimization easier; or just because it ...

1

SVM doesn't take into account that prior knowledge about the distribution of the classes, so if you want a classifier that takes advantage of that, you'll need a different classifier. In particular, if you know the distribution of each class and the priors (class balance), you can use Bayes rule to classify each point directly, without needing any fancy ...

1

This is not answerable without knowing something about the data itself and where it comes from and what it means. But it sounds like you're trying to do something awfully dubious. It sounds like you're trying to come up with labels for free, with a purely automated method. That usually doesn't work. There's no free lunch. You'll need to obtain labels ...

1

The first line of your equation for $d$ is incorrect. It should be $$d = \left\|\vec w\frac{\vec x_+\cdot\vec w}{\vec w\cdot\vec w} - \vec w\frac{\vec x_-\cdot\vec w}{\vec w\cdot\vec w}\right\|.$$ (Why? The distance between two points $u,v$ is $\|u-v\|$, not $\|u\|+\|v\|$.) If you continue the derivation from there, you will obtain the result you were ...

1

Map all the points to the new space. Then find the maximum-margin linear separator of those mapped points. Algorithms for finding a maximum-largin linear separator are described in the literature on SVMs (that's exactly what you need to do to train a linear SVM). In other words, you can map the points to the new space, then can take those mapped points ...

1

As D.W. points out, in principle every machine learning algorithm can overfit a finite data sample provided you give it enough flexibility and degrees of freedom, e.g., by adding layers or additional features. However, different methods will be more or less prone to overfitting, and their tendency to overfit is typically studied by theoretical notions such ...

1

I don't see how there should be any negative consequences of this. To see why, you can "vectorize" your feature matrix so that you still have scalars for each "new" feature. That is, instead of a matrix of n features each with k values, you have a vector of length n×k with scalar features. SVMs with a mixture of categorical/integral + real components is ...

1

First of all, the formula given for the distance between a point and a line is not a definition, it is a calculation. The distance between a point $p$ and a line $\ell$ is the minimum distance between $p$ and a point on $\ell$. Similarly, if $\ell,\ell'$ are two lines then the distance between them is the minimum distance between a point on $\ell$ to a ...

1

Option 1: You can crop the images to the smallest sizes in all dimensions. However, blindly cropping images will cause you to lose important information, if you don't have a region of interest. For example, if you are focusing on faces, it is fine to define a ROI and crop around the face. On the other hand, for example if you are doing pure color-wise ...

1

A SVM with a polynomial kernel is a SVM classifier. A kernel perceptron is a perceptron classifier, or in other words, a neural net. A SVM is quite different from a neural net. So, that's one way that they differ. However, Wikipedia says that SVMs are in some respects a generalization of a kernel perceptron, generalized with regularization. ...

Only top voted, non community-wiki answers of a minimum length are eligible