Hot answers tagged

3

A SVM classifier requires a fixed-length feature vector, i.e., all feature vectors must have the same length. There are multiple solutions: Pad out the strings to fixed length. Choose a different set of features, so that there is a fixed number of features. Pick a fixed number $k$, and look at windows of length $k$ (i.e., substrings of length $k$). ...


2

Yes, they can overfit too. Overfitting is especially a risk when the number of features is much larger than the number of samples in the training set.


2

To train your SVM inevitably you need some baseline, which can be accquired from FERA 2011 contest. The whole procedure using this data is described in Emotion Recognition Using PHOG and LPQ features by Abhinav Dhall, Akshay Asthana, Roland Goecke and Tom Gedeon. After reading that there is a follow up FERA 2015.


2

This is just the explicit form of the feature map for a homogenous, degree 2 polynomial kernel. Here your input space is $\mathbb{R}^2$ and the feature space can be identified with $\mathbb{R}^3$. The point the author is making is that instead of computing the feature map explicitly, and then taking the Euclidean inner product $\langle \Phi(p),\Phi(q)\...


1

SVM doesn't take into account that prior knowledge about the distribution of the classes, so if you want a classifier that takes advantage of that, you'll need a different classifier. In particular, if you know the distribution of each class and the priors (class balance), you can use Bayes rule to classify each point directly, without needing any fancy ...


1

The first line of your equation for $d$ is incorrect. It should be $$d = \left\|\vec w\frac{\vec x_+\cdot\vec w}{\vec w\cdot\vec w} - \vec w\frac{\vec x_-\cdot\vec w}{\vec w\cdot\vec w}\right\|.$$ (Why? The distance between two points $u,v$ is $\|u-v\|$, not $\|u\|+\|v\|$.) If you continue the derivation from there, you will obtain the result you were ...


1

I think the answer to this deals with convex functions and duality. $$L(w, b, \alpha) = \frac{1}{2}\|w\|^2 + \sum_i α_i(y_i(w \bullet x+b)-1).$$ When you minimize this, you are minimizing it over $w,b$. As you stated, this will produce the Lagrangian dual function: $\Theta (\alpha) = inf L(w, b, \alpha) = \sum_iα_i-\frac{1}{2}\sum_{i,j}y_iy_jα_iα_j(x_i\...


1

As D.W. points out, in principle every machine learning algorithm can overfit a finite data sample provided you give it enough flexibility and degrees of freedom, e.g., by adding layers or additional features. However, different methods will be more or less prone to overfitting, and their tendency to overfit is typically studied by theoretical notions such ...


1

I don't see how there should be any negative consequences of this. To see why, you can "vectorize" your feature matrix so that you still have scalars for each "new" feature. That is, instead of a matrix of n features each with k values, you have a vector of length n×k with scalar features. SVMs with a mixture of categorical/integral + real components is ...


1

First of all, the formula given for the distance between a point and a line is not a definition, it is a calculation. The distance between a point $p$ and a line $\ell$ is the minimum distance between $p$ and a point on $\ell$. Similarly, if $\ell,\ell'$ are two lines then the distance between them is the minimum distance between a point on $\ell$ to a ...


1

Option 1: You can crop the images to the smallest sizes in all dimensions. However, blindly cropping images will cause you to lose important information, if you don't have a region of interest. For example, if you are focusing on faces, it is fine to define a ROI and crop around the face. On the other hand, for example if you are doing pure color-wise ...


1

A SVM with a polynomial kernel is a SVM classifier. A kernel perceptron is a perceptron classifier, or in other words, a neural net. A SVM is quite different from a neural net. So, that's one way that they differ. However, Wikipedia says that SVMs are in some respects a generalization of a kernel perceptron, generalized with regularization. ...


Only top voted, non community-wiki answers of a minimum length are eligible