Skip to main content
6 votes
Accepted

What algorithm do SVMs use to minimize their objective function?

The SVM problem (and other related problems) can be described as a minimization \ maximization of a quadratic function. This can be easily solved with the gradient descent algorithm, however I ...
nir shahar's user avatar
  • 11.6k
4 votes
Accepted

Why do we try to maximize Lagrangian in SVMs?

I think the answer to this deals with convex functions and duality. $$L(w, b, \alpha) = \frac{1}{2}\|w\|^2 + \sum_i α_i(y_i(w \bullet x+b)-1).$$ When you minimize this, you are minimizing it over $...
MadhavanRP's user avatar
3 votes

How to tackle different sample size in the training set in SVM

A SVM classifier requires a fixed-length feature vector, i.e., all feature vectors must have the same length. There are multiple solutions: Pad out the strings to fixed length. Choose a different ...
D.W.'s user avatar
  • 162k
2 votes

Overfitting in Machine Learning Algorithms

Yes, they can overfit too. Overfitting is especially a risk when the number of features is much larger than the number of samples in the training set.
D.W.'s user avatar
  • 162k
2 votes
Accepted

How to use PHOG and LPQ features for Emotion Recognition?

To train your SVM inevitably you need some baseline, which can be accquired from FERA 2011 contest. The whole procedure using this data is described in Emotion Recognition Using PHOG and LPQ features ...
Evil's user avatar
  • 9,465
2 votes

Find the exactly correct separating hyperplane of SVM when the data is not perfectly linearly separable

I think you can create a "half-hard" SVM problem. It will be like the hard SVM for positive labels (without the error term) but for negative example it will be the like the soft SVM (with ...
nir shahar's user avatar
  • 11.6k
1 vote

Classifying vectors that only contains 1001110101 numbers - Is Support Vector Machine the solution?

If I understand your question correctly, the features in your case are exclusively categorical. Support Vector Machines might work, but they would not be my model of choice for such data. SVMs try to ...
DirkT's user avatar
  • 991
1 vote

Lagrange Multipliers and Hard Margin SVMs

The Lagrange multipliers $\alpha_i$ are also unknowns. You may not ultimately need the values, but you do need to solve for them. Think of $L$ as a function of the unknown variables: $$L(\theta_0, \...
Pseudonym's user avatar
  • 22.3k
1 vote
Accepted

SVMs - Fat (Margin) Boundary: Why is $\max\frac1{||\theta||}=\min \frac{ ||\theta||^2}{2}$?

Minimizing $\|\theta\|$ is equivalent to minimizing $\|\theta\|^2/2$, in the sense that the minimum is achieved for the same value of $\theta$. Since our goal is primarily to find $\theta$, this ...
D.W.'s user avatar
  • 162k
1 vote

Overfitting in Machine Learning Algorithms

As D.W. points out, in principle every machine learning algorithm can overfit a finite data sample provided you give it enough flexibility and degrees of freedom, e.g., by adding layers or additional ...
Seb Destercke's user avatar
1 vote
Accepted

SVM with a priori information about class probabilities

SVM doesn't take into account that prior knowledge about the distribution of the classes, so if you want a classifier that takes advantage of that, you'll need a different classifier. In particular, ...
D.W.'s user avatar
  • 162k
1 vote

k-means clustered data: how to label newly incoming data

This is not answerable without knowing something about the data itself and where it comes from and what it means. But it sounds like you're trying to do something awfully dubious. It sounds like you'...
D.W.'s user avatar
  • 162k
1 vote
Accepted

Why doesn't this derivation of the margin in a SVM give the correct result?

The first line of your equation for $d$ is incorrect. It should be $$d = \left\|\vec w\frac{\vec x_+\cdot\vec w}{\vec w\cdot\vec w} - \vec w\frac{\vec x_-\cdot\vec w}{\vec w\cdot\vec w}\right\|.$$ (...
D.W.'s user avatar
  • 162k
1 vote

How to find max margin for non-separable SVM?

Map all the points to the new space. Then find the maximum-margin linear separator of those mapped points. Algorithms for finding a maximum-largin linear separator are described in the literature on ...
D.W.'s user avatar
  • 162k
1 vote

Featurizing images of different dimensions

Option 1: You can crop the images to the smallest sizes in all dimensions. However, blindly cropping images will cause you to lose important information, if you don't have a region of interest. For ...
ilke444's user avatar
  • 507

Only top scored, non community-wiki answers of a minimum length are eligible