Questions tagged [svm]
Questions about Support Vector Machines. SVMs are supervised learning models used for classification and regression tasks.
39
questions
1
vote
0
answers
32
views
How does a One-Class SVM work?
My teacher explained the function of a one-class SVM to us.
However, the sketch used is not correct from my point of view. I understand the function of a one-class SVM as follows:
The SVM is trained ...
1
vote
0
answers
39
views
How exactly does minimizing the L2 norm of w in SVR affect the model's ability to fit the data?
I have a solid grasp on why we minimize the L2 norm of the weight vector ( w ) in Support Vector Machines (SVM) for classification problems — it maximizes the margin between classes and helps the ...
0
votes
1
answer
29
views
Classifying vectors that only contains 1001110101 numbers - Is Support Vector Machine the solution?
Assume that you are given a vector 0b1101001001011001 etc. And you are going to classify it.
One can use Support Vector Machine, but is that method good for ...
0
votes
0
answers
39
views
Incompresensible about $w$ $x$+ $b$ = 0
I don't actually understand the meaning of $w$$x$ + $b$ = 0
when it is defined in support vector machine.
In my own understanding, in order for the equation to be true, the hyperplane would always ...
1
vote
1
answer
100
views
Lagrange Multipliers and Hard Margin SVMs
With hard margin support vector machines (SVMs), it suffices to find the critical points of the Lagrangian $L = \frac{1}{2}||\theta||^2 - \sum_{n=1}^{N} \alpha_n (y^{(n)} (\vec{\theta}^T\vec{x}^{(n)} +...
0
votes
0
answers
20
views
Radial Basis kernel producing the same decision boundary as a Linear kernel
The following question is from the MIT 6.034 2006 Final Exam paper.
In answering part 6.5, I wasn't certain why the radial basis kernel would produce the same decision boundary as a linear kernel (...
1
vote
0
answers
51
views
Why is quadratic programming used to the solve the support vector machine optimization problem instead of the analytic approach?
(Readers familiar with the mathematical framework of support vector machines may skip to "The problem")
https://youtu.be/eHsErlPJWUU
This is a lecture video by Abu Mostafa on support vector ...
1
vote
1
answer
50
views
Find the exactly correct separating hyperplane of SVM when the data is not perfectly linearly separable
I am thinking about the following case where the data in region 1 is always positive and the data in region 2 is always negative, but the data in region 3 can be both positive and negative. Are there ...
0
votes
1
answer
86
views
SVMs - Fat (Margin) Boundary: Why is $\max\frac1{||\theta||}=\min \frac{ ||\theta||^2}{2}$?
I am trying to understand SVMs in depth watching lectures from MIT.
The professor to reduces the classification problem into an optimization problem. To do that, he first defines the decision and ...
5
votes
1
answer
254
views
What algorithm do SVMs use to minimize their objective function?
Support Vector Machines turn machine learning linear classification tasks into a linear optimization problems.
$$ \text{minimize } J(\theta,\theta_0) = \frac1n \sum_1^n \text{HingeLoss}(\theta,\...
0
votes
0
answers
108
views
why divide norm of w in svm
So I get this through the math behind the SVM (Support Vector Machine), and I get this formula
$$(w^T)(x_1-x_2) = 2.$$
We then divide both side with norm of $w$ then we get the new formula
$$ \frac{w^...
0
votes
0
answers
16
views
Perpendicular Training Vectors SVM
Claim: If we are training a hard-margin SVM on a set of perpendicular training vectors which can either be classified as "positive" or "negative," will every training vector end up ...
0
votes
1
answer
48
views
What is a Black Box attack against Machine Learning algorithms?
And is there an attack strategy that you can use to approximate the architecture of a machine learning system with the knowledge of class labels and some data points?
1
vote
0
answers
37
views
How can a classifier using Laplacian kernel achieve no error on the input samples?
If we have a sample dataset $S = \{(x_1, y_i),\dots,(x_n,y_n)\}$ where $y_i = \{0,1\}$, how can we tune $\sigma$ such that there is no error on $S$ from a classifier using the Laplacian kernel?
...
0
votes
1
answer
75
views
Classification accuracy based on top 3 most likely classifications
My goal is to recommend jobs to job seekers based on their skill set.
Currently I'm using an SVM for this, which is outputting one prediction, e.g. "software engineer at Microsoft". However, consider ...
0
votes
1
answer
48
views
SVM with a priori information about class probabilities
Given are two 2-d sets, each with its own bivariate normal distribution. I need to build an SVM classifier. The a priori probabilities of each class corresponds to the size of its set (20/50 for the ...
0
votes
1
answer
55
views
How to understand a equation related to speaker recognition?
This question refers to the following paper:
Support Vector Machines for Speaker and Language Recognition, W. M. Campbell, J. P. Campbell, D. A. Reynolds, E. Singer, P. A. Torres-Carrasquillo, ...
5
votes
1
answer
87
views
k-means clustered data: how to label newly incoming data
I have a data set with labels that were produced by a $k$-means clustering
algorithm. Now there is some data (with the same data structure) from another
source and I wonder what is the most sensible ...
3
votes
1
answer
40
views
Why doesn't this derivation of the margin in a SVM give the correct result?
I'm trying to derive the optimization objective for an SVM (namely $1/\|w\|$), but I'm running into a little trouble. I've already read this question, which has certainly offered a lot of insight into ...
1
vote
1
answer
379
views
How to find max margin for non-separable SVM?
I am new to Machine Learning. Suppose a training set of positive (square) and negative (circle) points is given like:
Obviously there would be no nice linear separator of positive and negative points....
1
vote
0
answers
48
views
How do we mathematically figure out if a SVM kernel function overfits?
Looking at the kernel function (Gaussian, polynomial. chi-squared, etc) how do we figure out that changing which value will cause overfitting? In my perspective, if we increase (for example) the ...
0
votes
0
answers
197
views
(Percision & recall) Vs (Accuracy)? which one do I have to consider?
I am running several machine learning classifiers to predict something from my data. If I visualized the precision and recall tables as a result, is it enough to get clear idea about the proposed mdel?...
8
votes
3
answers
3k
views
Why do we try to maximize Lagrangian in SVMs?
I was learning about support vector machines from MIT OpenCourseWare. I figured it out. I understand why we try to minimize $\frac{1}{2} w^2$. I just did not get why we try to maximize Lagrange ...
2
votes
1
answer
543
views
How to tackle different sample size in the training set in SVM
I have to train a SVM for a classification problem. I have some strings that are the paths in a deterministic finite automata (DFA). If the alphabet is -01- then possible strings are 011101110 or ...
2
votes
2
answers
155
views
Overfitting in Machine Learning Algorithms
I am new in the ML. I know that overfitting is memorizing the data while training. Like in Neural Network, if we make lots of layers and lots of hidden nodes, we can memorize all the data, but it can ...
1
vote
0
answers
60
views
Machine Learning Algorithm Recommendation For Sensor Data [closed]
I would like to classify data coming from a sensor. In the literature Hidden Markov Model and SVM are used, but I would like to improve results with another methods.The picture how data and classes ...
1
vote
1
answer
401
views
SVM with different length features
I need to train SVM with 2 different features..the problem is one feature is the HOG with length 144 and other an RGB value with length 3.. Can i combine these two features to train SVM and test ...
2
votes
0
answers
57
views
Support Vectors in SVM
It might be a very basic question.$\\$
I am considering the SVM optimization problem here.$\\$
In a training set where the data is linearly separable, and we are using a hard margin (no slack allowed),...
2
votes
1
answer
360
views
How to use PHOG and LPQ features for Emotion Recognition?
I have a database that consists of PHOG and LPQ features for each image. Now, I wish to train an SVM on these features for emotion recognition i.e SVM classifies the images on basis of emotion in the ...
0
votes
2
answers
407
views
Support Vector Machines vs K-Nearest Neighbors
Let's say we have trained a Support Vector Machine with a Gaussian Kernel. When we feed our model an example, it classifies it based on its similarity to landmarks (distance to examples in our ...
1
vote
0
answers
84
views
How to mitigate the hierarchical error propagation in tree-structured classification
Suppose we have a multi-class classification problem, where the number of classes $K \geq 3$
We use a tree structure of multiple SVMs to divide and conquer the problem, with one example in the figure ...
1
vote
0
answers
185
views
How to train SVM in matlab / python for MultiLabel data? [closed]
I am training a problem such that my output (y) could be more than one class. For example, the SVM could say, this input vector is class 1, but it could also say, this input vector is classes 1 AND 5. ...
2
votes
0
answers
260
views
What is the complexity of classification with SVMs?
I'm interested in how fast SVMs can classify new data with $c \in \mathbb{N}_{\geq 2}$ classes and $n \in \mathbb{N}_{\geq 1}$ features.
Example for Neural Networks
For neural networks, this depends ...
1
vote
1
answer
274
views
SVM Maximizing Margin
I am trying to make sense of the following:
Taken from the MIT website.
So I understand that we want to maximize the distance between the planes H0 and H1, where H0 is defined as ...
0
votes
1
answer
175
views
Featurizing images of different dimensions
I'm building a non linear svm for images to solve a classification problem with domain {0, 1} and I'm currently doing featurization. What I want to do is create 3 features for each pixel representing ...
3
votes
1
answer
589
views
Kernel Perceptron vs Polynomial Perceptron
I was looking at Support Vector machines (SVM) kernels. Looking at Polynomial Kernel and Kernel Perceptron I was curious how they differ?
Work Done
Polynomial Kernel:
$d_{k+1}(x)=d_{k}(\bar{x})+\...
1
vote
0
answers
180
views
Computer vision training procedures: SVM/AdaBoost vs Neural Networks
With SVM, adaboost or similar alogrithms, image training sets must be cropped with specific constraints (keep image cropping ratio the same, have object tightly cropped, same resolution)
In general, ...
0
votes
1
answer
1k
views
Transformation from one feature space to another
I have found the following example:
As an example consider the case when the input space $ {\mathcal{X}}$
consists of images of $ 16\times 16$ pixels, i.e. $ 256$ dimensional
vectors, and we ...
2
votes
1
answer
195
views
Feature values range
Suppose I am about to use SVM for learning a classification or ranking function. Suppose that my feature vectors are two dimensional and that values for one dimension are, say, natural numbers and the ...