We’re rewarding the question askers & reputations are being recalculated! Read more.

Questions tagged [machine-learning]

Questions about computer algorithms that automatically discover patterns in data and make good decisions based on them.

Filter by
Sorted by
Tagged with
89
votes
7answers
17k views

Why is deep learning hyped despite bad VC dimension?

The Vapnik–Chervonenkis (VC)-dimension formula for neural networks ranges from $O(E)$ to $O(E^2)$, with $O(E^2V^2)$ in the worst case, where $E$ is the number of edges and $V$ is the number of nodes. ...
45
votes
5answers
13k views

Why has research on genetic algorithms slowed?

While discussing some intro level topics today, including the use of genetic algorithms; I was told that research has really slowed in this field. The reason given was that most people are focusing on ...
33
votes
2answers
2k views

Are there improvements on Dana Angluin's algorithm for learning regular sets

In her 1987 seminal paper Dana Angluin presents a polynomial time algorithm for learning a DFA from membership queries and theory queries (counterexamples to a proposed DFA). She shows that if you ...
29
votes
4answers
5k views

How to determine likely connections in a social network?

I am curious in determining an approach to tackling a "suggested friends" algorithm. Facebook has a feature in which it will recommended individuals to you which it thinks you may be acquainted with. ...
29
votes
2answers
653 views

Why do neural networks seem to perform better with restrictions placed on their topology?

Fully connected (at least layer to layer with more than 2 hidden layers) backprop networks are universal learners. Unfortunately, they are often slow to learn and tend to over-fit or have awkward ...
28
votes
4answers
69k views

What exactly is the difference between supervised and unsupervised learning?

I am trying to understand clustering methods. What I I think I understood: In supervised learning, the categories/labels data is assigned to are known before computation. So, the labels, classes or ...
28
votes
1answer
28k views

Which machine learning algorithms can be used for time series forecasts?

Currently I am playing around with time series forecasts (specifically for Forex). I have seen some scientific papers about echo state networks which are applied to Forex forecast. Are there other ...
27
votes
12answers
9k views

Why is overfitting bad?

I've studied this lots, and they say overfitting the actions in machine learning is bad, yet our neurons do become very strong and find the best actions/senses that we go by or avoid, plus can be de-...
24
votes
1answer
13k views

What is Temperature in LSTM (and neural networks generally)?

One of the hyperparameters for LSTM networks is temperature. What is it?
23
votes
4answers
13k views

What is the difference between a Neural Network, a Deep Learning System and a Deep Belief Network?

What is the difference between a Neural Network, a Deep Learning System and a Deep Belief Network? As I recall your basic neural network is a 3 layers kinda thing, and I have had Deep Belief Systems ...
23
votes
1answer
626 views

Clustering of Songs (The Joe Walsh Problem)

The Eagles are a rock supergroup from the 70s and 80s, responsible for such classics as Hotel California. They have two quite distinctive sounds, one where guitarist Joe Walsh is present (for example, ...
22
votes
2answers
2k views

What combination of data structures efficiently stores discrete Bayesian networks?

I understand the theory behind Bayesian networks, and am wondering what it takes to build one in practice. Let's say for this example, that I have a Bayesian (directed) network of 100 discrete random ...
20
votes
2answers
657 views

Why are diploid (dominant/recessive) genes not used widely in genetic algorithms?

In most implementations of genetic algorithms, the focus is on crossover and mutation. But somehow, most of them leave out diploid (dominant/recessive) nature of genes. As far as my (limited) ...
19
votes
3answers
6k views

Line separates two sets of points

If there is a way to identify if two sets of points can be separated by a line? We have two sets of points $A$ and $B$ if there is a line that separates $A$ and $B$ such that all points of $A$ and ...
19
votes
1answer
3k views

Who coined the term “machine learning”?

I'm trying to figure out who coined the term "machine learning". An ancillary question is from where is Arthur Samuel cited as defining the field of "machine learning" in 1959 as: the field of ...
18
votes
1answer
773 views

Efficiently computing or approximating the VC-dimension of a neural network

My goal is to solve the following problem, which I have described by its input and output: Input: A directed acyclic graph $G$ with $m$ nodes, $n$ sources, and $1$ sink ($m > n \geq 1$). Output: ...
16
votes
2answers
9k views

Must Neural Networks always converge?

Introduction Step One I wrote a standard backpropegating neural network, and to test it, I decided to have it map XOR. It is a 2-2-1 network (with tanh activation function) ...
16
votes
1answer
252 views

Can a perceptron forget?

I would like to build an online web-based machine learning system, where users can continuously add classified samples, and have the model updated online. I would like to use a perceptron or a similar ...
16
votes
1answer
114 views

Methods to evaluate a system of written rules

I was trying to come up with a system that would evaluate bylaws for an organization as to determine their underlying logic. I think a first-order predicate system would work for representing the ...
14
votes
4answers
3k views

How to devise an algorithm that suggests feasible cooking recipes?

I once had a veteran in my course that created an algorithm that would suggest cooking recipes. At first, all sort of crazy recipes would come out. Then, she would train the cooking algorithm with ...
14
votes
2answers
7k views

Machine learning algorithm to play Connect Four

I'm currently reading about machine learning and wondered how to apply it to playing Connect Four. My current attempt is a simple multiclass classificator using a sigmoid function model and the one-...
13
votes
4answers
2k views

What is the relation between correlation and causation in machine learning?

It is a well-known fact that "Correlation doesn't equal causation", but machine learning seems to be almost entirely based on correlation. I'm working on a system to estimate the performance of ...
13
votes
1answer
14k views

Smoothing in Naive Bayes model

A Naive Bayes predictor makes its predictions using this formula: $$P(Y=y|X=x) = \alpha P(Y=y)\prod_i P(X_i=x_i|Y=y)$$ where $\alpha$ is a normalizing factor. This requires estimating the parameters ...
12
votes
2answers
10k views

How to encode date as input in neural network?

I am using neural networks to predict a time series. The question I'm facing now is how do I encode date/time/serial no. of each input set as an input to the neural network? Should I use 1 of C ...
12
votes
4answers
5k views

Machine Learning vs System Identification?

Could anyone explain to me the differences & similarities between machine learning and system identifications? Are these just two names of the same thing? In this page, they say: Machine ...
11
votes
2answers
263 views

A text-classifier that explains its decisions

I am building a text categorizer for short sentences. In addition to telling the user "the category of the text you entered is C", I want to be able to explain why I made this decision, in a short ...
11
votes
2answers
184 views

Non-Parametric Methods Like K-Nearest-Neighbours in High Dimensional Feature Space

The main idea of k-Nearest-Neighbour takes into account the $k$ nearest points and decides the classification of the data by majority vote. If so, then it should not have problems in higher ...
11
votes
1answer
217 views

Google DeepDream Elaborated

I've seen a few questions on this site about Deep Dream, however none of them seem to actually speak as to what DeepDream is doing, specifically. As far as I've gathered, they seem to have changed the ...
10
votes
4answers
5k views

Evolving artificial neural networks for solving NP problems

I've recently read a really interesting blog entry from Google Research Blog talking about neural network. Basically they use this neural networks for solving various problems like image recognition. ...
10
votes
2answers
1k views

Smallest DFA that accepts given strings and rejects other given strings

Given two sets $A,B$ of strings over alphabet $\Sigma$, can we compute the smallest deterministic finite-state automaton (DFA) $M$ such that $A \subseteq L(M)$ and $L(M) \subseteq \Sigma^*\setminus B$?...
10
votes
1answer
582 views

Implementation of Naive Bayes

I am implementing a Naive Bayes algorithm for text categorization with Laplacian smoothing. The problem I am having is that the probability approaches zero because I am multiplying many small ...
10
votes
1answer
6k views

What is meant by the term “prior” in machine learning

I am new to machine learning. I have read several papers where they have employed deep learning for various applications and have used the term "prior" in most of the model design cases, say prior in ...
10
votes
2answers
326 views

How do I classify my emulator input optimization problem, and with which algorithm should I approach it?

Due to the nature of the question, I have to include lots of background information (because my question is: how do I narrow this down?) That said, it can be summarized (to the best of my knowledge) ...
10
votes
1answer
605 views

Is genetic programming relevant today?

My main concern is whether the genetic programming is an active field of research, with some promising applications in practice. It seems like in field of machine learning, the neural networks are the ...
10
votes
2answers
204 views

Which classifier is more accurate for a SVM classification?

I am learning the SVM classification and encounter a problem. I am not sure if this dilemma has a terminology for it. Assume we would like to classify patient by SVM given the samples of healthy ...
10
votes
0answers
1k views

Alternatives to SVD for rank factorization

I have rank-deficient matrix $M \in \mathbb{R}^{n\times m}$ with $\text{rank}(M) = k$ and I want to find a rank factorization $M = PQ$ with $P \in \mathbb{R}^{n \times k}$ and $Q \in \mathbb{R}^{k \...
9
votes
2answers
262 views

What was going on before PAC learning

I am investigating PAC learning (computational learning theory) as a beginner with no previous knowledge of machine learning / AI. I am investigating the model mainly from a historical point of view. ...
9
votes
1answer
23k views

How does the momentum term for backpropagation algorithm work?

When updating the weights of a neural network using the backpropagation algorithm with a momentum term, should the learning rate be applied to the momentum term as well? Most of the information I ...
9
votes
1answer
153 views

Is there a general algorithm to fill holes in terms of the Calculus of Constructions?

Suppose that you extend the Calculus of Constructions with "holes" - i.e., incomplete pieces of code that you didn't fill yet. I wonder if there is an algorithm to fill those roles automatically. For ...
9
votes
3answers
3k views

Kernelization trick, for neural networks

I've been learning about neural networks and SVMs. The tutorials I've read have emphasized how important kernelization is, for SVMs. Without a kernel function, SVMs are just a linear classifier. ...
9
votes
1answer
602 views

Machine Learning algorithms based on “structural risk minimization”?

Which machine learning algorithms (besides SVM's) use the principle of structural risk minimization?
9
votes
1answer
906 views

Conflict Driven Clause Learning backtracking clarification

On the wikipedia page here it describes pretty well the CDCL algorithm (and it seems the pictures were taken from slides created by Sharad Malik at Princeton). However when describing how to backtrack ...
9
votes
1answer
2k views

Why are weights of Neural Networks initialized with random numbers?

Why are neural networks initial weights initialized as random numbers? I had read somewhere that this is done to "break the symmetry" and this makes the neural network learn faster. How does breaking ...
9
votes
1answer
293 views

Pseudo-random sequence prediction

Disclaimer: I am a biologist, so sorry for (perhaps) basic question phrased in such crude terms. I am not sure if I should ask this question here or on DS/SC, but CS is the largest of three, so ...
9
votes
1answer
704 views

Intuitive description for training of LSTM (with forget gate/peephole)?

I am a CS undergraduate (but I don't know much about AI though, did not take any courses on it, and definitely nothing about NN until recently) who is about to do a school project in AI, so I pick a ...
9
votes
1answer
187 views

Fixed-length decision-tree-like feature selection to minimize average search performance

I have a complex query $Q$ used to search a dataset $S$ to find $H_\text{exact} = \{s \in S \mid \text{where $Q(s)$ is True}\}$. Each query takes on average time $t$ so the overall time in the linear ...
8
votes
2answers
3k views

What can be learned from the weights in a neural network?

I'm very new to neural networks, and have been trying to figure some things out. So, let's say you come across a neural network which has 100 inputs, a hidden layer with 200 nodes, and 32 outputs. ...
8
votes
1answer
475 views

Vapnik-Chervonenkis Dimension: why cannot four points on a line be shattered by rectangles?

So I'm reading "Introduction to Machine Learning" 2nd edition, by Bishop, et. all. On page 27 they discuss the Vapnik-Chervonenkis Dimension which is, "The maximum number of points that can be ...
8
votes
1answer
2k views

Should activation function be monotonic in neural networks?

A lot of activation functions in neural networks (sigmoid, tanh, softmax) are monotonic, continuous and differentiable (except of may be a couple of points, where derivative does not exist). I ...
8
votes
1answer
5k views

Neural network diverging instead of converging

I have implemented a neural network (using CUDA) with 2 layers. (2 Neurons per layer). I'm trying to make it learn 2 simple quadratic polynomial functions using backpropagation. But instead of ...