Questions tagged [machine-learning]

Questions about computer algorithms that automatically discover patterns in data and make good decisions based on them.

Filter by
Sorted by
Tagged with
103 votes
7 answers
23k views

Why is deep learning hyped despite bad VC dimension?

The Vapnik–Chervonenkis (VC)-dimension formula for neural networks ranges from $O(E)$ to $O(E^2)$, with $O(E^2V^2)$ in the worst case, where $E$ is the number of edges and $V$ is the number of nodes. ...
yters's user avatar
  • 1,409
46 votes
5 answers
14k views

Why has research on genetic algorithms slowed?

While discussing some intro level topics today, including the use of genetic algorithms; I was told that research has really slowed in this field. The reason given was that most people are focusing on ...
FossilizedCarlos's user avatar
35 votes
1 answer
30k views

What is Temperature in LSTM (and neural networks generally)?

One of the hyperparameters for LSTM networks is temperature. What is it?
Justin Shenk's user avatar
  • 1,025
35 votes
2 answers
3k views

Are there improvements on Dana Angluin's algorithm for learning regular sets

In her 1987 seminal paper Dana Angluin presents a polynomial time algorithm for learning a DFA from membership queries and theory queries (counterexamples to a proposed DFA). She shows that if you are ...
Artem Kaznatcheev's user avatar
30 votes
12 answers
10k views

Why is overfitting bad?

I've studied this lots, and they say overfitting the actions in machine learning is bad, yet our neurons do become very strong and find the best actions/senses that we go by or avoid, plus can be de-...
Friendly Person 44's user avatar
30 votes
4 answers
9k views

Why can't we mimic a dog's ability to smell COVID?

As far as I can tell, we have invented tools and algorithm to: Detect a wider range of colors at a larger range than humans or any other animals on the planet Detect sound with wavelengths ...
jonjbar's user avatar
  • 418
30 votes
4 answers
71k views

What exactly is the difference between supervised and unsupervised learning?

I am trying to understand clustering methods. What I I think I understood: In supervised learning, the categories/labels data is assigned to are known before computation. So, the labels, classes or ...
Prot's user avatar
  • 403
30 votes
2 answers
733 views

Why do neural networks seem to perform better with restrictions placed on their topology?

Fully connected (at least layer to layer with more than 2 hidden layers) backprop networks are universal learners. Unfortunately, they are often slow to learn and tend to over-fit or have awkward ...
Artem Kaznatcheev's user avatar
29 votes
4 answers
5k views

How to determine likely connections in a social network?

I am curious in determining an approach to tackling a "suggested friends" algorithm. Facebook has a feature in which it will recommended individuals to you which it thinks you may be acquainted with. ...
phwd's user avatar
  • 621
29 votes
1 answer
29k views

Which machine learning algorithms can be used for time series forecasts?

Currently I am playing around with time series forecasts (specifically for Forex). I have seen some scientific papers about echo state networks which are applied to Forex forecast. Are there other ...
Maecky's user avatar
  • 393
23 votes
4 answers
14k views

What is the difference between a Neural Network, a Deep Learning System and a Deep Belief Network?

What is the difference between a Neural Network, a Deep Learning System and a Deep Belief Network? As I recall your basic neural network is a 3 layers kinda thing, and I have had Deep Belief Systems ...
Frames Catherine White's user avatar
23 votes
1 answer
658 views

Clustering of Songs (The Joe Walsh Problem)

The Eagles are a rock supergroup from the 70s and 80s, responsible for such classics as Hotel California. They have two quite distinctive sounds, one where guitarist Joe Walsh is present (for example, ...
Dave Clarke's user avatar
  • 20.2k
23 votes
2 answers
3k views

What combination of data structures efficiently stores discrete Bayesian networks?

I understand the theory behind Bayesian networks, and am wondering what it takes to build one in practice. Let's say for this example, that I have a Bayesian (directed) network of 100 discrete random ...
rxmnnxfpvg's user avatar
21 votes
2 answers
803 views

Why are diploid (dominant/recessive) genes not used widely in genetic algorithms?

In most implementations of genetic algorithms, the focus is on crossover and mutation. But somehow, most of them leave out diploid (dominant/recessive) nature of genes. As far as my (limited) ...
Shayan RC's user avatar
  • 633
19 votes
3 answers
10k views

Line separates two sets of points

If there is a way to identify if two sets of points can be separated by a line? We have two sets of points $A$ and $B$ if there is a line that separates $A$ and $B$ such that all points of $A$ and ...
com's user avatar
  • 3,169
19 votes
1 answer
1k views

Efficiently computing or approximating the VC-dimension of a neural network

My goal is to solve the following problem, which I have described by its input and output: Input: A directed acyclic graph $G$ with $m$ nodes, $n$ sources, and $1$ sink ($m > n \geq 1$). Output: ...
Artem Kaznatcheev's user avatar
19 votes
1 answer
3k views

Who coined the term "machine learning"?

I'm trying to figure out who coined the term "machine learning". An ancillary question is from where is Arthur Samuel cited as defining the field of "machine learning" in 1959 as: the field of ...
robguinness's user avatar
18 votes
3 answers
21k views

What is meant by the term "prior" in machine learning

I am new to machine learning. I have read several papers where they have employed deep learning for various applications and have used the term "prior" in most of the model design cases, say prior in ...
Amy's user avatar
  • 183
16 votes
2 answers
10k views

Must Neural Networks always converge?

Introduction Step One I wrote a standard backpropegating neural network, and to test it, I decided to have it map XOR. It is a 2-2-1 network (with tanh activation function) ...
user avatar
16 votes
1 answer
313 views

Can a perceptron forget?

I would like to build an online web-based machine learning system, where users can continuously add classified samples, and have the model updated online. I would like to use a perceptron or a similar ...
Erel Segal-Halevi's user avatar
16 votes
1 answer
141 views

Methods to evaluate a system of written rules

I was trying to come up with a system that would evaluate bylaws for an organization as to determine their underlying logic. I think a first-order predicate system would work for representing the ...
jonsca's user avatar
  • 561
15 votes
4 answers
4k views

How to devise an algorithm that suggests feasible cooking recipes?

I once had a veteran in my course that created an algorithm that would suggest cooking recipes. At first, all sort of crazy recipes would come out. Then, she would train the cooking algorithm with ...
Oeufcoque Penteano's user avatar
14 votes
2 answers
9k views

Machine learning algorithm to play Connect Four

I'm currently reading about machine learning and wondered how to apply it to playing Connect Four. My current attempt is a simple multiclass classificator using a sigmoid function model and the one-...
Tom's user avatar
  • 141
13 votes
4 answers
2k views

What is the relation between correlation and causation in machine learning?

It is a well-known fact that "Correlation doesn't equal causation", but machine learning seems to be almost entirely based on correlation. I'm working on a system to estimate the performance of ...
Casebash's user avatar
  • 313
13 votes
2 answers
15k views

How to encode date as input in neural network?

I am using neural networks to predict a time series. The question I'm facing now is how do I encode date/time/serial no. of each input set as an input to the neural network? Should I use 1 of C ...
Shayan RC's user avatar
  • 633
13 votes
5 answers
9k views

Machine Learning vs System Identification?

Could anyone explain to me the differences & similarities between machine learning and system identifications? Are these just two names of the same thing? In this page, they say: Machine ...
CherryQu's user avatar
  • 231
13 votes
1 answer
16k views

Smoothing in Naive Bayes model

A Naive Bayes predictor makes its predictions using this formula: $$P(Y=y|X=x) = \alpha P(Y=y)\prod_i P(X_i=x_i|Y=y)$$ where $\alpha$ is a normalizing factor. This requires estimating the parameters ...
Chris Taylor's user avatar
13 votes
1 answer
2k views

Alternatives to SVD for rank factorization

I have rank-deficient matrix $M \in \mathbb{R}^{n\times m}$ with $\text{rank}(M) = k$ and I want to find a rank factorization $M = PQ$ with $P \in \mathbb{R}^{n \times k}$ and $Q \in \mathbb{R}^{k \...
Artem Kaznatcheev's user avatar
12 votes
2 answers
2k views

Smallest DFA that accepts given strings and rejects other given strings

Given two sets $A,B$ of strings over alphabet $\Sigma$, can we compute the smallest deterministic finite-state automaton (DFA) $M$ such that $A \subseteq L(M)$ and $L(M) \subseteq \Sigma^*\setminus B$?...
D.W.'s user avatar
  • 158k
12 votes
1 answer
393 views

Google DeepDream Elaborated

I've seen a few questions on this site about Deep Dream, however none of them seem to actually speak as to what DeepDream is doing, specifically. As far as I've gathered, they seem to have changed the ...
Bob's user avatar
  • 121
11 votes
1 answer
32k views

How does the momentum term for backpropagation algorithm work?

When updating the weights of a neural network using the backpropagation algorithm with a momentum term, should the learning rate be applied to the momentum term as well? Most of the information I ...
guskenny83's user avatar
11 votes
2 answers
316 views

A text-classifier that explains its decisions

I am building a text categorizer for short sentences. In addition to telling the user "the category of the text you entered is C", I want to be able to explain why I made this decision, in a short ...
Erel Segal-Halevi's user avatar
11 votes
2 answers
224 views

Non-Parametric Methods Like K-Nearest-Neighbours in High Dimensional Feature Space

The main idea of k-Nearest-Neighbour takes into account the $k$ nearest points and decides the classification of the data by majority vote. If so, then it should not have problems in higher ...
Strin's user avatar
  • 1,505
10 votes
6 answers
6k views

Evolving artificial neural networks for solving NP problems

I've recently read a really interesting blog entry from Google Research Blog talking about neural network. Basically they use this neural networks for solving various problems like image recognition. ...
nmomn's user avatar
  • 377
10 votes
1 answer
655 views

Implementation of Naive Bayes

I am implementing a Naive Bayes algorithm for text categorization with Laplacian smoothing. The problem I am having is that the probability approaches zero because I am multiplying many small ...
sam's user avatar
  • 101
10 votes
3 answers
889 views

Resources for studying the mathematical foundations of machine learning, for someone from a math/physics background

I am a soon-to-be physics graduate student with a background in theoretical and experimental cosmology. In my work, I've often found myself applying machine learning models and techniques for the ...
10GeV's user avatar
  • 201
10 votes
2 answers
4k views

Should activation function be monotonic in neural networks?

A lot of activation functions in neural networks (sigmoid, tanh, softmax) are monotonic, continuous and differentiable (except of may be a couple of points, where derivative does not exist). I ...
Salvador Dali's user avatar
10 votes
1 answer
682 views

Machine Learning algorithms based on "structural risk minimization"?

Which machine learning algorithms (besides SVM's) use the principle of structural risk minimization?
Classifire's user avatar
10 votes
2 answers
383 views

How do I classify my emulator input optimization problem, and with which algorithm should I approach it?

Due to the nature of the question, I have to include lots of background information (because my question is: how do I narrow this down?) That said, it can be summarized (to the best of my knowledge) ...
GManNickG's user avatar
  • 269
10 votes
1 answer
991 views

Is genetic programming relevant today?

My main concern is whether the genetic programming is an active field of research, with some promising applications in practice. It seems like in field of machine learning, the neural networks are the ...
Jaroslav Loebl's user avatar
10 votes
2 answers
276 views

Which classifier is more accurate for a SVM classification?

I am learning the SVM classification and encounter a problem. I am not sure if this dilemma has a terminology for it. Assume we would like to classify patient by SVM given the samples of healthy ...
Cassie's user avatar
  • 305
10 votes
1 answer
375 views

Pseudo-random sequence prediction

Disclaimer: I am a biologist, so sorry for (perhaps) basic question phrased in such crude terms. I am not sure if I should ask this question here or on DS/SC, but CS is the largest of three, so here ...
Sergey Antopolskiy's user avatar
9 votes
2 answers
6k views

Typical NP-complete/hard problems in machine learning

I know little about machine Learning, but I work on optimization (solving NP-hard problems with SAT solvers or MIP). Examples of this would be solving TSP, Steiner tree problems, path finding with ...
excalibur1491's user avatar
9 votes
2 answers
335 views

What was going on before PAC learning

I am investigating PAC learning (computational learning theory) as a beginner with no previous knowledge of machine learning / AI. I am investigating the model mainly from a historical point of view. ...
codd's user avatar
  • 701
9 votes
1 answer
204 views

Is there a general algorithm to fill holes in terms of the Calculus of Constructions?

Suppose that you extend the Calculus of Constructions with "holes" - i.e., incomplete pieces of code that you didn't fill yet. I wonder if there is an algorithm to fill those roles automatically. For ...
MaiaVictor's user avatar
  • 4,127
9 votes
3 answers
4k views

Kernelization trick, for neural networks

I've been learning about neural networks and SVMs. The tutorials I've read have emphasized how important kernelization is, for SVMs. Without a kernel function, SVMs are just a linear classifier. ...
D.W.'s user avatar
  • 158k
9 votes
1 answer
7k views

Neural network diverging instead of converging

I have implemented a neural network (using CUDA) with 2 layers. (2 Neurons per layer). I'm trying to make it learn 2 simple quadratic polynomial functions using backpropagation. But instead of ...
Shayan RC's user avatar
  • 633
9 votes
1 answer
3k views

What's the input to the decoder in a sequence to sequence autoencoder?

What's the input to the decoder part of a sequence to sequence autoencoder? I've seen certain examples of such an autoencoder (using LSTM's more often than not) but am still unclear. For example, ...
Mathguy's user avatar
  • 411
9 votes
1 answer
1k views

Conflict Driven Clause Learning backtracking clarification

On the wikipedia page here it describes pretty well the CDCL algorithm (and it seems the pictures were taken from slides created by Sharad Malik at Princeton). However when describing how to backtrack ...
Jake's user avatar
  • 3,800
9 votes
1 answer
2k views

Why are weights of Neural Networks initialized with random numbers?

Why are neural networks initial weights initialized as random numbers? I had read somewhere that this is done to "break the symmetry" and this makes the neural network learn faster. How does breaking ...
Shayan RC's user avatar
  • 633

1
2 3 4 5
25