The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [machine-learning]

Questions about computer algorithms that automatically discover patterns in data and make good decisions based on them.

303 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
10
votes
0answers
1k views

Alternatives to SVD for rank factorization

I have rank-deficient matrix $M \in \mathbb{R}^{n\times m}$ with $\text{rank}(M) = k$ and I want to find a rank factorization $M = PQ$ with $P \in \mathbb{R}^{n \times k}$ and $Q \in \mathbb{R}^{k \...
7
votes
0answers
2k views

Detecting the damaged regions in cars

Detecting the regions where a car has been damaged and the extent to which it has been damaged is a very interesting problem. It has potential applications in automatic auto insurance claims. ...
7
votes
0answers
153 views

Formulating shortest path as submodular minimization

I'm curious about the general question of whether any combinatorial optimization problem with polynomial time solution can necessarily be reformulated as minimizing a submodular function. The answer ...
7
votes
0answers
64 views

Why are kernel methods with RBFs effective for handwritten digits (letters) classification?

The question emerged while reading Ch. 3 of Rasmussen & Williams http://www.gaussianprocess.org/gpml/. In the end of this chapter, the authors gave results for the problem of handwritten digits ...
7
votes
0answers
1k views

What machine learning method for diabetes prediction SW?

I'm thinking of an application for diabetics, that, given previous values of blood glucose and insulin dosage, predicts the glucose level for the next few hours. I know a few things about neural ...
6
votes
1answer
381 views

Machine learning approach to auction game

I am newbie with machine learning. In order to learn more I decided to try solving a specific problem/game that I have in mind. The problem is the following: I have a list of $N$ items which are ...
5
votes
1answer
188 views

Why does the effectiveness of my reinforcement based neural network recede after a while?

I have a reinforcement based neural network training on the OpenAI gym CartPole-v1 environment. For the structure and training algorithm, assume it is the same as the one in this article. Typically, ...
4
votes
0answers
343 views

graph signal processing

What's the intuition behind a ''Graph fourier transform'' ? I'm not so much interested in mathematical details or technical applications. I'm trying to grasp what a graph fourier transform actually ...
4
votes
1answer
239 views

reinforcement learning in gridworld with subgoals

Andrew Ng, Daishi Harada, Stuart Russell published a conference paper entitled Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping. There is a specific example ...
4
votes
0answers
50 views

Infer probabilities, for concatenation of words

Fix an alphabet $\Sigma$, and a set of words, $W = \{w_1,\dots,w_n\} \subseteq \Sigma^*$. I have a randomized model that works like this: Alice generates a random sequence of words, using some ...
4
votes
0answers
192 views

What is the simplest written language to work on in handwriting recognition?

Handwriting recognition is an important but very complicated domain of Computer Sciences. Computers nowadays do a quite fair job even if there is room for improvement in the future. I am wondering if ...
4
votes
0answers
162 views

some kernel and greater margin, how this occures?

I read following notes, and couldn't get it. any idea or hint would highly appreciated. a SVM classifier using a second order polynomial kernel. The first polynomial kernel maps each input data x to ...
3
votes
0answers
32 views

How to compute the loss and backprop of word2vec skip-gram using hierarchical softmax?

So we are calculating the loss $$J(\theta) = -\frac{1}{T}\sum_{t=1}^T\sum_{-m \leq j \leq m} \log P(w_{t+j}|w_t;\theta)$$ and to do this we need to calculate $$P(o|c) = \frac{\exp(u_o^Tv_c)}{\sum \...
3
votes
0answers
79 views

Was there a phase in Machine Learning timeline when researchers thought some Neural Networks could not be trained?

I was talking to a professor who made a comment to my question. Me: So much of quality literature around this topic ( IP Protection for Neural Weights) emanated in 1990-1991, I'm truly at loss ...
3
votes
0answers
85 views

Clarification in model precision term for Gaussian Processes

Following the blog post, where the dropout in the deep learning models has been approximated to a Gaussian process. Research paper and its appendix. Looking at the suggestion of the author from the ...
3
votes
0answers
644 views

Square or circle neural network detection

I am trying to use a simple perceptron to recognize if there is a square or a circle on an image. The images I generated are 300x300 px and I am having issues training the network since the images are ...
3
votes
0answers
1k views

What is the Big-Oh asymptotic complexity of learning in Random Forests?

Random Forests is a bagged ensemble of CART learners. The following is what I've found, but am not sure if I'm completely right. CART (Classification and Regression Trees) uses the Gini index for ...
3
votes
0answers
67 views

Number of parameters to be optimized in Artificial Bee Colony

I was reading this paper - Software defect prediction using cost-sensitive neural network by Ömer Faruk Arara and Kürsat Ayan It uses Artificial Bee Colony algorithm to train the neural network. In ...
3
votes
0answers
75 views

What is the level of generality of the bias-variance tradeoff?

I have just learned about the "bias-variance tradeoff" in machine learning, in the context of it being applied to simple regression models. Now I am wondering: is this tradeoff a general universal ...
3
votes
0answers
103 views

What are the theoretical and practical contributions of Multiagent Systems to science?

Speaking about multiagent systems (MAS) is about as fuzzy as talking about artificial intelligence systems (AI). They are in essence the distributed counterpart of AI. While there are no so-called "...
3
votes
0answers
1k views

Which machine learning algorithm is appropriate for predicting a vector?

I have a very large set of animal migration data, consisting of many series of vectors - each series is basically a path of a single animal. The dataset I'm using consists of 244 of these series. I ...
3
votes
0answers
735 views

The runtime of a neural net with given numbers of observations, features, and neurons

If I have $n$ training observations, $m$ number of features per observation, and my neural network has $x$ neurons in the 1st layer, $y$ neurons in the 2nd layer, and 1 output neuron, what is the ...
3
votes
0answers
420 views

What is the activation function, label and loss function for Hierachical Softmax

Several papers(1 (originator), 2, 3) suggest the use of Hierachical Softmax instead of softmax for classification where the number of classes is large (eg many thousand). I haven't been able to get ...
3
votes
0answers
480 views

Reinforcement learning - state space and action space

I am working on a reinforcement learning strategy for parameter control of a local search heuristic. The complete state for this RL problem can be defined as $S = \{s, p\}$, where $s$ and $p$ ...
3
votes
0answers
76 views

Document clustering for summarization

I am curious as to what steps one would reasonably need to take to perform an extraction-based text summarizer. I've taken a look at some papers I've found on Google such as this one, which explains ...
3
votes
0answers
252 views

How to find the shattered set size for unknown hypothesis target

My aim is to prove a VC-dimension $d$ for different problems. All the problems I have do not have a target function (or concept) explicitly stated. This unlike most of the examples I came through. For ...
3
votes
0answers
167 views

Graphs invariant to permutations of vertices

I am reading a paper on Semi Supervised Learning and I am confused about a term. The paper talks about graphs that are invariant to permutations of the vertices. Can somebody explain or perhaps give ...
3
votes
1answer
434 views

Encoding of graphs as an input to neural nets

Given a directed graph $G=(V,E)$ with a node labelling function $l:V\rightarrow L$, how would you encode this best for a neural network? If it simplifies the problem, we can add the following ...
2
votes
0answers
5 views

Beam search in the context of Genetic Algorithm

As per Machine Learning, Tom M. Mitchell, Indian Edition, pp. 249-262, it is mentioned that "genetic algorithms employ a randomized beam search method to seek a maximally fit hypothesis". I have ...
2
votes
1answer
20 views

Neural network game players and incremental updates

Neural networks in recent years have been successfully used for gameplaying. A difference between games and e.g. image processing is that the game boards get updated incrementally. Do any neural ...
2
votes
0answers
21 views

The No-Free-Lunch Theorem and K-NN consistency

In computational learning, The NFL theorem states that there is no universal learner. For every learning algorithm , there is a distribution that causes the learner to output a hypotesis with a large ...
2
votes
0answers
24 views

What is the purpose of standardization in machine learning?

I'm just getting started with learning about K-nearest neighbor and am having a hard time understanding why standardization is required. Reading through, I came across a section saying When ...
2
votes
0answers
21 views

Cross Entropy Function

I've seen two versions of the cross entropy cost function, and conflicting information about it. \begin{equation}J(\theta) = -\frac{1}{N} \sum_{n=1}^N\sum_{i=1}^C y_{ni}\log \hat{y}_{n_i} (\theta)\end{...
2
votes
1answer
37 views

k-means clustered data: how to label newly incoming data

I have a data set with labels that were produced by a $k$-means clustering algorithm. Now there is some data (with the same data structure) from another source and I wonder what is the most sensible ...
2
votes
0answers
47 views

Card dealing problem with constraints (blacklisting),

A friend of mine and I are trying to teach a bot play a card game (bela) We are using monte carlo tree search (MCTS) to estimate the probability of winning hand in regards to multiple possible (!...
2
votes
0answers
83 views

Moon lander algorithm

Not talking about the actual moon lander, but an old game that was inspired by it (see screenshot). Suppose I wanted to write a program that "plays" this game: Can only operate the vertical thruster ...
2
votes
0answers
79 views

Approximate dot product between neural network output layer's parameter vector and input activations with winner-take-all hashing

In the paper Deep Networks with Large Output Spaces, Vijayanarasimhan et al. describe their approach to approximating the dot product between a neural network's output layer's parameter vector and ...
2
votes
0answers
45 views

About gradient descent on non-convex functions

There is this "folklore" result that gradient descent on a non-convex function takes $O(\frac n {\epsilon^2})$ steps to get to a point whose gradient norm is below $\epsilon$ and with SGD this takes $...
2
votes
0answers
75 views

Simulated annealing upper bound seems way too high

In a nutshell I found this paper that provides an upper bound for the amount of iterations we expect before visiting the global optimum at least once. (It then uses that number to find a lower bound ...
2
votes
0answers
15 views

Induction / (machine learning) of Resource Grammars for Grammatical Framework?

Grammatical Framework is based in Abstract Categorial Grammars. It is known that Combinatory Categorial Grammars have grammar induction/learning capabilities see e.g. https://link.springer.com/article/...
2
votes
0answers
24 views

How often should I read out information from an echo state recurrent neural network?

Recurrent neural networks makes it possible to implement some kind of memory, which can be very useful for a lot of tasks, incl. (but not limited to) robot control, which I am interested in. For ...
2
votes
0answers
28 views

Can machine learning be used in this scenario?

I'm keen to learn more about machine learning, as I feel it could be something that might have its uses in my place of work. I write PC software that controls scientific instruments, and one of its ...
2
votes
0answers
38 views

What's the reason for sqrt(n) bounds in online learning?

I have a question regarding no-regret algorithms (of online learning). As far as I can see, such algorithms allow the absolute regret up to round $n$, which is $R_n$, to grow by $\sqrt{n}$. So, in the ...
2
votes
0answers
34 views

Combine all known ML algorithms into one through Bayes

Let there be an infinite series of pieces of data we want to predict. Take the set of all algorithms ("hypotheses") ML has discovered; different hyper-parameters count as another algorithm. Give each ...
2
votes
0answers
485 views

Tuning the parameters of Particle swarm optimization (PSO)

To tune the parameters of Particle swarm optimization (PSO), there are two methods offline and online. In offline manner, the meta-optimization is used to tune the parameters of PSO by using another ...
2
votes
0answers
251 views

Faster RCNN: how to translate coordinates

I'm trying to understand and use the Faster R-CNN algorithm on my own data. My question is about ROI coordinates: what we have as labels, and what we want in the end, are ROI coordinates in the input ...
2
votes
0answers
189 views

How can I improve my KNN classifier?

I'm trying to teach myself a bit about machine learning, so one of the first things I did was implement a KNN classifier in ruby. My goal was to classify text product reviews into 8 classes: books-...
2
votes
0answers
169 views

Multilayer perceptron memory requirements

How much memory do we need to train a multilayer perceptron? I've started to figure this out myself, but I'm stuck. I have one-layer MLP. Each training example is a vector of 100 real numbers in ...
2
votes
0answers
80 views

Why to use moving statistics instead of population statistics in batch normalization implementation at inference?

Many (tensorflow) implementations of batch normalization seem to use moving statistics at its inference phase (R2RT, tf.contrib.layers.batch_norm). But in the algorithm from the original paper, they ...
2
votes
0answers
112 views

Trying to understand Le's “cat paper”

QV Le et al. show in Building high-level features using large scale unsupervised learning (2012) how to use unsupervised learning of a deep neural net to recognize faces and cats. Working through this ...