Questions tagged [machine-learning]
Questions about computer algorithms that automatically discover patterns in data and make good decisions based on them.
303
questions with no upvoted or accepted answers
10
votes
0answers
1k views
Alternatives to SVD for rank factorization
I have rank-deficient matrix $M \in \mathbb{R}^{n\times m}$ with $\text{rank}(M) = k$ and I want to find a rank factorization $M = PQ$ with $P \in \mathbb{R}^{n \times k}$ and $Q \in \mathbb{R}^{k \...
7
votes
0answers
2k views
Detecting the damaged regions in cars
Detecting the regions where a car has been damaged and the extent to which it has been damaged is a very interesting problem. It has potential applications in automatic auto insurance claims. ...
7
votes
0answers
153 views
Formulating shortest path as submodular minimization
I'm curious about the general question of whether any combinatorial optimization problem with polynomial time solution can necessarily be reformulated as minimizing a submodular function.
The answer ...
7
votes
0answers
64 views
Why are kernel methods with RBFs effective for handwritten digits (letters) classification?
The question emerged while reading Ch. 3 of Rasmussen & Williams http://www.gaussianprocess.org/gpml/. In the end of this chapter, the authors gave results for the problem of handwritten digits ...
7
votes
0answers
1k views
What machine learning method for diabetes prediction SW?
I'm thinking of an application for diabetics, that, given previous values of blood glucose and insulin dosage, predicts the glucose level for the next few hours.
I know a few things about neural ...
6
votes
1answer
381 views
Machine learning approach to auction game
I am newbie with machine learning. In order to learn more I decided to try solving a specific problem/game that I have in mind. The problem is the following:
I have a list of $N$ items which are ...
5
votes
1answer
188 views
Why does the effectiveness of my reinforcement based neural network recede after a while?
I have a reinforcement based neural network training on the OpenAI gym CartPole-v1 environment. For the structure and training algorithm, assume it is the same as the one in this article.
Typically, ...
4
votes
0answers
343 views
graph signal processing
What's the intuition behind a ''Graph fourier transform'' ? I'm not so much interested in mathematical details or technical applications. I'm trying to grasp what a graph fourier transform actually ...
4
votes
1answer
239 views
reinforcement learning in gridworld with subgoals
Andrew Ng, Daishi Harada, Stuart Russell published a conference paper entitled Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping.
There is a specific example ...
4
votes
0answers
50 views
Infer probabilities, for concatenation of words
Fix an alphabet $\Sigma$, and a set of words, $W = \{w_1,\dots,w_n\} \subseteq \Sigma^*$.
I have a randomized model that works like this: Alice generates a random sequence of words, using some ...
4
votes
0answers
192 views
What is the simplest written language to work on in handwriting recognition?
Handwriting recognition is an important but very complicated domain of Computer Sciences. Computers nowadays do a quite fair job even if there is room for improvement in the future.
I am wondering if ...
4
votes
0answers
162 views
some kernel and greater margin, how this occures?
I read following notes, and couldn't get it. any idea or hint would highly appreciated.
a SVM classifier using a second order polynomial kernel. The first polynomial kernel maps each input data x to ...
3
votes
0answers
32 views
How to compute the loss and backprop of word2vec skip-gram using hierarchical softmax?
So we are calculating the loss
$$J(\theta) = -\frac{1}{T}\sum_{t=1}^T\sum_{-m \leq j \leq m} \log P(w_{t+j}|w_t;\theta)$$
and to do this we need to calculate
$$P(o|c) = \frac{\exp(u_o^Tv_c)}{\sum \...
3
votes
0answers
79 views
Was there a phase in Machine Learning timeline when researchers thought some Neural Networks could not be trained?
I was talking to a professor who made a comment to my question.
Me: So much of quality literature around this topic ( IP Protection
for Neural Weights) emanated in 1990-1991, I'm truly at loss ...
3
votes
0answers
85 views
Clarification in model precision term for Gaussian Processes
Following the blog post, where the dropout in the deep learning models has been approximated to a Gaussian process. Research paper and its appendix.
Looking at the suggestion of the author from the ...
3
votes
0answers
644 views
Square or circle neural network detection
I am trying to use a simple perceptron to recognize if there is a square or a circle on an image. The images I generated are 300x300 px and I am having issues training the network since the images are ...
3
votes
0answers
1k views
What is the Big-Oh asymptotic complexity of learning in Random Forests?
Random Forests is a bagged ensemble of CART learners. The following is what I've found, but am not sure if I'm completely right.
CART (Classification and Regression Trees) uses the Gini index for ...
3
votes
0answers
67 views
Number of parameters to be optimized in Artificial Bee Colony
I was reading this paper - Software defect prediction using cost-sensitive neural network by Ömer Faruk Arara and Kürsat Ayan
It uses Artificial Bee Colony algorithm to train the neural network. In ...
3
votes
0answers
75 views
What is the level of generality of the bias-variance tradeoff?
I have just learned about the "bias-variance tradeoff" in machine learning, in the context of it being applied to simple regression models.
Now I am wondering: is this tradeoff a general universal ...
3
votes
0answers
103 views
What are the theoretical and practical contributions of Multiagent Systems to science?
Speaking about multiagent systems (MAS) is about as fuzzy as talking about artificial intelligence systems (AI). They are in essence the distributed counterpart of AI.
While there are no so-called "...
3
votes
0answers
1k views
Which machine learning algorithm is appropriate for predicting a vector?
I have a very large set of animal migration data, consisting of many series of vectors - each series is basically a path of a single animal. The dataset I'm using consists of 244 of these series.
I ...
3
votes
0answers
735 views
The runtime of a neural net with given numbers of observations, features, and neurons
If I have $n$ training observations, $m$ number of features per observation, and my neural network has $x$ neurons in the 1st layer, $y$ neurons in the 2nd layer, and 1 output neuron, what is the ...
3
votes
0answers
420 views
What is the activation function, label and loss function for Hierachical Softmax
Several papers(1 (originator), 2, 3) suggest the use of Hierachical Softmax instead of softmax for classification where the number of classes is large (eg many thousand).
I haven't been able to get ...
3
votes
0answers
480 views
Reinforcement learning - state space and action space
I am working on a reinforcement learning strategy for parameter control of a local search heuristic.
The complete state for this RL problem can be defined as $S = \{s, p\}$, where $s$ and $p$ ...
3
votes
0answers
76 views
Document clustering for summarization
I am curious as to what steps one would reasonably need to take to perform an extraction-based text summarizer.
I've taken a look at some papers I've found on Google such as this one, which explains ...
3
votes
0answers
252 views
How to find the shattered set size for unknown hypothesis target
My aim is to prove a VC-dimension $d$ for different problems. All the problems I have do not have a target function (or concept) explicitly stated. This unlike most of the examples I came through. For ...
3
votes
0answers
167 views
Graphs invariant to permutations of vertices
I am reading a paper on Semi Supervised Learning and I am confused about a term. The paper talks about graphs that are invariant to permutations of the vertices. Can somebody explain or perhaps give ...
3
votes
1answer
434 views
Encoding of graphs as an input to neural nets
Given a directed graph $G=(V,E)$ with a node labelling function $l:V\rightarrow L$, how would you encode this best for a neural network?
If it simplifies the problem, we can add the following ...
2
votes
0answers
5 views
Beam search in the context of Genetic Algorithm
As per Machine Learning, Tom M. Mitchell, Indian Edition, pp. 249-262, it is mentioned that "genetic algorithms employ a randomized beam search method to seek a maximally fit hypothesis".
I have ...
2
votes
1answer
20 views
Neural network game players and incremental updates
Neural networks in recent years have been successfully used for gameplaying. A difference between games and e.g. image processing is that the game boards get updated incrementally. Do any neural ...
2
votes
0answers
21 views
The No-Free-Lunch Theorem and K-NN consistency
In computational learning, The NFL theorem states that there is no universal learner. For every learning algorithm , there is a distribution that causes the learner to output a hypotesis with a large ...
2
votes
0answers
24 views
What is the purpose of standardization in machine learning?
I'm just getting started with learning about K-nearest neighbor and am having a hard time understanding why standardization is required. Reading through, I came across a section saying
When ...
2
votes
0answers
21 views
Cross Entropy Function
I've seen two versions of the cross entropy cost function, and conflicting information about it.
\begin{equation}J(\theta) = -\frac{1}{N} \sum_{n=1}^N\sum_{i=1}^C y_{ni}\log \hat{y}_{n_i} (\theta)\end{...
2
votes
1answer
37 views
k-means clustered data: how to label newly incoming data
I have a data set with labels that were produced by a $k$-means clustering
algorithm. Now there is some data (with the same data structure) from another
source and I wonder what is the most sensible ...
2
votes
0answers
47 views
Card dealing problem with constraints (blacklisting),
A friend of mine and I are trying to teach a bot play a card game (bela)
We are using monte carlo tree search (MCTS) to estimate the probability of winning hand in regards to multiple possible (!...
2
votes
0answers
83 views
Moon lander algorithm
Not talking about the actual moon lander, but an old game that was inspired by it (see screenshot).
Suppose I wanted to write a program that "plays" this game:
Can only operate the vertical thruster ...
2
votes
0answers
79 views
Approximate dot product between neural network output layer's parameter vector and input activations with winner-take-all hashing
In the paper Deep Networks with Large Output Spaces, Vijayanarasimhan et al. describe their approach to approximating the dot product between a neural network's output layer's parameter vector and ...
2
votes
0answers
45 views
About gradient descent on non-convex functions
There is this "folklore" result that gradient descent on a non-convex function takes $O(\frac n {\epsilon^2})$ steps to get to a point whose gradient norm is below $\epsilon$ and with SGD this takes $...
2
votes
0answers
75 views
Simulated annealing upper bound seems way too high
In a nutshell
I found this paper that provides an upper bound for the amount of iterations we expect before visiting the global optimum at least once. (It then uses that number to find a lower bound ...
2
votes
0answers
15 views
Induction / (machine learning) of Resource Grammars for Grammatical Framework?
Grammatical Framework is based in Abstract Categorial Grammars. It is known that Combinatory Categorial Grammars have grammar induction/learning capabilities see e.g. https://link.springer.com/article/...
2
votes
0answers
24 views
How often should I read out information from an echo state recurrent neural network?
Recurrent neural networks makes it possible to implement some kind of memory, which can be very useful for a lot of tasks, incl. (but not limited to) robot control, which I am interested in. For ...
2
votes
0answers
28 views
Can machine learning be used in this scenario?
I'm keen to learn more about machine learning, as I feel it could be something that might have its uses in my place of work. I write PC software that controls scientific instruments, and one of its ...
2
votes
0answers
38 views
What's the reason for sqrt(n) bounds in online learning?
I have a question regarding no-regret algorithms (of online learning). As far as I can see, such algorithms allow the absolute regret up to round $n$, which is $R_n$, to grow by $\sqrt{n}$. So, in the ...
2
votes
0answers
34 views
Combine all known ML algorithms into one through Bayes
Let there be an infinite series of pieces of data we want to predict. Take the set of all algorithms ("hypotheses") ML has discovered; different hyper-parameters count as another algorithm. Give each ...
2
votes
0answers
485 views
Tuning the parameters of Particle swarm optimization (PSO)
To tune the parameters of Particle swarm optimization (PSO), there are two methods offline and online. In offline manner, the meta-optimization is used to tune the parameters of PSO by using another ...
2
votes
0answers
251 views
Faster RCNN: how to translate coordinates
I'm trying to understand and use the Faster R-CNN algorithm on my own data.
My question is about ROI coordinates: what we have as labels, and what we want in the end, are ROI coordinates in the input ...
2
votes
0answers
189 views
How can I improve my KNN classifier?
I'm trying to teach myself a bit about machine learning, so one of the first things I did was implement a KNN classifier in ruby. My goal was to classify text product reviews into 8 classes:
books-...
2
votes
0answers
169 views
Multilayer perceptron memory requirements
How much memory do we need to train a multilayer perceptron?
I've started to figure this out myself, but I'm stuck.
I have one-layer MLP. Each training example is a vector of 100 real numbers in ...
2
votes
0answers
80 views
Why to use moving statistics instead of population statistics in batch normalization implementation at inference?
Many (tensorflow) implementations of batch normalization seem to use moving statistics at its inference phase (R2RT, tf.contrib.layers.batch_norm).
But in the algorithm from the original paper, they ...
2
votes
0answers
112 views
Trying to understand Le's “cat paper”
QV Le et al. show in Building high-level features using large scale unsupervised learning (2012) how to use unsupervised learning of a deep neural net to recognize faces and cats. Working through this ...
