Questions tagged [machine-learning]
Questions about computer algorithms that automatically discover patterns in data and make good decisions based on them.
388
questions with no upvoted or accepted answers
11
votes
0answers
2k views
Alternatives to SVD for rank factorization
I have rank-deficient matrix $M \in \mathbb{R}^{n\times m}$ with $\text{rank}(M) = k$ and I want to find a rank factorization $M = PQ$ with $P \in \mathbb{R}^{n \times k}$ and $Q \in \mathbb{R}^{k \...
7
votes
0answers
2k views
Detecting the damaged regions in cars
Detecting the regions where a car has been damaged and the extent to which it has been damaged is a very interesting problem. It has potential applications in automatic auto insurance claims. ...
7
votes
0answers
185 views
Formulating shortest path as submodular minimization
I'm curious about the general question of whether any combinatorial optimization problem with polynomial time solution can necessarily be reformulated as minimizing a submodular function.
The answer ...
7
votes
0answers
69 views
Why are kernel methods with RBFs effective for handwritten digits (letters) classification?
The question emerged while reading Ch. 3 of Rasmussen & Williams http://www.gaussianprocess.org/gpml/. In the end of this chapter, the authors gave results for the problem of handwritten digits ...
7
votes
0answers
1k views
What machine learning method for diabetes prediction SW?
I'm thinking of an application for diabetics, that, given previous values of blood glucose and insulin dosage, predicts the glucose level for the next few hours.
I know a few things about neural ...
6
votes
1answer
542 views
Machine learning approach to auction game
I am newbie with machine learning. In order to learn more I decided to try solving a specific problem/game that I have in mind. The problem is the following:
I have a list of $N$ items which are ...
4
votes
0answers
285 views
Axis aligned rectangles: why is A an ERM in the case of infinite domain?
I'm working on a problem 2.3a in Shalev-Shwartz/Ben-David's Machine learning textbook, which states:
An axis aligned rectangle classifier in the plane is a classifier that assigns 1 to a point if and ...
4
votes
0answers
102 views
Clarification in model precision term for Gaussian Processes
Following the blog post, where the dropout in the deep learning models has been approximated to a Gaussian process. Research paper and its appendix.
Looking at the suggestion of the author from the ...
4
votes
1answer
312 views
reinforcement learning in gridworld with subgoals
Andrew Ng, Daishi Harada, Stuart Russell published a conference paper entitled Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping.
There is a specific example ...
4
votes
0answers
59 views
Infer probabilities, for concatenation of words
Fix an alphabet $\Sigma$, and a set of words, $W = \{w_1,\dots,w_n\} \subseteq \Sigma^*$.
I have a randomized model that works like this: Alice generates a random sequence of words, using some ...
4
votes
0answers
240 views
What is the simplest written language to work on in handwriting recognition?
Handwriting recognition is an important but very complicated domain of Computer Sciences. Computers nowadays do a quite fair job even if there is room for improvement in the future.
I am wondering if ...
4
votes
0answers
197 views
Graphs invariant to permutations of vertices
I am reading a paper on Semi Supervised Learning and I am confused about a term. The paper talks about graphs that are invariant to permutations of the vertices. Can somebody explain or perhaps give ...
3
votes
0answers
30 views
What is the purpose of standardization in machine learning?
I'm just getting started with learning about K-nearest neighbor and am having a hard time understanding why standardization is required. Reading through, I came across a section saying
When ...
3
votes
0answers
60 views
How to compute the loss and backprop of word2vec skip-gram using hierarchical softmax?
So we are calculating the loss
$$J(\theta) = -\frac{1}{T}\sum_{t=1}^T\sum_{-m \leq j \leq m} \log P(w_{t+j}|w_t;\theta)$$
and to do this we need to calculate
$$P(o|c) = \frac{\exp(u_o^Tv_c)}{\sum \...
3
votes
0answers
370 views
graph signal processing
What's the intuition behind a ''Graph fourier transform'' ? I'm not so much interested in mathematical details or technical applications. I'm trying to grasp what a graph fourier transform actually ...
3
votes
0answers
90 views
Was there a phase in Machine Learning timeline when researchers thought some Neural Networks could not be trained?
I was talking to a professor who made a comment to my question.
Me: So much of quality literature around this topic ( IP Protection
for Neural Weights) emanated in 1990-1991, I'm truly at loss ...
3
votes
0answers
1k views
Is Chinese Checkers a solved game
Is there a machine or algorithm available to solve the board game "Chinese checkers"? If there is please let me know about relevant material. I have already searched over internet unfortunately i am ...
3
votes
0answers
622 views
What are the inputs to an LSTM for Slot Filling Task
I am confused on the inputs of a Long-Short Term Memory (LSTM) for the slot filling task in Spoken Language Understanding.
Before I worked on this, I implemented a language model with a Recurrent ...
3
votes
0answers
750 views
Square or circle neural network detection
I am trying to use a simple perceptron to recognize if there is a square or a circle on an image. The images I generated are 300x300 px and I am having issues training the network since the images are ...
3
votes
0answers
1k views
What is the Big-Oh asymptotic complexity of learning in Random Forests?
Random Forests is a bagged ensemble of CART learners. The following is what I've found, but am not sure if I'm completely right.
CART (Classification and Regression Trees) uses the Gini index for ...
3
votes
0answers
70 views
Number of parameters to be optimized in Artificial Bee Colony
I was reading this paper - Software defect prediction using cost-sensitive neural network by Ömer Faruk Arara and Kürsat Ayan
It uses Artificial Bee Colony algorithm to train the neural network. In ...
3
votes
0answers
81 views
What is the level of generality of the bias-variance tradeoff?
I have just learned about the "bias-variance tradeoff" in machine learning, in the context of it being applied to simple regression models.
Now I am wondering: is this tradeoff a general universal ...
3
votes
0answers
126 views
What are the theoretical and practical contributions of Multiagent Systems to science?
Speaking about multiagent systems (MAS) is about as fuzzy as talking about artificial intelligence systems (AI). They are in essence the distributed counterpart of AI.
While there are no so-called "...
3
votes
0answers
2k views
Which machine learning algorithm is appropriate for predicting a vector?
I have a very large set of animal migration data, consisting of many series of vectors - each series is basically a path of a single animal. The dataset I'm using consists of 244 of these series.
I ...
3
votes
0answers
922 views
The runtime of a neural net with given numbers of observations, features, and neurons
If I have $n$ training observations, $m$ number of features per observation, and my neural network has $x$ neurons in the 1st layer, $y$ neurons in the 2nd layer, and 1 output neuron, what is the ...
3
votes
0answers
458 views
What is the activation function, label and loss function for Hierachical Softmax
Several papers(1 (originator), 2, 3) suggest the use of Hierachical Softmax instead of softmax for classification where the number of classes is large (eg many thousand).
I haven't been able to get ...
3
votes
0answers
536 views
Reinforcement learning - state space and action space
I am working on a reinforcement learning strategy for parameter control of a local search heuristic.
The complete state for this RL problem can be defined as $S = \{s, p\}$, where $s$ and $p$ ...
3
votes
0answers
86 views
Document clustering for summarization
I am curious as to what steps one would reasonably need to take to perform an extraction-based text summarizer.
I've taken a look at some papers I've found on Google such as this one, which explains ...
3
votes
0answers
263 views
How to find the shattered set size for unknown hypothesis target
My aim is to prove a VC-dimension $d$ for different problems. All the problems I have do not have a target function (or concept) explicitly stated. This unlike most of the examples I came through. For ...
3
votes
1answer
554 views
Encoding of graphs as an input to neural nets
Given a directed graph $G=(V,E)$ with a node labelling function $l:V\rightarrow L$, how would you encode this best for a neural network?
If it simplifies the problem, we can add the following ...
2
votes
1answer
38 views
What is the practical difference between "differentiable programming" and existing autograd libaries?
People have recently started talking about "differentiable programming". I have listened to some people talk about this at a philosophical level, but I don't see the practical difference ...
2
votes
1answer
38 views
Why is $\mathcal{D}^m(\{S:L_{(\mathcal{D},f)}(A(S))\gt \epsilon\})\leq \mathcal{D}^m\left(\bigcup^4_{i=1}F_i\right)$ true?
I am studying the book "Understanding Machine Learning:
From Theory to Algorithms". I am struggling to understand the solution to exercise 3 (2) on page 41.
Exercise:
An axis aligned ...
2
votes
0answers
76 views
Monte Carlo tree search optimizations
I am designing a algoritm for a game using a Monte Carlo tree search AI that I implemented. I play against another player and want to get the best move. Every time I do a move I completely build a new ...
2
votes
0answers
25 views
Simple back propagation example
Sorry if this is too simplistic of a question, but over the last couple of months I have been working through the course mathematical foundations of machine learning at my college. I think I am really ...
2
votes
0answers
38 views
Covering numbers to show that H is agnostically PAC-learnable
Suppose $X=[0,1]$ and $Y=[0,1]$, and we use the squared loss
Let's define the hypothesis class $H = {h(x) = (x-a)^2 : a \in [0,1]}$.
Question: How can covering numbers be used to show that this ...
2
votes
0answers
27 views
Beam search in the context of Genetic Algorithm
As per Machine Learning, Tom M. Mitchell, Indian Edition, pp. 249-262, it is mentioned that "genetic algorithms employ a randomized beam search method to seek a maximally fit hypothesis".
I have ...
2
votes
0answers
45 views
The No-Free-Lunch Theorem and K-NN consistency
In computational learning, The NFL theorem states that there is no universal learner. For every learning algorithm , there is a distribution that causes the learner to output a hypotesis with a large ...
2
votes
0answers
29 views
Cross Entropy Function
I've seen two versions of the cross entropy cost function, and conflicting information about it.
\begin{equation}J(\theta) = -\frac{1}{N} \sum_{n=1}^N\sum_{i=1}^C y_{ni}\log \hat{y}_{n_i} (\theta)\end{...
2
votes
0answers
79 views
How is momentum an approximation of Hessian based optimization?
In the answer to "what is the Hessian" at this site:
https://stackoverflow.com/questions/23297090/how-calculating-hessian-works-for-neural-network-learning
the person answering the question ...
2
votes
0answers
58 views
Card dealing problem with constraints (blacklisting),
A friend of mine and I are trying to teach a bot play a card game (bela)
We are using monte carlo tree search (MCTS) to estimate the probability of winning hand in regards to multiple possible (!...
2
votes
0answers
199 views
Moon lander algorithm
Not talking about the actual moon lander, but an old game that was inspired by it (see screenshot).
Suppose I wanted to write a program that "plays" this game:
Can only operate the vertical thruster ...
2
votes
0answers
122 views
Approximate dot product between neural network output layer's parameter vector and input activations with winner-take-all hashing
In the paper Deep Networks with Large Output Spaces, Vijayanarasimhan et al. describe their approach to approximating the dot product between a neural network's output layer's parameter vector and ...
2
votes
0answers
53 views
About gradient descent on non-convex functions
There is this "folklore" result that gradient descent on a non-convex function takes $O(\frac n {\epsilon^2})$ steps to get to a point whose gradient norm is below $\epsilon$ and with SGD this takes $...
2
votes
0answers
93 views
Simulated annealing upper bound seems way too high
In a nutshell
I found this paper that provides an upper bound for the amount of iterations we expect before visiting the global optimum at least once. (It then uses that number to find a lower bound ...
2
votes
0answers
18 views
Induction / (machine learning) of Resource Grammars for Grammatical Framework?
Grammatical Framework is based in Abstract Categorial Grammars. It is known that Combinatory Categorial Grammars have grammar induction/learning capabilities see e.g. https://link.springer.com/article/...
2
votes
0answers
31 views
How often should I read out information from an echo state recurrent neural network?
Recurrent neural networks makes it possible to implement some kind of memory, which can be very useful for a lot of tasks, incl. (but not limited to) robot control, which I am interested in. For ...
2
votes
0answers
29 views
Can machine learning be used in this scenario?
I'm keen to learn more about machine learning, as I feel it could be something that might have its uses in my place of work. I write PC software that controls scientific instruments, and one of its ...
2
votes
0answers
49 views
What's the reason for sqrt(n) bounds in online learning?
I have a question regarding no-regret algorithms (of online learning). As far as I can see, such algorithms allow the absolute regret up to round $n$, which is $R_n$, to grow by $\sqrt{n}$. So, in the ...
2
votes
0answers
36 views
Combine all known ML algorithms into one through Bayes
Let there be an infinite series of pieces of data we want to predict. Take the set of all algorithms ("hypotheses") ML has discovered; different hyper-parameters count as another algorithm. Give each ...
2
votes
0answers
560 views
Tuning the parameters of Particle swarm optimization (PSO)
To tune the parameters of Particle swarm optimization (PSO), there are two methods offline and online. In offline manner, the meta-optimization is used to tune the parameters of PSO by using another ...
