The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [neural-networks]

Network structure inspired by simplified models of biological neurons (brain cells). Neural networks are trained to "learn" by supervised and unsupervised techniques, and can be used to solve optimization problems, approximation problems, classify patterns, and combinations thereof.

Filter by
Sorted by
Tagged with
3
votes
1answer
151 views

Learning rule of multilayer neural networks

Suppose we have a 2-layer neural network completely connected with $d^{(0)}$ input units, $d^{(1)}$ hidden units and $d^{(2)}$ output units. We consider the error function given by $J(w) = \frac{1}{2}\...
3
votes
1answer
2k views

HOG vs. neural networks for person detection

I am very new to computer vision, (a high school student) and I am working on a project to count the number of people present in a room. I have tried to use the HOGDecriptor for person detection ...
3
votes
1answer
69 views

Training given pairs of similar values, not labels

I have pairs of "similar" values $(x_i, y_i)$ drawn from a space $x_i, y_i \in S$, and want to train a neural network $N$ such that $N(x_i)$ would be "close" to $N(y_i)$ for all $i$, yet, to make it ...
3
votes
0answers
34 views

When is currying more efficient in deep neural nets?

I'm reading a blog post on deep Q-learning, and it contrasts traditional lookup-table-based Q-learning with deep Q-learning: What I wonder about in this picture is: Why does the deep NN not also ...
3
votes
0answers
79 views

Was there a phase in Machine Learning timeline when researchers thought some Neural Networks could not be trained?

I was talking to a professor who made a comment to my question. Me: So much of quality literature around this topic ( IP Protection for Neural Weights) emanated in 1990-1991, I'm truly at loss ...
3
votes
0answers
85 views

Clarification in model precision term for Gaussian Processes

Following the blog post, where the dropout in the deep learning models has been approximated to a Gaussian process. Research paper and its appendix. Looking at the suggestion of the author from the ...
3
votes
0answers
643 views

Square or circle neural network detection

I am trying to use a simple perceptron to recognize if there is a square or a circle on an image. The images I generated are 300x300 px and I am having issues training the network since the images are ...
3
votes
0answers
78 views

What is the practical difference between an neuromorphic processors and GPU's?

IBM is working on a neuromorphic computer, Synapse, intended for computation based on artificial neural networks. My question is, what makes these types of computers more efficient at processing ...
3
votes
0answers
66 views

Number of parameters to be optimized in Artificial Bee Colony

I was reading this paper - Software defect prediction using cost-sensitive neural network by Ömer Faruk Arara and Kürsat Ayan It uses Artificial Bee Colony algorithm to train the neural network. In ...
3
votes
0answers
735 views

The runtime of a neural net with given numbers of observations, features, and neurons

If I have $n$ training observations, $m$ number of features per observation, and my neural network has $x$ neurons in the 1st layer, $y$ neurons in the 2nd layer, and 1 output neuron, what is the ...
3
votes
0answers
420 views

What is the activation function, label and loss function for Hierachical Softmax

Several papers(1 (originator), 2, 3) suggest the use of Hierachical Softmax instead of softmax for classification where the number of classes is large (eg many thousand). I haven't been able to get ...
3
votes
0answers
85 views

How to select topology for neural network?

I was given a target function to design a neural network and train: $y = (x_1 \wedge x_2) \vee (x_3 \wedge x_4)$ The number of inputs and outputs seems obvious (4 and 1). And the training data can ...
3
votes
0answers
117 views

practical use of a Boltzmann machine

I am reading "Neural Networks and Learning Machines" and in Chapter 11 the book covers Boltzman machines and it is stated "the network [Boltzmann machine] can perform pattern completion", but does not ...
3
votes
2answers
324 views

About the behaviour of multi-layer perceptrons

I have a multilayer perceptron. It has an input layer with two neurons, a hidden layer with an arbitrary number of neurons, and an output layer with two neurons. Given that ...
2
votes
1answer
774 views

What are the limitations of RNNs?

For a school project, I'm planning to compare Spiking Neural Networks (SNNs) and Deep Learning recurrent neural networks, such as Long Short Term Memory (LSTMs) networks in learning a time-series. I ...
2
votes
2answers
313 views

What kind of model is used by 20 Questions?

Which kind of machine learning concept / model is used in 20 Questions? Is this kind of thing best solved by a neural network? Where I can read something about it?
2
votes
2answers
109 views

How broad is the meaning of “algorithm”?

This is a purely terminological question. The word algorithm, as I have learnt it refers to something like an "effective method, a sequence of steps, for doing something". There are alternative ...
2
votes
1answer
44 views

Is Artificial General Intelligence possible with our current machine learning models? [closed]

In other words, is artificial human level intelligence not possible yet just because of limitations in processing power and amount of data required to train the models? Or we don't have the knowledge ...
2
votes
1answer
76 views

Is there a universal learning rate for NeuralNetworks?

I'm currently creating a NeuralNetwork with backpropagation/gradient descent. There is this hyperparameter introduced called "learning rate" (η). Which has to be chosen to guarantee not overshooting ...
2
votes
2answers
352 views

Support Vector Machines as Neural Nets?

This is more of a conceptual question. I have learned about Neural Nets, and I have some clue as to how Support Vector Machines work. I read somewhere however that given the appropriate kernel (is ...
2
votes
1answer
329 views

How can I optimize 3 variables in order to maximize the end product?

I am in the process of making a cryptocurrency trading bot. Currently, I am doing backtesting over a period of 7 months in which I provide a portion of historical data as if it were in real-life. By ...
2
votes
2answers
98 views

How deep do neural networks need to be?

My question is a bit on the philosophical side, and there is probably not one single 'correct' answer on this. Nonetheless, I'm curious to hear your opinion... I'm currently designing a convolutional ...
2
votes
1answer
342 views

How to tackle different sample size in the training set in SVM

I have to train a SVM for a classification problem. I have some strings that are the paths in a deterministic finite automata (DFA). If the alphabet is -01- then possible strings are 011101110 or ...
2
votes
2answers
956 views

Measuring difference between two sets of neural network weights?

Suppose that we take a neural network of a given topology, and run it through two training processes, obtaining two different sets of converged weights at the end of the training. What is a good way ...
2
votes
1answer
1k views

How to show that cross entropy is minimized?

This Question is taken from the book Neural Networks and DeepLearning by Michael Nielsen The Question: In a single-neuron ,It is argued that the cross-entropy is small if σ(z)≈y for all training ...
2
votes
2answers
142 views

Overfitting in Machine Learning Algorithms

I am new in the ML. I know that overfitting is memorizing the data while training. Like in Neural Network, if we make lots of layers and lots of hidden nodes, we can memorize all the data, but it can ...
2
votes
1answer
160 views

Time Series Prediction with an LSTM

I have a time series that I want to predict with an LSTM. I am able to get very good results using 50 datapoints predicting 51, but I struggle to get any accuracy using something like 200 datapoints ...
2
votes
1answer
112 views

Approximating sign function by tanh in Multi Layer perceptrons

This is an exercise problem from the book "Learning from Data". Given $w_1$ and $\epsilon \gt 0$, find $w_2$ such that, $$\lvert \mbox{sign}(w_1^Tx_n)-\tanh(w_2^Tx_n)\rvert \le \epsilon$$ Where $...
2
votes
1answer
400 views

What kind of Neural Network (if any) could fit two sets of data points?

I have two datasets, one of animal migration patterns (collected over the course of a couple years) that consists of many points on an x, y plane (latitude, longitude), and the other of ocean surface ...
2
votes
1answer
3k views

How to reconstruct the image from a neural network output?

I am trying to use the genetic algorithm to optimise a multi-layered neural network for image classification (i am using a subset of the MNIST handwritten digit data set as my initial dataset, but ...
2
votes
1answer
62 views

Progressive discrete multifunction optimization

I have a set of functions $F$. The functions effectively take a set $S$ that is always a subset of a global set of all possible values $G$, where $|G|>1000$. (alternatively, they could take a $|G|$...
2
votes
3answers
212 views

Could neural networks be considered metaheuristics?

A metaheuristic is defined as a metaheuristic is a higher-level procedure or heuristic designed to find, generate, or select a heuristic (partial search algorithm) that may provide a sufficiently ...
2
votes
1answer
98 views

How to represent symbolic knowledge using real numbers - theory about neural networks and natural/analog computing?

One can define the semantics of one definite word using the references to real world entities, relationships with the other words and other concepts and represent all this knowledge about this one ...
2
votes
1answer
248 views

Parameter sharing / weight constraints in Neural Networks

I would like to train a neural network whose parameters (alternatively, weights) are subject to linear constraints such as $w_{i,j} = w_{i',j'}$, where $w_{i,j}$ denotes the weight from input node $...
2
votes
1answer
30 views

Neural Network | What is the purpose of hidden layers and how many should I use?

I am pretty new to Neural Networks and I have two questions about hidden layers: 1. What is the purpose of hidden layers? I was wondering this because obviously you can get every result you want with ...
2
votes
1answer
342 views

What can't be done with a neural network

This reference from the german wikipedia article on neural networks states: There are also many other important problems that are so difficult that a neural network will be unable to learn them ...
2
votes
1answer
1k views

Beam size is a parameter in some RNNs like TensorFlow's Magenta. What is beam size?

Magenta's melody_rnn_generate method includes a parameter beam_size. What is it and how does affect the melody?
2
votes
2answers
156 views

What algorithm to use for training combinations

I would be very glad if someone could help me with my machine learning task. I have palettes of 5 colors each (in RGB format), and would like to train the neural network so that I can input a color, ...
2
votes
1answer
268 views

Non linear neural networks?

The activation of a perceptron style neuron is: $DotProduct(Inputs, Weights)+Bias > 0$ That is essentially classifying what side of a (hyper)plane a point is on (positive or negative side), like ...
2
votes
1answer
904 views

NeuroEvolution: NEAT algorithm innovation numbers

I have been reading up on the NeuronEvolution of Augmented Topologies and there's this little thing that's been bothering me. While reading Kenneth Stanley's Paper on NEAT I came on this figure here: ...
2
votes
1answer
92 views

Use of Activation Units in Facial Expression Understanding

The topic of extracting the Facial Action Coding System (FACS) Action Units (AUs) [1] from images and it's translation into emotion prediction [2] is pretty well studied, but I'm not clear on how it ...
2
votes
1answer
46 views

How to model a set of categorical values in the input of a neural network

One of the inputs to my neural network is a set. I have a set $S = \{s_0, s_1, ..., s_n\}$ in which all values $s_i$ are constant. An example of such a set could be the set of French wines (...
2
votes
1answer
712 views

Calculating gradient in a neural net using batches

I am a CS student learning about neural nets. Currently I am confused about how to train a neural net in batches. If I calculate error in a batch, I will get a vector of errors e.g. real1 - predicted1,...
2
votes
1answer
495 views

Where I can I find Pre Trained CNN Datasets for Facial Emotion Recognition?

I am working on a project that involves recognising emotion from images. There are two parts in the project: One where I will generate features from images using API's and then classify the emotion ...
2
votes
1answer
34 views

Change in Training Accuracy

I am solving a problem in the Kaggle learn section : https://www.kaggle.com/c/facial-keypoints-detection The problem involves detecting facial keypoints in a 96x96 image, some 30 features are ...
2
votes
1answer
355 views

How does a recurrent connection in a neural network work?

I am reading a very interesting paper on genetic algorithms which define neural networks. I am familiar with how a feedforward neural network operates, but then I came across this: Where node #4 ...
2
votes
1answer
607 views

Role of computational power in recent AI developments

Today Google's AI won its first game of Go against Lee Sedol, one of the best Go players on the planet. Image interpretation and self-driving cars are other recent success stories in machine learning. ...
2
votes
1answer
985 views

Differences between linear/nonlinear vs. deterministic/nondeterministic neural nets

When speaking of neural networks, I don't get the difference between nonlinear and non-deterministic. Basically, both say that the output of something is not directly correlated to the input? Hope ...
2
votes
1answer
110 views

How do I “tell” a simple perceptron/adaline neural netrork its output can't be negative?

I have made and trained a simple neural network which now seems to produce outputs reasonable in all the aspects but one: it gives negative values from time to time even though the outputs are always ...
2
votes
2answers
303 views

Neural Network Normalization and de-Normalisation of data

I am developing a simple backprop neural network with n inputs and 1 output. I am using a sigmoid activation function. [Aforge.Net] I have read that it is good to normalise the input and output data ...