Network structure inspired by simplified models of biological neurons (brain cells). Neural networks are trained to "learn" by supervised and unsupervised techniques, and can be used to solve optimization problems, approximation problems, classify patterns, and combinations thereof.

learn more… | top users | synonyms (1)

0
votes
0answers
33 views

Jacobian elements calculation for MLP(multiple outputs) with modified Backpropagation (delta rule)

Good afternoon! Fig. 1 2 layer MLP network with 1 hidden layer I am currently calculating Jacobian Matrix elements (partial derivatives of Error from particular output with respect to weights) for ...
3
votes
0answers
32 views

Flaw with Cross Entropy Error in Neural Networks

I've recently been working on creating a neural network to classify handwritten digits. I implemented 1-of-N encoding such that there are the same number of output nodes as possible digits (The ...
3
votes
2answers
39 views

What restrictions apply to query and target vector encoding on fast-forward neural networks?

I'm currently studying fast-forward multi-layer neural networks with back propagation, in the book I see that all query and target vectors are binary-encoded, this makes me believe that this is the ...
0
votes
0answers
21 views

Perceptron XOR calculation

I have following perceptron for the XOR-problem. I'm do not understand the perceptron or my calculations are wrong or both. My thought process: f.e. A is true and B is false. The function I use ...
1
vote
0answers
23 views

What are the most desirable properties of a neural network? [closed]

I'm trying to compare a custom neural network architecture with other existing ones. I'm quite new to the CS field and I'm looking for desirable properties and/or applications of neural ...
1
vote
2answers
31 views

What are the advantages of online learning when training neural networks?

Stochastic gradient descent with a batch size of 1 is apparently used to learn from single examples as they arrive, but I don't understand why you would use such a small batch size instead of batching ...
1
vote
1answer
61 views

Can an artificial neural network convert from cartesian coordinates to polar coordinates?

Given cartesian coordinates $x$ and $y$ as input, can a neural network output $r$ and $\theta$, the equivalent polar coordinates? This would seem to require an approximation of the pythagorean ...
1
vote
1answer
26 views

How is the Delta Rule derived in neural networks and what is the explanation for the algebra?

I am currently trying to learn how the Delta Rule works in a neural network. So far I completely understand the concept of the Delta Rule, but the derivation doesn't make sense. My question is how is ...
0
votes
1answer
20 views

Backpropagation on a matrix of functions

As nicely described in this article, backpropagation for multi-layer perceptrons defines the error term for a neuron in terms of the partial derivative of the weights. It's traditional to represent ...
1
vote
0answers
29 views

Text data comparison

Okay lets say i have two data structures . two phone data for example containing their Name and spec ( cpu , ram , display etc ) . I want to check if these two phones are the same or not . Their names ...
1
vote
2answers
48 views

Large number of layers in a neural network?

How large can the number of layers in a neural network be? Can there be maybe 1000 layers with, say, 1000 neurons in each layer? I imagine the brain has hundreds of thousands of layers.
0
votes
1answer
17 views

Genetic Algorithm Neural Network- Inputs for evolving creatures

I made a GANN program for evolving creatures. The genes that get put into the GA for each ind are the weights that go into the neural net for each creature. The NN is a basic 1 hidden layer NN with ...
0
votes
0answers
9 views

Is it possible to retrieve element in a matrix which was trained using Neural Network?

I just studied neural networks and know the basics. Problem: A matrix is trained using neural network (by giving position of an element and the element) One should get the element at (i,j)th position ...
1
vote
1answer
31 views

How can I compare two different neural networks, from a theorical point of view?

Let's say I have a problem (i.e. Given f(x), find x) and two neural networks(i.e. feedforward and recurrent). I would like to know if one works better than the other one. I could run the twos on a ...
1
vote
1answer
479 views

Convolutional Neural Network Example in Tensorflow

I want to ask the dimension change in different convolution and max-pooling layer. I am referring to the example in TensorFlow tutorial: ...
0
votes
0answers
9 views

How might one go about building neural network used to make a numerical forecast?

I've been doing some reading on the Interwebs about neural networks, and I still can't exactly wrap my head around this idea. I get that a neural network basically replicates the human brain, and ...
1
vote
1answer
77 views

How do neural networks learn concepts?

I've been learning neural networks and some back propagation stuff, and I heard about google's Tensorflow and how it could learn things like how to carry on conversations. It got me thinking about how ...
0
votes
0answers
13 views

Multilayer perceptron, adaptive learning with momentum

So I am experimenting with different configurations of multilayer perceptrons in Matlab and my training data are extracted from images which I want to classify. -I am currently using adaptive ...
5
votes
1answer
126 views

Train a neural network to play tic tac toe using a genetic algorithm

I have an assignment for school, in which I have to build a neural network that will play tic tac toe, using genetic algorithms for training. The thing is that I am clueless on how to connect the two. ...
4
votes
3answers
100 views

How are Neural Networks made so general?

After reading this blog about Deep Neural Networks learning about selfies I'm struck by how generic the network in question is. In short: I'm thinking of trying to write something vaguely similar for ...
1
vote
0answers
28 views

Language grammar correction with supervised learning

I want to work on automatic grammar correction using machine learning (possibly using recurrent or deep neural networks). The algorithm will be supplied with both corrected and initial documents for ...
1
vote
0answers
38 views

Teaching perceptrons colors? [closed]

I am learning about artificial neural networks and I've decided to go with perceptrons. I already made a sample program that can learn based on the learning data, but when I tried to make it recognize ...
2
votes
0answers
9 views

What would happen if a multi-layer perceptron is insufficient to learn a function?

Suppose that something is wrong with its configuration (ie topology), for example there are too few neurons in a certain layer, or not enough layers. I have an intuition that some neurons' "delta" ...
2
votes
1answer
64 views

Approximating sign function by tanh in Multi Layer perceptrons

This is an exercise problem from the book "Learning from Data". Given $w_1$ and $\epsilon \gt 0$, find $w_2$ such that, $$\lvert \mbox{sign}(w_1^Tx_n)-\tanh(w_2^Tx_n)\rvert \le \epsilon$$ Where ...
0
votes
2answers
66 views

Backpropagation for 'Classification' Neural Network

Question: How does one formulate a back propagation algorithm (either batch, gradient, anything that works) for a neural net, playing a game of Tic Tac Toe? (Java is being utilized) Scenario: There ...
2
votes
0answers
44 views

Using the random forest algorithm to predict vectors [duplicate]

I know this might sound like a newbie question, but bear with me. I have read a paper where researchers use a random forest to predict species distribution, but in their study, they only predict a ...
4
votes
3answers
732 views

Evolving artificial neural networks for solving NP problems

I've recently read a really interesting blog entry from Google Research Blog talking about neural network. Basically they use this neural networks for solving various problems like image recognition. ...
6
votes
1answer
53 views

The theory behind backpropogation, gradient calculation

I am reading a book about Neural Networks, that explains the backpropagation algorithm. It explains that the error function is the sum of error of all inputs, and that the algorithm minimizes this ...
1
vote
1answer
83 views

What kind of Neural Network (if any) could fit two sets of data points?

I have two datasets, one of animal migration patterns (collected over the course of a couple years) that consists of many points on an x, y plane (latitude, longitude), and the other of ocean surface ...
4
votes
1answer
188 views

Google Deep Dream has these understandings?

From my own exploration of Google Deep Dream, largely with Dreamify for IOS, and also from Google Image results on the topic, a few of the images I have produced/seen have led me to 3 conclusions ...
3
votes
1answer
43 views

rationale behind multi-layer networks

I recently read that a feed forward neural network with a single hidden layer can represent any continuous function to great precision (ref.). Then how can we justify adding more that one hidden layer ...
0
votes
0answers
10 views

Memory storage capacity of Bienenstock-Cooper-Munro rule

I would like to know the memory storage capacity of the BCM learning rule when it is implemented on a Hopfield network. I understand that it will be a function of n where n is the number of neurons.
0
votes
1answer
22 views

What is the difference of temporal dynamics in RNNs and the NEF

Both spiking neural networks created with the Neural Engineering Framework (NEF) and Recurrent Neural Networks (RNNs) can be connected recurrently to exhibit neural dynamics. What is the difference ...
0
votes
0answers
19 views

Neural Network, chain rule to calculate error

I've been reading a book on neural networks: Neural Networks and Deep Learning and I am puzzled by the way the author derives a specific formula. In Chapter two where he talks about relating the error ...
1
vote
0answers
23 views

How to calculate activation of hidden nodes in a recurrent neural network?

Usually, when I program recurrent neural networks, I use a loop for each neuron to figure out it's state. What I realized with this is that in this case, no neuron gets any feedback. They just pump ...
2
votes
1answer
90 views

Should activation function be monotonic in neural networks?

A lot of activation functions in neural networks (sigmoid, tanh, softmax) are monotonic, continuous and differentiable (except of may be a couple of points, where derivative does not exist). I ...
3
votes
0answers
86 views

The runtime of a neural net with given numbers of observations, features, and neurons

If I have $n$ training observations, $m$ number of features per observation, and my neural network has $x$ neurons in the 1st layer, $y$ neurons in the 2nd layer, and 1 output neuron, what is the ...
0
votes
0answers
21 views

Reference for Elliptical Basis Function Neural Networks

I found the following assertion in a neural networks FAQ: Radial networks typically have only one hidden layer, but it can be useful to include a linear layer for dimensionality reduction or ...
5
votes
1answer
44 views

The meaning of discount factor on reinforcement learning

After reading of the google deepmind achievements on Atari's games, I am trying to understand the q-learning and q-networks, but I am little bit confused. The confusion arise in the concept of the ...
0
votes
2answers
132 views

Train Neural network with infinite amount of data [closed]

Does a sufficiently complex neural network guarantee to find the optimal solution, given an infinite amount of data and the back propagation technique for training? In other words, given an infinite ...
4
votes
1answer
52 views

Which matrix of Q values is being used here?

This question refers to this paper: Using Free Energies to Represent Q-values in a Multiagent Reinforcement Learning Task In section 2.1, equations (5) and (6), I am wondering which Q values are ...
1
vote
0answers
57 views

How can neural networks learn to create new things (sentences for example)? [closed]

I have already taken a college course at my uni on machine learning where we implemented all the basic ML programs: linear regression, logistic regression, basic neural network with logistic ...
1
vote
0answers
60 views

How to select the maximum weight value for a bias node in a neural network?

I'm programming a neural network. I know that I should initialize the network by picking random weights. How do I pick a random weight for the connections to bias nodes? What distribution should I ...
2
votes
0answers
128 views

What is the activation function, label and loss function for Hierachical Softmax

Several papers(1 (originator), 2, 3) suggest the use of Hierachical Softmax instead of softmax for classification where the number of classes is large (eg many thousand). I haven't been able to get ...
0
votes
0answers
14 views

Using fibered neural networks to approximate any polynomial

I'm reading the paper Fibring Neural Networks by Artur S. d’Avila Garcez and Dov M. Gabbay, and in it they construct a fibered neural-network that can learn any polynomial. It looks like this: For ...
0
votes
0answers
34 views

Lattice of a fuzzy number?

Can somebody define what is a fuzzy lattice? How to compute lattice of a fuzzy number? Please try to be generic and basic in the explanation as I'm a beginner in studying fuzzy theory and soft ...
0
votes
1answer
63 views

Tackling overlearning neural network issue

I'm trying to train a neural network using this sort of data (for a homework): Number of Features : 42, Target data : 0 or 1, Number of Samples : 111 Individuals ( 69 Cases + 42 Controls ) However ...
0
votes
0answers
43 views

about perceptron criterion function and geometric interpretation

I have found the following problem: I suppose that the value of the J(w) is zero for correct classified examples because the dot product of the two vectors will be orthogonal. So I can assume that ...
2
votes
2answers
92 views

Why can Multilayer neural networks solve non-linear problems

I understand what a multilayer neural network is, but what about them allows them to solve non-linear problems unlike perceptrons? Is it the fact that they can extend to any number of outputs/hidden ...
0
votes
0answers
13 views

Which algorithms are suitable for detecting biological pattern formations?

I'm looking for approaches to automatic detection of biological pattern formations in digital images, similar to those displayed on this wikipedia page: Patterned vegetation The framework i'm working ...