Network structure inspired by simplified models of biological neurons (brain cells). Neural networks are trained to "learn" by supervised and unsupervised techniques, and can be used to solve optimization problems, approximation problems, classify patterns, and combinations thereof.

learn more… | top users | synonyms (1)

0
votes
0answers
10 views

What is the difference of temporal dynamics in RNNs and the NEF

Both spiking neural networks created with the Neural Engineering Framework (NEF) and Recurrent Neural Networks (RNNs) can be connected recurrently to exhibit neural dynamics. What is the difference ...
-1
votes
0answers
17 views

TrueNorth As a GPU [on hold]

IBM made a brain inspired chip that will do the job of neural network I'm not a professional in neural networks but i understand that it takes inputs map them to there desired outputs with ...
0
votes
0answers
11 views

Neural Network, chain rule to calculate error

I've been reading a book on neural networks: Neural Networks and Deep Learning and I am puzzled by the way the author derives a specific formula. In Chapter two where he talks about relating the error ...
1
vote
0answers
21 views

How to calculate activation of hidden nodes in a recurrent neural network?

Usually, when I program recurrent neural networks, I use a loop for each neuron to figure out it's state. What I realized with this is that in this case, no neuron gets any feedback. They just pump ...
2
votes
1answer
19 views

Should activation function be monotonic in neural networks?

A lot of activation functions in neural networks (sigmoid, tanh, softmax) are monotonic, continuous and differentiable (except of may be a couple of points, where derivative does not exist). I ...
3
votes
0answers
51 views

The runtime of a neural net with given numbers of observations, features, and neurons

If I have $n$ training observations, $m$ number of features per observation, and my neural network has $x$ neurons in the 1st layer, $y$ neurons in the 2nd layer, and 1 output neuron, what is the ...
0
votes
0answers
11 views

Reference for Elliptical Basis Function Neural Networks

I found the following assertion in a neural networks FAQ: Radial networks typically have only one hidden layer, but it can be useful to include a linear layer for dimensionality reduction or ...
5
votes
1answer
28 views

The meaning of discount factor on reinforcement learning

After reading of the google deepmind achievements on Atari's games, I am trying to understand the q-learning and q-networks, but I am little bit confused. The confusion arise in the concept of the ...
0
votes
2answers
92 views

Train Neural network with infinite amount of data [closed]

Does a sufficiently complex neural network guarantee to find the optimal solution, given an infinite amount of data and the back propagation technique for training? In other words, given an infinite ...
4
votes
1answer
48 views

Which matrix of Q values is being used here?

This question refers to this paper: Using Free Energies to Represent Q-values in a Multiagent Reinforcement Learning Task In section 2.1, equations (5) and (6), I am wondering which Q values are ...
1
vote
0answers
30 views

How can neural networks learn to create new things (sentences for example)? [closed]

I have already taken a college course at my uni on machine learning where we implemented all the basic ML programs: linear regression, logistic regression, basic neural network with logistic ...
1
vote
0answers
50 views

How to select the maximum weight value for a bias node in a neural network?

I'm programming a neural network. I know that I should initialize the network by picking random weights. How do I pick a random weight for the connections to bias nodes? What distribution should I ...
1
vote
0answers
41 views

What is the activation function, label and loss function for Hierachical Softmax

Several papers(1 (originator), 2, 3) suggest the use of Hierachical Softmax instead of softmax for classification where the number of classes is large (eg many thousand). I haven't been able to get ...
0
votes
0answers
9 views

Using fibered neural networks to approximate any polynomial

I'm reading the paper Fibring Neural Networks by Artur S. d’Avila Garcez and Dov M. Gabbay, and in it they construct a fibered neural-network that can learn any polynomial. It looks like this: For ...
0
votes
0answers
24 views

Lattice of a fuzzy number?

Can somebody define what is a fuzzy lattice? How to compute lattice of a fuzzy number? Please try to be generic and basic in the explanation as I'm a beginner in studying fuzzy theory and soft ...
0
votes
1answer
46 views

Tackling overlearning neural network issue

I'm trying to train a neural network using this sort of data (for a homework): Number of Features : 42, Target data : 0 or 1, Number of Samples : 111 Individuals ( 69 Cases + 42 Controls ) However ...
0
votes
0answers
24 views

about perceptron criterion function and geometric interpretation

I have found the following problem: I suppose that the value of the J(w) is zero for correct classified examples because the dot product of the two vectors will be orthogonal. So I can assume that ...
2
votes
2answers
58 views

Why can Multilayer neural networks solve non-linear problems

I understand what a multilayer neural network is, but what about them allows them to solve non-linear problems unlike perceptrons? Is it the fact that they can extend to any number of outputs/hidden ...
0
votes
0answers
13 views

Which algorithms are suitable for detecting biological pattern formations?

I'm looking for approaches to automatic detection of biological pattern formations in digital images, similar to those displayed on this wikipedia page: Patterned vegetation The framework i'm working ...
1
vote
1answer
52 views

Differences between linear/nonlinear vs. deterministic/nondeterministic neural nets

When speaking of neural networks, I don't get the difference between nonlinear and non-deterministic. Basically, both say that the output of something is not directly correlated to the input? Hope ...
-1
votes
2answers
60 views

Different weight sets for same problem?

Considering the case that we have a fixed set of training examples and a fixed ANN (i.e same number of input,output and intermediate layers). Is it possible that there exists more than one set of ...
1
vote
0answers
41 views

Computer vision training procedures: SVM/AdaBoost vs Neural Networks

With SVM, adaboost or similar alogrithms, image training sets must be cropped with specific constraints (keep image cropping ratio the same, have object tightly cropped, same resolution) In general, ...
1
vote
1answer
24 views

Creating a single layer perceptron for the OR problem

I am working on the following problem Find the linear least squares unit weights for the `OR' problem, ie. $v_1^T = (0,0), v_2^T = (1,0), v_3^T = (0,1), v_4^T = (1,1)$ and $u_1 = 0, u_2 = u_3 = u_4 = ...
1
vote
1answer
44 views

Updating connections weights in neural networks

I am learning about neural networks and have a couple of things I don't understand. Firstly, in competitive learning I understand that only the neuron with the strongest output is reinforced. That is ...
2
votes
0answers
35 views

How to select topology for neural network?

I was given a target function to design a neural network and train: $y = (x_1 \wedge x_2) \vee (x_3 \wedge x_4)$ The number of inputs and outputs seems obvious (4 and 1). And the training data can ...
2
votes
1answer
35 views

weights in a simple neural network

I have seen that in the material made by Andrew Ng about neural networks, he uses the following weights: so when I replace the final values of h_theta(x) in the formula: I got values near 0 and ...
1
vote
0answers
54 views

Hessian-Free instead of LSTM for Recurrent Net Machine Translation

Last year, Ilya Sutskever and collaborators came out with a paper about a recurrent LSTM net that learns sequence to sequence mappings for machine translation. It's somewhat surprising that the ...
4
votes
1answer
307 views

Genetic Algorithm, Neural Network, Deep Learning, Machine Learning Similarities and Applications? [closed]

I am a computer engineering student and trying to get the idea behind all these Artificial Intelligence Concepts and applications. I know little theoretically about machine learning and some high ...
2
votes
2answers
339 views

ANN - Backpropagation with multiple output neurons

Can I utilize the backpropagation algorithm in a layered, feed-forward ANN in instances where there are multiple output neurons? If so, how? Links to (somewhat) comprehensible resources would be ...
1
vote
0answers
13 views

How do NEF networks differ from ANNs in applications?

As I've described here, the Neural Engineering Framework (NEF) has some functional similarities to Artificial Neural Networks (ANNs). Naturally, there is also an overlap in applications between ...
0
votes
0answers
15 views

Is current hardware adequate for neural networks ? Are there more adequate hardware?

If you have a large neural network and you use more than 10 cores, it will be limited by the fact each core will need to read/write data that it can't access fast enough. I've read about some samsung ...
1
vote
0answers
31 views

backpropagation algorithm seems to be forcing output values to middle than extremes

I have been playing around with artificial neural networks lately, specifically with the prospect of trying to replace the contrastive divergence algorithm with some type of evolutionary metaheuristic ...
3
votes
1answer
2k views

How does the momentum term for backpropagation algorithm work?

When updating the weights of a neural network using the backpropagation algorithm with a momentum term, should the learning rate be applied to the momentum term as well? Most of the information I ...
0
votes
1answer
27 views

ANN architectures that perform faster/efficient prediction over time?

ANN when compared to biological neural systems have this common concept of reducing error in prediction over time (training) and becoming more good at predicting correctly. But there is one behavior ...
5
votes
1answer
139 views

How exactly do you calculate the hidden layer gradients in the backpropagation algorithm?

I have been going through the description of the backpropagation algorithm found here. and I am having a bit of trouble getting my head around some of the linear algebra. Say I have a final output ...
0
votes
0answers
15 views

Can HTM CLA be applied to the problem of autonomous navigation of bots or drones?

See: http://numenta.org/resources/HTM_CorticalLearningAlgorithms.pdf Hierarchical Temporal Memory (HTM) is a detailed computational theory of the neocortex. At the core of HTM are time-based learning ...
1
vote
1answer
32 views

are the activations of hidden nodes in an ANN binary or real valued?

this may seem to be a pretty basic question, but it is something i have been puzzling over for some time. when calculating the activations of nodes in a hidden layer in an ANN using sigmoid neurons ...
1
vote
1answer
45 views

Neural Network Weights per input nodes

Sorry, I'm really new to neural networks and this question is probably pretty obvious. If you have any resource that can help clarify these concepts to me it would be much appreciated. The way I ...
1
vote
0answers
143 views

Improving MSE as fitness function for a genetic algorithm

I am implementing an autoencoder neural network in matlab, the weights of which are being optimised by a genetic algorithm. At the moment I am working on the first layer, trying to get an improved ...
3
votes
1answer
120 views

Disadvantages to using simple step functions for activation in neural networks?

From what I have read, the main advantage to using tanh(x) or sigmoid(x) as an activation function for neural networks is that it is very easily differentiable. I am trying to implement a neural ...
0
votes
1answer
212 views

How to reconstruct the image from a neural network output?

I am trying to use the genetic algorithm to optimise a multi-layered neural network for image classification (i am using a subset of the MNIST handwritten digit data set as my initial dataset, but ...
2
votes
2answers
91 views

Is it possible to implement a Neural Network using a graph data structure?

I'm trying to implement a feedforward neural network using a graph. The thing is: I haven't found any example in which is used a graph data structure. So far the examples I've found used arrays. Can ...
2
votes
5answers
320 views

Machine Learning and Neural Networks for High School Students

I hope this question is appropriate for this forum. In this summer I am giving a 3-day workshop on machine learning and neural networks for advanced and very enthusiastic high school students which ...
1
vote
3answers
101 views

Are neural networks dynamical systems?

Dynamical systems are those whose evolution can be described by a rule, evolves with time and is deterministic. In this context can I say that Neural networks have a rule of evolution which is the ...
2
votes
1answer
52 views

Progressive discrete multifunction optimization

I have a set of functions $F$. The functions effectively take a set $S$ that is always a subset of a global set of all possible values $G$, where $|G|>1000$. (alternatively, they could take a ...
2
votes
1answer
72 views

Closed form solution for a single layer linear perceptron

Let f be a one-layer neural network which is linear (ie. no activation function). Let it have $p$ inputs and $q$ outputs. These are fully connected by weights $W$. We have $n$ inputs $x \in ...
0
votes
0answers
59 views

Where to start studying about HTM?

I am looking for references (pedagogic and beginner friendly!) to these two topics, hierarchical temporal memory algorithms applied to deep planning problems (multi-layer) neural networks trained ...
3
votes
1answer
33 views

Training given pairs of similar values, not labels

I have pairs of "similar" values $(x_i, y_i)$ drawn from a space $x_i, y_i \in S$, and want to train a neural network $N$ such that $N(x_i)$ would be "close" to $N(y_i)$ for all $i$, yet, to make it ...
1
vote
1answer
30 views

Matching training with Neural Network

I have a matching algorithm that is based on making an comparison score. This score is divided into parts. Example: 5 - Points for attributes (lets say they have 3 common attributes, would the score ...
-2
votes
1answer
74 views

Neural Network Design Challenge

i'm studying for PHD Entrance Exam on Stanford. one of previous material exam designed very challenging. i want to design a NN for classifying following 2-class problem. 1) output should be -1 or ...