Network structure inspired by simplified models of biological neurons (brain cells). Neural networks are trained to "learn" by supervised and unsupervised techniques, and can be used to solve optimization problems, approximation problems, classify patterns, and combinations thereof.

learn more… | top users | synonyms (1)

4
votes
0answers
30 views

Which matrix of Q values is being used here?

This question refers to this paper: Using Free Energies to Represent Q-values in a Multiagent Reinforcement Learning Task In section 2.1, equations (5) and (6), I am wondering which Q values are ...
1
vote
0answers
23 views

How can neural networks learn to create new things (sentences for example)? [closed]

I have already taken a college course at my uni on machine learning where we implemented all the basic ML programs: linear regression, logistic regression, basic neural network with logistic ...
1
vote
0answers
46 views

How to select the maximum weight value for a bias node in a neural network?

I'm programming a neural network. I know that I should initialize the network by picking random weights. How do I pick a random weight for the connections to bias nodes? What distribution should I ...
0
votes
0answers
8 views

Feedforward net lagged prediction [migrated]

I am trying to predict y(t+1) of a function by using a feedforward neural network with Matlab. The inputs are the previous 3 previous values (y(t-2), y(t-1), y(t)) and the training output is the ...
1
vote
0answers
34 views

What is the activation function, label and loss function for Hierachical Softmax

Several papers(1 (originator), 2, 3) suggest the use of Hierachical Softmax instead of softmax for classification where the number of classes is large (eg many thousand). I haven't been able to get ...
0
votes
0answers
9 views

Using fibered neural networks to approximate any polynomial

I'm reading the paper Fibring Neural Networks by Artur S. d’Avila Garcez and Dov M. Gabbay, and in it they construct a fibered neural-network that can learn any polynomial. It looks like this: For ...
0
votes
0answers
23 views

Lattice of a fuzzy number?

Can somebody define what is a fuzzy lattice? How to compute lattice of a fuzzy number? Please try to be generic and basic in the explanation as I'm a beginner in studying fuzzy theory and soft ...
0
votes
1answer
40 views

Tackling overlearning neural network issue

I'm trying to train a neural network using this sort of data (for a homework): Number of Features : 42, Target data : 0 or 1, Number of Samples : 111 Individuals ( 69 Cases + 42 Controls ) However ...
0
votes
0answers
17 views

about perceptron criterion function and geometric interpretation

I have found the following problem: I suppose that the value of the J(w) is zero for correct classified examples because the dot product of the two vectors will be orthogonal. So I can assume that ...
2
votes
2answers
52 views

Why can Multilayer neural networks solve non-linear problems

I understand what a multilayer neural network is, but what about them allows them to solve non-linear problems unlike perceptrons? Is it the fact that they can extend to any number of outputs/hidden ...
0
votes
0answers
13 views

Which algorithms are suitable for detecting biological pattern formations?

I'm looking for approaches to automatic detection of biological pattern formations in digital images, similar to those displayed on this wikipedia page: Patterned vegetation The framework i'm working ...
1
vote
1answer
46 views

Differences between linear/nonlinear vs. deterministic/nondeterministic neural nets

When speaking of neural networks, I don't get the difference between nonlinear and non-deterministic. Basically, both say that the output of something is not directly correlated to the input? Hope ...
-1
votes
2answers
60 views

Different weight sets for same problem?

Considering the case that we have a fixed set of training examples and a fixed ANN (i.e same number of input,output and intermediate layers). Is it possible that there exists more than one set of ...
1
vote
0answers
34 views

Computer vision training procedures: SVM/AdaBoost vs Neural Networks

With SVM, adaboost or similar alogrithms, image training sets must be cropped with specific constraints (keep image cropping ratio the same, have object tightly cropped, same resolution) In general, ...
1
vote
1answer
22 views

Creating a single layer perceptron for the OR problem

I am working on the following problem Find the linear least squares unit weights for the `OR' problem, ie. $v_1^T = (0,0), v_2^T = (1,0), v_3^T = (0,1), v_4^T = (1,1)$ and $u_1 = 0, u_2 = u_3 = u_4 = ...
0
votes
1answer
39 views

Updating connections weights in neural networks

I am learning about neural networks and have a couple of things I don't understand. Firstly, in competitive learning I understand that only the neuron with the strongest output is reinforced. That is ...
2
votes
0answers
34 views

How to select topology for neural network?

I was given a target function to design a neural network and train: $y = (x_1 \wedge x_2) \vee (x_3 \wedge x_4)$ The number of inputs and outputs seems obvious (4 and 1). And the training data can ...
2
votes
1answer
34 views

weights in a simple neural network

I have seen that in the material made by Andrew Ng about neural networks, he uses the following weights: so when I replace the final values of h_theta(x) in the formula: I got values near 0 and ...
1
vote
0answers
47 views

Hessian-Free instead of LSTM for Recurrent Net Machine Translation

Last year, Ilya Sutskever and collaborators came out with a paper about a recurrent LSTM net that learns sequence to sequence mappings for machine translation. It's somewhat surprising that the ...
4
votes
1answer
281 views

Genetic Algorithm, Neural Network, Deep Learning, Machine Learning Similarities and Applications? [closed]

I am a computer engineering student and trying to get the idea behind all these Artificial Intelligence Concepts and applications. I know little theoretically about machine learning and some high ...
2
votes
2answers
301 views

ANN - Backpropagation with multiple output neurons

Can I utilize the backpropagation algorithm in a layered, feed-forward ANN in instances where there are multiple output neurons? If so, how? Links to (somewhat) comprehensible resources would be ...
1
vote
0answers
11 views

How do NEF networks differ from ANNs in applications?

As I've described here, the Neural Engineering Framework (NEF) has some functional similarities to Artificial Neural Networks (ANNs). Naturally, there is also an overlap in applications between ...
0
votes
0answers
15 views

Is current hardware adequate for neural networks ? Are there more adequate hardware?

If you have a large neural network and you use more than 10 cores, it will be limited by the fact each core will need to read/write data that it can't access fast enough. I've read about some samsung ...
1
vote
0answers
31 views

backpropagation algorithm seems to be forcing output values to middle than extremes

I have been playing around with artificial neural networks lately, specifically with the prospect of trying to replace the contrastive divergence algorithm with some type of evolutionary metaheuristic ...
3
votes
1answer
1k views

How does the momentum term for backpropagation algorithm work?

When updating the weights of a neural network using the backpropagation algorithm with a momentum term, should the learning rate be applied to the momentum term as well? Most of the information I ...
0
votes
1answer
27 views

ANN architectures that perform faster/efficient prediction over time?

ANN when compared to biological neural systems have this common concept of reducing error in prediction over time (training) and becoming more good at predicting correctly. But there is one behavior ...
5
votes
1answer
129 views

How exactly do you calculate the hidden layer gradients in the backpropagation algorithm?

I have been going through the description of the backpropagation algorithm found here. and I am having a bit of trouble getting my head around some of the linear algebra. Say I have a final output ...
0
votes
0answers
13 views

Can HTM CLA be applied to the problem of autonomous navigation of bots or drones?

See: http://numenta.org/resources/HTM_CorticalLearningAlgorithms.pdf Hierarchical Temporal Memory (HTM) is a detailed computational theory of the neocortex. At the core of HTM are time-based learning ...
1
vote
1answer
32 views

are the activations of hidden nodes in an ANN binary or real valued?

this may seem to be a pretty basic question, but it is something i have been puzzling over for some time. when calculating the activations of nodes in a hidden layer in an ANN using sigmoid neurons ...
0
votes
0answers
19 views

What math is required for machine learning, neural networks etc.? [duplicate]

I'm planning to start learning more in depth about neural networks and machine learning. Can someone tell me which math will I need the most, and also recommend me good books or internet lessons in ...
1
vote
1answer
43 views

Neural Network Weights per input nodes

Sorry, I'm really new to neural networks and this question is probably pretty obvious. If you have any resource that can help clarify these concepts to me it would be much appreciated. The way I ...
1
vote
0answers
136 views

Improving MSE as fitness function for a genetic algorithm

I am implementing an autoencoder neural network in matlab, the weights of which are being optimised by a genetic algorithm. At the moment I am working on the first layer, trying to get an improved ...
3
votes
1answer
105 views

Disadvantages to using simple step functions for activation in neural networks?

From what I have read, the main advantage to using tanh(x) or sigmoid(x) as an activation function for neural networks is that it is very easily differentiable. I am trying to implement a neural ...
0
votes
1answer
188 views

How to reconstruct the image from a neural network output?

I am trying to use the genetic algorithm to optimise a multi-layered neural network for image classification (i am using a subset of the MNIST handwritten digit data set as my initial dataset, but ...
2
votes
2answers
87 views

Is it possible to implement a Neural Network using a graph data structure?

I'm trying to implement a feedforward neural network using a graph. The thing is: I haven't found any example in which is used a graph data structure. So far the examples I've found used arrays. Can ...
2
votes
5answers
309 views

Machine Learning and Neural Networks for High School Students

I hope this question is appropriate for this forum. In this summer I am giving a 3-day workshop on machine learning and neural networks for advanced and very enthusiastic high school students which ...
1
vote
3answers
98 views

Are neural networks dynamical systems?

Dynamical systems are those whose evolution can be described by a rule, evolves with time and is deterministic. In this context can I say that Neural networks have a rule of evolution which is the ...
2
votes
1answer
52 views

Progressive discrete multifunction optimization

I have a set of functions $F$. The functions effectively take a set $S$ that is always a subset of a global set of all possible values $G$, where $|G|>1000$. (alternatively, they could take a ...
2
votes
1answer
70 views

Closed form solution for a single layer linear perceptron

Let f be a one-layer neural network which is linear (ie. no activation function). Let it have $p$ inputs and $q$ outputs. These are fully connected by weights $W$. We have $n$ inputs $x \in ...
0
votes
0answers
58 views

Where to start studying about HTM?

I am looking for references (pedagogic and beginner friendly!) to these two topics, hierarchical temporal memory algorithms applied to deep planning problems (multi-layer) neural networks trained ...
3
votes
1answer
33 views

Training given pairs of similar values, not labels

I have pairs of "similar" values $(x_i, y_i)$ drawn from a space $x_i, y_i \in S$, and want to train a neural network $N$ such that $N(x_i)$ would be "close" to $N(y_i)$ for all $i$, yet, to make it ...
1
vote
1answer
29 views

Matching training with Neural Network

I have a matching algorithm that is based on making an comparison score. This score is divided into parts. Example: 5 - Points for attributes (lets say they have 3 common attributes, would the score ...
-2
votes
1answer
73 views

Neural Network Design Challenge

i'm studying for PHD Entrance Exam on Stanford. one of previous material exam designed very challenging. i want to design a NN for classifying following 2-class problem. 1) output should be -1 or ...
2
votes
0answers
74 views

How can you use HMMs and ANNs for on-line handwriting recognition?

On-line handwriting recognition is the task of converting a series of $(x(t),y(t))$ coordinates to symbols and words. In contrast to off-line handwriting recognition, where you only have a bitmap of ...
0
votes
0answers
92 views

Backpropagation Through Time Recursive Algorithm

Would it be plausible to write a recursive version of backpropagation through time for recurrent neural network training? I've only found the iterative version: ...
4
votes
0answers
32 views

How to choose proper activation functions for hidden and output layers of a perceptron neural network?

As far as I know choosing an activation function for the input layer is relatively straightforward: I use Sigmoid if the input data domain is (0,1) and TANH if it is (-1,1). But what activation ...
2
votes
1answer
44 views

How do I “tell” a simple perceptron/adaline neural netrork its output can't be negative?

I have made and trained a simple neural network which now seems to produce outputs reasonable in all the aspects but one: it gives negative values from time to time even though the outputs are always ...
1
vote
2answers
101 views

How to determine if a state is a fixed point in a Hopfield network?

I have been reading a lot and I am still unsure of how to determine this. Let's say I have an initial binary state vector (1, 1, 1). How would I go about determining whether (1, 1, 1) is a fixed point ...
-2
votes
2answers
285 views

Perceptron learning rule for classification

That's the problem $$y=(x,w,\rho) = \begin{cases} 1 & \sum_{i=1}^3 w_ix_i >\rho\\ 0 & \text{otherwise} \end{cases},$$ where $x=\{x_1,x_2,x_3\}$ are inputs, $w=\{w_1,w_2,w_3\}$ are ...
1
vote
2answers
86 views

Perceptron learning rule doesn't work [closed]

i'm a little bit lost ... can you help me ? So I have this table of date (each row give a point with its group) So i took a random weight let's say : [1, -2] H = 1 if n =< 0 0 otherwise ...