Questions tagged [neural-networks]

Network structure inspired by simplified models of biological neurons (brain cells). Neural networks are trained to "learn" by supervised and unsupervised techniques, and can be used to solve optimization problems, approximation problems, classify patterns, and combinations thereof.

Filter by
Sorted by
Tagged with
4
votes
2answers
1k views

Z-score relations to perceptrons

I am learning about preceptrons and my professor noted that z-scores are a commmon pre-processing step to normalizing input variables. Following this, I am having trouble understanding why z-scores ...
4
votes
3answers
828 views

Large number of layers in a neural network?

How large can the number of layers in a neural network be? Can there be maybe 1000 layers with, say, 1000 neurons in each layer? I imagine the brain has hundreds of thousands of layers.
4
votes
2answers
378 views

Why is the manifold hypothesis true?

The manifold hypothesis is the statement that real-world high dimensional data (such as images) lie on low-dimensional manifolds embedded in the high-dimensional space. It has been tested to be true ...
4
votes
1answer
282 views

How do neural networks create results like its inputs?

I understand basic neural networks (input layer, hidden layers, output layer) and gradient descent learning. However I keep hearing about news talking about neural networks painting and making jazz ...
4
votes
3answers
276 views

How are Neural Networks made so general?

After reading this blog about Deep Neural Networks learning about selfies I'm struck by how generic the network in question is. In short: I'm thinking of trying to write something vaguely similar for ...
4
votes
2answers
1k views

How can I compare two different neural networks, from a theorical point of view?

Let's say I have a problem (i.e. Given f(x), find x) and two neural networks(i.e. feedforward and recurrent). I would like to know if one works better than the other one. I could run the twos on a ...
4
votes
1answer
110 views

Are genetic algorithms an effective way to train neural networks?

It seems to me that genetic algorithms would be an ideal way to train neural networks so that they come to have the right weights, since they are especially good at escaping local minima, and ...
4
votes
1answer
1k views

Why update weights and biases after training a Neural Network on whole set of training samples

I am reading the book Neural Networks and Deep Learning by Micheal Nielsen. In the second chapter of his book, he describes the following algorithm for updating weights and biases for a neural ...
4
votes
2answers
6k views

ANN - Backpropagation with multiple output neurons

Can I utilize the backpropagation algorithm in a layered, feed-forward ANN in instances where there are multiple output neurons? If so, how? Links to (somewhat) comprehensible resources would be ...
4
votes
1answer
90 views

How are two layer feed-forward neural networks universal?

Across my studies I have noticed the following statement in my Subject Guide; namely, that two-layer feed-forward neural networks using the sigmoidal activation function are universal. My question is ...
4
votes
1answer
1k views

Genetic Algorithm, Neural Network, Deep Learning, Machine Learning Similarities and Applications? [closed]

I am a computer engineering student and trying to get the idea behind all these Artificial Intelligence Concepts and applications. I know little theoretically about machine learning and some high ...
4
votes
1answer
286 views

How do RNN's handle providing output with different dimension than input

It seems like an RNN has to have the ht-1 needs to be the same size as the input vector since they're being added to one another, but if you're doing something like modeling to another language or ...
4
votes
1answer
101 views

Which matrix of Q values is being used here?

This question refers to this paper: Using Free Energies to Represent Q-values in a Multiagent Reinforcement Learning Task In section 2.1, equations (5) and (6), I am wondering which Q values are ...
4
votes
1answer
40 views

Do all the cells in a recurrent neural network share learned parameters?

Most descriptions of modern RNNs present a "folded" characterisation, that is to say, a single cell with a loop back to itself transmitting the hidden state from one step to the next. However, in ...
4
votes
1answer
43 views

Given a Noisy Curve, Write a Function to Output Likely Slopes

I am training a Variational Autoencoder (type of convolutional neural network), and have been plotting cost over time. The result is a noisy curve, shown here: I would like to write a function that ...
4
votes
1answer
404 views

How to choose proper activation functions for hidden and output layers of a perceptron neural network?

As far as I know choosing an activation function for the input layer is relatively straightforward: I use Sigmoid if the input data domain is (0,1) and TANH if it is (-1,1). But what activation ...
4
votes
0answers
20 views

How to represent sentences with their dependency parses as input to an RNN?

I am working on a task embedding sentences into a lower-dimensional space according to style, both grammatical and lexical. As such, I want to have as input the linear ordering of tokens in each ...
3
votes
6answers
2k views

Machine Learning and Neural Networks for High School Students

I hope this question is appropriate for this forum. In this summer I am giving a 3-day workshop on machine learning and neural networks for advanced and very enthusiastic high school students which ...
3
votes
2answers
151 views

Finding Feature Representation Such That Two Samples Are Similar in Feature Space

Consider one specific useful function of our human brain: abstraction of object. Take the example of two pictures: if we are told the pictures are similar, we actually make conclusion about the ...
3
votes
2answers
103 views

Why neural nets are split into layers?

Why is every artificial neural network layered? Why isn't each node just a separate process?
3
votes
2answers
4k views

What are the advantages of online learning when training neural networks?

Stochastic gradient descent with a batch size of 1 is apparently used to learn from single examples as they arrive, but I don't understand why you would use such a small batch size instead of batching ...
3
votes
1answer
70 views

rationale behind multi-layer networks

I recently read that a feed forward neural network with a single hidden layer can represent any continuous function to great precision (ref.). Then how can we justify adding more that one hidden layer ...
3
votes
1answer
52 views

Why do people insist to use the term “multilayer perceptron” instead of “multilayer perceptron network”?

The perceptron model describes a linear classifier. Often people use the term "multilayer perceptron" to describe a feedforward neural network that uses perceptrons. This terminology simply sounds ...
3
votes
1answer
54 views

How can image processing neural networks be effectively trained?

I was just thinking about image processing neural networks and how to effectively train them in regard of the available dataset. Let's say you'd want to build a neural network which can distinguish ...
3
votes
1answer
209 views

What method of collective recogintion to use for digits recognition?

The structure of the question is as follows: at first, I provide the concept of collective recognition, further I provide explanation of the various methods of group classification that I found, in ...
3
votes
1answer
394 views

Why updating only a part of all neural network weights does not work?

I am having a problem with my program of deep neural network using Theano. In my deep neural network, I have several layers of neural network to predict an output given a certain input. Because of an ...
3
votes
1answer
360 views

Convolutional Neural Network Feature Engineering?

I'm working through the tensorflow tutorial, and I see how you go from 28 x 28 to zero-padding and applying a 5x5x32 convolution to get 28x28x32 and max-pooling etc. What I'm confused about is the 32 ...
3
votes
2answers
151 views

What restrictions apply to query and target vector encoding on fast-forward neural networks?

I'm currently studying fast-forward multi-layer neural networks with back propagation, in the book I see that all query and target vectors are binary-encoded, this makes me believe that this is the ...
3
votes
2answers
682 views

Why can Multilayer neural networks solve non-linear problems

I understand what a multilayer neural network is, but what about them allows them to solve non-linear problems unlike perceptrons? Is it the fact that they can extend to any number of outputs/hidden ...
3
votes
1answer
990 views

Closed form solution for a single layer linear perceptron

Let f be a one-layer neural network which is linear (ie. no activation function). Let it have $p$ inputs and $q$ outputs. These are fully connected by weights $W$. We have $n$ inputs $x \in \mathbb{R}^...
3
votes
1answer
75 views

How do you protect an AI from a human doing “illogical” moves?

Using a monte carlo approach and evalutation function. Some moves will deemed to be more advantageous than others. As a computer plays itself, it will generally go for the best moves possible. And ...
3
votes
1answer
53 views

Applying a 1x1 convolution on an existing layer

I am looking at the following github. In this, layer3 has a shape of (1, 20, 72, 256) and I interpret this is as a single layer ...
3
votes
3answers
48 views

Prove a network is a feedforward network if and only if the numbering of its cells satisfy these conditions

Undergraduate math student here attempting to understand neural networks. Picked up a text on sale, "Neural Network Learning and Expert Systems" (Gallant), and I'm just starting on the exercises for ...
3
votes
1answer
79 views

Are Perceptrons the neural network equivalence of Linear and Logistic Regression?

am I right in the assumption that both linear and logistic regression algorithms can be represented as the simplest form of neural networks,a perceptron, which consists of a two layers, an Input and ...
3
votes
2answers
248 views

Language Classification + AWS ML: what am I doing wrong?

I'm evaluating Amazon's machine learning platform, and thought that I would give it a "simple" classification problem first. As a disclaimer, I am quite new to machine learning (hence my interest in ...
3
votes
1answer
86 views

Confusion regarding terminology surrounding perceptron learning

I am having trouble understanding the terminology with perceptron learning. Is my current understanding correct? Let's say I have some data that classifies what type of flower a particular flower is. ...
3
votes
1answer
3k views

Batch regularization & L2 regularization

Is performing batch regularization in addition to L2 regularization redundant? Batch regularization: http://arxiv.org/abs/1502.03167 Notice how when performing batch regularization, you forgo using ...
3
votes
1answer
1k views

Disadvantages to using simple step functions for activation in neural networks?

From what I have read, the main advantage to using tanh(x) or sigmoid(x) as an activation function for neural networks is that it is very easily differentiable. I am trying to implement a neural ...
3
votes
1answer
49 views

Neural net that alters its own parameters

Is it possible for an AI neural net to be able to modify its own parameters/hyperparameters and/or add new parameters/hyperparameters? If so, how has it been implemented? For example, to simulate a ...
3
votes
1answer
220 views

Use machine learning algorithms data structure to store information (memory)

DISCLAIMER : I'm pretty new to machine learning field, so forgive me if my questions are somehow naive ... I already searched on Internet about this topic and find nothing interesting so I am ...
3
votes
1answer
2k views

What are the drawbacks of fully-convolutional neural networks?

It is possible to replace the fully-connected layers of a CNN with convolutional layers, making it fully convolutional. Fully-convolutional networks (FCNs) can be applied to inputs of various sizes, ...
3
votes
1answer
606 views

Why does the neural network logistic regression cost function sum for all layers only for lambda?

I'm taking Andrew Ng's machine learning course and week 5 covers the training of neural networks. The modified cost function for neural network training is derived from the logistic regression cost ...
3
votes
1answer
1k views

Multiple neural networks or multiple outputs?

Suppose you have data of the form input a matrix A, and output a matrix B, where each row of each is one datapoint. Should you create multiple neural networks, one for each column of B, or one NN with ...
3
votes
1answer
84 views

Classification training data, but regression prediction

Suppose I'm performing machine learning on a simple dataset, and have a bunch of training data of the form: ...
3
votes
2answers
1k views

Can an artificial neural network convert from cartesian coordinates to polar coordinates?

Given cartesian coordinates $x$ and $y$ as input, can a neural network output $r$ and $\theta$, the equivalent polar coordinates? This would seem to require an approximation of the pythagorean ...
3
votes
1answer
1k views

Neural Network weight selection using Genetic Algorithm

Hi I want to ask about weight selection in neural network using genetic algorithm. Right now what I understand is Initialize population Encode the weight of the neural network to the chromosome ...
3
votes
1answer
498 views

How to compare the output of a neural network with his target?

I am coding a neural network implementation, but a I have problems in the design. I was wondering about how to compare the output with the target, my neural networks has three outputs ...
3
votes
1answer
591 views

Analog circuits for neural networks?

Neural networks in machine learning are inherently a continuous model of computation. Yet we use digital logic circuits with floating point numbers to "emulate" this continuity. I am wondering: is ...
3
votes
1answer
110 views

Weird behaviour of softmax derivative?

I have been implementing some neural networks in MATLAB and recently I noticed a weird thing while implementing softmax derivative: Setting the derivative to one, rather than using the actual ...
3
votes
1answer
61 views

Clustered Regions by Each Neuron in Self Organizing Map (SOM)

I was given a question about SOM. There is a SOM which have 4x4 neurons and each neuron's x1 and x2 values (coordinates) given. Also neighborhood function and weight update rule given. How can i ...