Questions tagged [neural-networks]
Network structure inspired by simplified models of biological neurons (brain cells). Neural networks are trained to "learn" by supervised and unsupervised techniques, and can be used to solve optimization problems, approximation problems, classify patterns, and combinations thereof.
497
questions
23
votes
4
answers
14k
views
What is the difference between a Neural Network, a Deep Learning System and a Deep Belief Network?
What is the difference between a Neural Network, a Deep Learning System and a Deep Belief Network?
As I recall your basic neural network is a 3 layers kinda thing,
and I have had Deep Belief Systems ...
- 346
4
votes
2
answers
2k
views
How can I compare two different neural networks, from a theorical point of view?
Let's say I have a problem (i.e. Given f(x), find x) and two neural networks(i.e. feedforward and recurrent). I would like to know if one works better than the other one. I could run the twos on a ...
- 72.4k
4
votes
2
answers
2k
views
Z-score relations to perceptrons
I am learning about preceptrons and my professor noted that z-scores are a commmon pre-processing step to normalizing input variables. Following this, I am having trouble understanding why z-scores ...
CommunityBot
- 1
4
votes
3
answers
2k
views
Large number of layers in a neural network?
How large can the number of layers in a neural network be? Can there be maybe 1000 layers with, say, 1000 neurons in each layer?
I imagine the brain has hundreds of thousands of layers.
- 368
3
votes
2
answers
128
views
Why neural nets are split into layers?
Why is every artificial neural network layered? Why isn't each node just a separate process?
- 149
1
vote
1
answer
220
views
When a trained RNN is tested, is the number of time-steps same for every input?
I am a beginner in deep learning so bear with me. If I want to unfold the RNN in order to represent the relation of output to the input as a non-recursive functions I would have to know the number of ...
- 368
4
votes
1
answer
460
views
How do neural networks create results like its inputs?
I understand basic neural networks (input layer, hidden layers, output layer) and gradient descent learning. However I keep hearing about news talking about neural networks painting and making jazz ...
- 368
3
votes
1
answer
602
views
How does a recurrent connection in a neural network work?
I am reading a very interesting paper on genetic algorithms which define neural networks. I am familiar with how a feedforward neural network operates, but then I came across this:
Where node #4 ...
- 368
3
votes
0
answers
72
views
Number of parameters to be optimized in Artificial Bee Colony
I was reading this paper - Software defect prediction using cost-sensitive neural network by Ömer Faruk Arara and Kürsat Ayan
It uses Artificial Bee Colony algorithm to train the neural network. In ...
- 143
0
votes
1
answer
240
views
Comparative study between Deep neural nets and Bayesian Networks
Is there any comparative study that showcases the powers of Bayesian Networks and Deep learning in their respective favorable setup and how they compare?
I tried to go through blogs but couldn't find ...
CommunityBot
- 1
3
votes
1
answer
243
views
What method of collective recogintion to use for digits recognition?
The structure of the question is as follows: at first, I provide the concept of collective recognition, further I provide explanation of the various methods of group classification that I found, in ...
CommunityBot
- 1
0
votes
0
answers
39
views
How would physical neural nets learn?
I understand how neural networks that are implemented in software learn. You simply change the synaptic weights in the program. But how would you get a hardware implementation of neural networks to ...
- 72.4k
0
votes
0
answers
33
views
Something wrong when recognizing data set using Binary Artificial Neural Network
I have learnt Neural Network for a couple weeks. I just met a problem when I tried to use ANN to recognize a dataset.
So I'm gonna describe the problem:
Set A has 15 members which are in set {1, 1.5, ...
2
votes
1
answer
43
views
Change in Training Accuracy
I am solving a problem in the Kaggle learn section : https://www.kaggle.com/c/facial-keypoints-detection
The problem involves detecting facial keypoints in a 96x96 image, some 30 features are ...
D.W.♦
- 159k
3
votes
1
answer
632
views
Why updating only a part of all neural network weights does not work?
I am having a problem with my program of deep neural network using Theano. In my deep neural network, I have several layers of neural network to predict an output given a certain input. Because of an ...
D.W.♦
- 159k
-1
votes
2
answers
1k
views
Perceptron learning rule for classification
That's the problem
$$y=(x,w,\rho) = \begin{cases}
1 & \sum_{i=1}^3 w_ix_i >\rho\\
0 & \text{otherwise}
\end{cases},$$
where $x=\{x_1,x_2,x_3\}$ are inputs, $w=\{w_1,w_2,w_3\}$ are ...
- 101
1
vote
3
answers
792
views
Is it possible to add "memory" to a neural network?
Suppose I have a NN with one hidden layer, 10 inputs and 5 outputs, intended to be used as a (for example) game-bot AI. Would it make any sense to add, say, 5 (insert any number here) more inputs and ...
- 156
4
votes
1
answer
2k
views
Why update weights and biases after training a Neural Network on whole set of training samples
I am reading the book Neural Networks and Deep Learning by Micheal Nielsen. In the second chapter of his book, he describes the following algorithm for updating weights and biases for a neural ...
CommunityBot
- 1
0
votes
0
answers
88
views
DFAs can be encoded as input/output for a neural network?
I would encode DFAs (Deterministic Finite State automata) as output (or input) of a neural network for a supervised learning; it is well-known [1] that efficacy of neural network training strongly ...
D.W.♦
- 159k
1
vote
0
answers
385
views
Noise injection to fight overfitting
I've been researching the specifics of adding noise injection to my input layer to fight against overfitting. It seems as though most papers recommend a Gaussian noise vector with mean of 0 and a "...
- 219
2
votes
0
answers
69
views
How to best model multidimensional, continuous, non-convex "shape" as neural network?
I have:
set of n-dimensional points that I know are inside of the shape
n >= 18, range on all dimensions has upper bound and lower bound (no axis goes to infinity).
shape is pretty large in this n-...
- 131
1
vote
1
answer
373
views
Which neural network topology is the most efficient to generate randomly shaped letters?
I have created some unique shapes, so-called "letters" for a custom alphabet, all of which can fit into 9x9 pixels. Instead of drawing countless more, I try to combine two solutions I saw in a ...
D.W.♦
- 159k
3
votes
1
answer
3k
views
Batch regularization & L2 regularization
Is performing batch regularization in addition to L2 regularization redundant?
Batch regularization: http://arxiv.org/abs/1502.03167
Notice how when performing batch regularization, you forgo using ...
D.W.♦
- 159k
0
votes
0
answers
68
views
Understanding This Graphical Depiction of a Radial-Basis Function Network
Let $f$ be a Radial-Basis Function Network:
$$
f(X) = \sum_{i=1}^N a_i p( \lvert \lvert b_i X - c_i \lvert \lvert)
$$
From Artificial Intelligence for Human Beings, the following depicts $f$:
In ...
- 285
3
votes
1
answer
3k
views
HOG vs. neural networks for person detection
I am very new to computer vision, (a high school student) and I am working on a project to count the number of people present in a room. I have tried to use the HOGDecriptor for person detection ...
1
vote
1
answer
67
views
Optimizing neural network on small training set
I'm in the process of optimizing my neural network. I'd like to optimize on a small training set (1000 rows) as opposed to my full training set (100K rows) for speed reasons.
Will the optimal hyper-...
D.W.♦
- 159k
3
votes
1
answer
94
views
Classification training data, but regression prediction
Suppose I'm performing machine learning on a simple dataset, and have a bunch of training data of the form:
...
D.W.♦
- 159k
2
votes
1
answer
207
views
Time Series Prediction with an LSTM
I have a time series that I want to predict with an LSTM. I am able to get very good results using 50 datapoints predicting 51, but I struggle to get any accuracy using something like 200 datapoints ...
D.W.♦
- 159k
1
vote
1
answer
1k
views
error computation in multi layered perceptron
I was reading about Multi Layered Perceptron(MLP) and how can we learn pattern using it. Algorithm was stated as
Initiate all weight to small values. Compute activation of each
neuron
using sigmoid ...
- 2,360
-1
votes
1
answer
72
views
Is it okay to select the best performers of test cases for scientific publication in neural network machine learning [closed]
If I split my data properly into 75% train, 15% test, and 15% validation, and there are over 100,000 samples, is it appropriate for me to train 100s of neural networks then select only a couple based ...
D.W.♦
- 159k
2
votes
1
answer
4k
views
How to determine the dimensions of the bias vector in a Convolutional layer (tensor flow)?
In the Deep MNIST Tutorial for Tensor Flow, (and in general), we create a convolutional layer with a weight tensor, say W, of shape (patch dims, #input channels, # output channels). However, when we ...
D.W.♦
- 159k
-2
votes
1
answer
152
views
In what way can Google deepdream be extended? Is there an image dataset which can use hallucinations produced? Any deepdream useful application? [closed]
Is there a particular section of image data which when trained on deepdream algorithm and given some input image produce a resulting image from which we can conclude that the deepdream can be used for ...
- 551
0
votes
0
answers
47
views
choosing right neural network architecture and input features
As a short example suppose we have a kind of pollution sensor and a jet fan in a tunnel.
Jet fan turn on/off according to the automatic scenario based on pollution sensor value.
Pollution itself ...
- 1
1
vote
2
answers
7k
views
What's the difference between a rule based system and an artificial neural network?
I'm currently doing some reading into AI and up to this point couldn't find a satisfying answer to this question: what's the difference between a rule based system and an artificial neural network?
...
- 101
5
votes
1
answer
922
views
Is there a "flaw" in the backpropagation algorithm?
While trying to find a better backpropagation algorithm, I came across a paradox in my algorithm and then I found out this also happens in the usual backpropagation algorithm.
Our neural network ...
D.W.♦
- 159k
1
vote
1
answer
2k
views
Difference between SNN RL and DNN RL?
In Reinfrocement Learning (RL) in Neural Networks (NNs), I've seen two approaches to Q-learning.
The first is to tile the state space with basis functions using Spiking Neural Networks (SNN) to ...
- 641
2
votes
1
answer
147
views
Why Isn't This Outlier Score/Reconstruction Error Not Squared?
I was looking through a paper called "AI2
: Training a big data machine to defend", and saw this (http://people.csail.mit.edu/kalyan/AI2_Paper.pdf)
$score(X_{i}) = \sum_{j=1}^{p} (|X_{i} − R^{j}_{i}|)...
CommunityBot
- 1
6
votes
1
answer
2k
views
Train a neural network to play tic tac toe using a genetic algorithm
I have an assignment for school, in which I have to build a neural network that will play tic tac toe, using genetic algorithms for training. The thing is that I am clueless on how to connect the two. ...
CommunityBot
- 1
1
vote
1
answer
368
views
Using a combination of spatial and non-spatial inputs for convolutional neural networks
I'm working on training a game AI using deep reinforcement learning to achieve specific examples based on pixel input and some additional state information.
Naturally, I'm using a convolutional ...
D.W.♦
- 159k
2
votes
1
answer
60
views
Determining if my artificial neural network needs additional layers
I have implemented a neural network for load forecasting in Microsoft Excel. My structure is very simplistic and involves only 1 hidden layer and 4 neurons. (See picture)
I trained my network with a ...
- 207
2
votes
0
answers
98
views
KDD Machine Learning using K-NN Algorithm Classification Problem
I'm trying to solve a classification problem from the KDD cup archive of 2004.
Details can be found here: KDD 2004 Archive
I'm only dong the particle physics part.
The description of dataset is as ...
- 207
0
votes
0
answers
58
views
understanding kohonen self organized feature maps
I was learning self organizing feature maps the other day. I want to intuitively understand it because I'm not that good at math. But I still am not very clear about it. I can easily understand ...
- 72.4k
2
votes
1
answer
453
views
Language grammar correction with supervised learning
I want to work on automatic grammar correction using machine learning (possibly using recurrent or deep neural networks). The algorithm will be supplied with both corrected and initial documents for ...
- 81.7k
6
votes
2
answers
6k
views
How do you determine the inputs to a neural network?
I'm looking at this tutorial on neural networks. The data that is given from the UCI study includes various attributes, such as "mean x of on pixels", "total # on pixels" etc, which are taken as input ...
- 8,091
4
votes
1
answer
361
views
How do RNN's handle providing output with different dimension than input
It seems like an RNN has to have the ht-1 needs to be the same size as the input vector since they're being added to one another, but if you're doing something like modeling to another language or ...
- 314
-2
votes
1
answer
35
views
Some basic help about Artificial Neural Networks
Firstly, I'd like to state that I do not have any basis to the Neural Networks and I'd like you to recommend me very simple and understandable resources.
While I have a home assignment for a short ...
- 277k
3
votes
1
answer
649
views
Role of computational power in recent AI developments
Today Google's AI won its first game of Go against Lee Sedol, one of the best Go players on the planet. Image interpretation and self-driving cars are other recent success stories in machine learning. ...
D.W.♦
- 159k
0
votes
1
answer
516
views
How different is the working of SNN (Spiking Neural Network) as compared to a real Neuron System in biological systems?
Assuming its one step closer to realism as compared to ANN, DNNs and other Neural Network models, what are the primary differences between a real neuron system and SNN?
- 2,360
1
vote
0
answers
235
views
Hessian-Free instead of LSTM for Recurrent Net Machine Translation
Last year, Ilya Sutskever and collaborators came out with a paper about a recurrent LSTM net that learns sequence to sequence mappings for machine translation. It's somewhat surprising that the ...
CommunityBot
- 1
3
votes
1
answer
393
views
Convolutional Neural Network Feature Engineering?
I'm working through the tensorflow tutorial, and I see how you go from 28 x 28 to zero-padding and applying a 5x5x32 convolution to get 28x28x32 and max-pooling etc. What I'm confused about is the 32 ...
CommunityBot
- 1