Questions tagged [neural-networks]

Network structure inspired by simplified models of biological neurons (brain cells). Neural networks are trained to "learn" by supervised and unsupervised techniques, and can be used to solve optimization problems, approximation problems, classify patterns, and combinations thereof.

Filter by
Sorted by
Tagged with
0
votes
2answers
99 views

Training a neural net without labels in Reinforcement Learning

I am trying to dig further into machine learning and I am making a program to play a game as a start. I have created a game that is based on the mobile game Flappy Bird and can be generalized to the ...
0
votes
1answer
114 views

Dimension of 1x1 convolution output

I am having a hard time understanding maths behind a 1x1 convolution and how is it actually performed. Assuming that I have a 6x6x32 input to my 1x1xK convolution layer similar to the one presented ...
1
vote
1answer
2k views

ResNet output dimensions

So I am currently going over the ResNet paper and trying to understand the output dimensions of each of the layer, and it seems that I am already stuck on the first layer and its output dimension. If ...
4
votes
1answer
44 views

Do all the cells in a recurrent neural network share learned parameters?

Most descriptions of modern RNNs present a "folded" characterisation, that is to say, a single cell with a loop back to itself transmitting the hidden state from one step to the next. However, in ...
0
votes
0answers
32 views

Gradient Descent in MLPs using Computational Graphs

I'm working through Deep Learning by Goodfellow et al. The textbook introduces backpropagation for MLPs in page 203 (http://www.deeplearningbook.org/contents/mlp.html). However, it does not expand ...
0
votes
0answers
54 views

Machine learning for recommendation systems (feed forward and recurrent neural networks)

I recently started to learn about machine learning. I have created a feed forward neural network (ffnn) and a recurrent neural network (rnn) to predict user ratings of movies. I am using a subset of ...
2
votes
1answer
283 views

How is adversarial autoencoder better than an ordinary autoencoder?

An adversarial autoencoder helps us to impose a prior distribution $p(z)$ on the encoded values of the inputs, or $q(z)$. On the contrary, an ordinary autoencoder (which we train like an ordinary ...
1
vote
2answers
1k views

Neural Network Shape / Structure

I am attempting to learn neural networks using the Keras libary on the MNIST hand written digit dataset, using dense layers only. I am trying to figure out what the best shape for the network should ...
0
votes
1answer
1k views

How to feed videos to a neural network

I have been coding and testing Neural Networks for a while but as of now I have only used IMAGE datasets. (i.e. I have M training images and N testing images). Some datasets are video datasets. The ...
3
votes
1answer
49 views

Neural net that alters its own parameters

Is it possible for an AI neural net to be able to modify its own parameters/hyperparameters and/or add new parameters/hyperparameters? If so, how has it been implemented? For example, to simulate a ...
1
vote
0answers
125 views

Recurrent neural networks (Hopfield-like) with short limit cycles

Standard Hopfield networks exhibit stable patterns (states) which are attractors of a dynamic system. I wonder how to modify standard Hopfield networks such that they exhibit stable limit cycles as ...
0
votes
1answer
101 views

Convolutional Neural Network for Insects

I've recently started learning CNN's. I need a CNN that is specialized for insects detection. Dead insects will be put on a piece of paper / container, then images will be taken from a same distance, ...
0
votes
1answer
815 views

Amount of classes for a semantic segmentation neural network

I am currently working on implementation of semantic segmentation of images neural network, and try to implement one of the already existing solution such as Fully Convolutional Neural Network 1. ...
3
votes
0answers
82 views

Was there a phase in Machine Learning timeline when researchers thought some Neural Networks could not be trained?

I was talking to a professor who made a comment to my question. Me: So much of quality literature around this topic ( IP Protection for Neural Weights) emanated in 1990-1991, I'm truly at loss ...
0
votes
0answers
188 views

Output distribution of neural network with normally distributed weights

I stumbled upon a statement like the following, in the context of prediction (classification) with a neural network: "If we predict the class of a given data point $x$ with random weights, the output ...
1
vote
0answers
21 views

How do RNN's map variable-length sequences to variable-length sequences?

According to Karpathy's blog "The Unreasonable Effectiveness of Recurrent Neural Networks", recurrent neural networks can map variable-length sequences to variable-length sequences, as shown by the ...
6
votes
1answer
125 views

What is the role of Numerical Gradient Computation in Backpropagation algorithm?

I was listening CS231n (2017) lectures and noted that there is a lot of attention to Numerical Gradient Computation (NGC). It starts @5:53 in this video and appears a few times later. Also, looking ...
1
vote
0answers
200 views

Distributing Neural Network Training World-Wide Using Blockchain?

I just finished reading a white-paper from a recent AI startup. The company, Deep Brain Chain, wants to distribute neural network training over computers worldwide, using blockchain technology. Here ...
2
votes
2answers
101 views

How deep do neural networks need to be?

My question is a bit on the philosophical side, and there is probably not one single 'correct' answer on this. Nonetheless, I'm curious to hear your opinion... I'm currently designing a convolutional ...
3
votes
1answer
61 views

Clustered Regions by Each Neuron in Self Organizing Map (SOM)

I was given a question about SOM. There is a SOM which have 4x4 neurons and each neuron's x1 and x2 values (coordinates) given. Also neighborhood function and weight update rule given. How can i ...
3
votes
1answer
53 views

Applying a 1x1 convolution on an existing layer

I am looking at the following github. In this, layer3 has a shape of (1, 20, 72, 256) and I interpret this is as a single layer ...
6
votes
1answer
123 views

Neural Network: Why can't we calculate derivatives during forward prop itself?

I have been studying Andrew Ng's Coursera course about Deep Learning. In that he mentions that we calculate the activation functions during the forward pass, and the derivatives $\dfrac{dL}{dz} \ $, $\...
1
vote
0answers
100 views

adjusted fitness in NEAT algorithm

I'm learning about NEAT from the following paper: http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf I'm having trouble understanding how adjusted fitness penalizes large species and prevents ...
2
votes
0answers
25 views

How often should I read out information from an echo state recurrent neural network?

Recurrent neural networks makes it possible to implement some kind of memory, which can be very useful for a lot of tasks, incl. (but not limited to) robot control, which I am interested in. For ...
1
vote
1answer
43 views

References for the computational complexity of training neural networks

I'm looking for a good review paper or book chapter that offers an accessible introduction to the computational complexity of training neural networks for classification problems. In particular, I'm ...
1
vote
1answer
41 views

Are neural networks inherently imperfect because they mimic an imperfect machine (human)?

Neural networks are the base for all(most?) of the machine learning / deep learning algorithms/programs. Humans don't have fixed algorithms to decide or do something. Initially, we don't know how a ...
0
votes
0answers
71 views

Is there something as good as a GRU or LSTM but simpler?

I was just reading this paper: Gate-Variants of Gated Recurrent Unit (GRU) Neural Networks Rahul Dey and Fathi M. Salem It seems to me that perhaps the architecture of LSTMs and GRUs are overly ...
0
votes
0answers
123 views

RNN learning an iterative algorithm

How do I solve the following problem with a recurrent neural network (RNN)? What architecture should I use for the (conv)-RNN? Let $s \in \mathbb{R}^N$ be a musical signal. We corrupt it with ...
0
votes
1answer
56 views

Neural network to detect connected parameters

I have a list of configurations. Each configuration contains a number of parameters. The parameters have the same name but sometimes different values. I now want a neural network, that can detect, ...
1
vote
0answers
43 views

Detecting shapes using CNNs in respect to other shapes in the image

I'm solving a problem of detecting an optic disc in a retina image. As you can see from the image: the optic disc is the epicentrum of the blood vessels, has an irregular circular shape and has a ...
1
vote
0answers
136 views

Can CNN be used for “unknown correlated” data?

Now the title may be misleading, but I don't know how to give it a better name. My question is if I have data in a 2d matrix and under the assumption that: -I know for sure that there are ...
-1
votes
3answers
391 views

How might neural networks use a symbolic A.I approach?

I’m fairly new to this area so my understanding is pretty basic. Am I correct in saying that symbolic a.i would take data and use logic to search through and find the right answer. An example is a ...
1
vote
1answer
46 views

Reducibility and Artificial Neural Networks

I have read (here and here ) about the computational power of neural networks and a doubt came up. There is a way to reduce an ANN to another ANN (not taking into count the training algorithm) ? e.g. ...
1
vote
1answer
1k views

How one epoch completes in Perceptron?

I am confusing on completing one epoch, I am using Single Layer Feed Forward neural Network approach. Lets suppose i have a data of OR Gate: ...
5
votes
2answers
208 views

Why does the effectiveness of my reinforcement based neural network recede after a while?

I have a reinforcement based neural network training on the OpenAI gym CartPole-v1 environment. For the structure and training algorithm, assume it is the same as the one in this article. Typically, ...
1
vote
0answers
185 views

If we can train a ML algorithm to recognize letters with 95% accuracy, why does OCR software still suck? [closed]

So I've seen all these leaps in machine learning alorithms these past years and they've only gotten better and better at recognizing handwritten text. I remember reading once that some algorithms have ...
2
votes
1answer
36 views

Detecting face like patterns using CNN based face detector

I have a CNN based object detector trained on WIDER Face dataset. It can successfully detect human faces in a given image. Now, I am trying to detect abstract face and minimalistic face patterns in ...
1
vote
1answer
124 views

Viable use of genetic algorithms to train neural nets in a poker bot?

  I am designing a bot to play Texas Hold'Em Poker on tables of up to ten players, and the design includes a few feed forward neural networks (FFNN). These neural nets each have 8 to 12 ...
1
vote
1answer
1k views

Can CNN be used for numeric data

I have a numpy array of shape N_Samples x 360, as an example, consider the following: ...
1
vote
0answers
49 views

Finding a line shape objects (bars) in video images using neural networks (instead of Hough transform)

I'm currently trying to learn about computer vision stuff and I want to try it on a simple project of real time tracking of figures of players in a foosball game while the camera can be moving a ...
2
votes
1answer
32 views

Neural Network | What is the purpose of hidden layers and how many should I use?

I am pretty new to Neural Networks and I have two questions about hidden layers: 1. What is the purpose of hidden layers? I was wondering this because obviously you can get every result you want with ...
0
votes
0answers
167 views

Neural Network Inputs For Video Game use (DotA 2)

I am currently working on a project to build and train a neural network to play DotA 2 (at least be able to play like a really bad human). The thing is all the neural network I have built before were ...
2
votes
1answer
399 views

What can't be done with a neural network

This reference from the german wikipedia article on neural networks states: There are also many other important problems that are so difficult that a neural network will be unable to learn them ...
2
votes
1answer
52 views

Does the specific size of matrices affect the performance of matrix operations?

I was reading DeepMind's paper on I2A's and realized that the sizes of the hidden layers in their model were all like 32, 64, 256, and so on: all powers of 2. I have found the same thing in other ...
2
votes
1answer
336 views

How to calculate the total error from an arbitrary hidden layer in a neural network back propogation?

I'm following the tutorial over at https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/ and so far everything makes sense to me. I am now trying to reason about how these formulas ...
0
votes
0answers
38 views

neural net architecture fails to approximate hybrid functions

I adopted a feedforward net to approximate the following hybrid function: $$ output= \begin{cases} f(x,y) & \text{if}~~~ z=1\\ g(x,y) & \text{if}~~~ z=2 \end{cases} $$ where $x \in R,y \in R,z ...
0
votes
1answer
50 views

create new data from trained ANN

I use a very simple neural network to make classification between classes. Once my ANN is trained I'm able to present new and unknown data, and get a good classification. Is there a simple way to ...
4
votes
1answer
111 views

Are genetic algorithms an effective way to train neural networks?

It seems to me that genetic algorithms would be an ideal way to train neural networks so that they come to have the right weights, since they are especially good at escaping local minima, and ...
0
votes
1answer
176 views

max, min gradient and other terms in Neural Network

This link contains a demo that trains a Convolutional Neural Network on the MNIST digits dataset in browser. I am not getting below terms- 1. max, min gradient in each layer. 2.max, min activation ...
1
vote
1answer
255 views

What is branch factor for beam search in RNNs like TensorFlow's Magenta?

TensorFlow's Magenta melody generation module has a parameter for branch_factor1. What is it?