Questions tagged [neural-networks]

Network structure inspired by simplified models of biological neurons (brain cells). Neural networks are trained to "learn" by supervised and unsupervised techniques, and can be used to solve optimization problems, approximation problems, classify patterns, and combinations thereof.

Filter by
Sorted by
Tagged with
2
votes
1answer
1k views

Beam size is a parameter in some RNNs like TensorFlow's Magenta. What is beam size?

Magenta's melody_rnn_generate method includes a parameter beam_size. What is it and how does affect the melody?
26
votes
1answer
14k views

What is Temperature in LSTM (and neural networks generally)?

One of the hyperparameters for LSTM networks is temperature. What is it?
0
votes
2answers
264 views

Trying to Classify MNIST where pixels are shuffled with a seed.Why is it not working?

I transformed the MNIST dataset as follows:(X (70000 x 784) is the training matrix) ...
1
vote
0answers
28 views

Hebbian rule doesn't get to a fixpoint

I'm trying to implement an Hopfield Network for pictures of 32x32 bits either 1 or -1; I have these 3 pictures and I transform each of them in a vector of 1024 elements. Then I take the 3 vectors and ...
0
votes
2answers
56 views

Convolutional Neural Network with constant kernels

I'm starting to learn about CNNs, and I have this question that I haven't been able to answer. Sorry if it is too basic. I know that in a CNN, the network learns to extract relevant features of ...
3
votes
1answer
223 views

Use machine learning algorithms data structure to store information (memory)

DISCLAIMER : I'm pretty new to machine learning field, so forgive me if my questions are somehow naive ... I already searched on Internet about this topic and find nothing interesting so I am ...
1
vote
1answer
429 views

Neural Networks: Simulate the working of Dynamic Fixed Point representation of the weights on hardware

I am looking to implement a neural network on hardware using Verilog. I have completed and tested with floating point representation and a 20 bit fixed point representation. I want to further reduce ...
2
votes
2answers
182 views

What algorithm to use for training combinations

I would be very glad if someone could help me with my machine learning task. I have palettes of 5 colors each (in RGB format), and would like to train the neural network so that I can input a color, ...
1
vote
0answers
535 views

Using a neural network to identify patterns in strings

I am currently doing a research on data sensing and I came across the following concept. The requirement for me is to identify different data types using neural networks. Please note that I don't want ...
4
votes
0answers
89 views

Clarification in model precision term for Gaussian Processes

Following the blog post, where the dropout in the deep learning models has been approximated to a Gaussian process. Research paper and its appendix. Looking at the suggestion of the author from the ...
1
vote
0answers
51 views

Training algorithm for Manual error correction in text

I want to work on a model where I make some manual corrections in my clinical notes data and want the neural network to learn those corrections.The algorithm will be supplied with both corrected and ...
0
votes
0answers
63 views

Neural networks fed with training data on demand?

What I want to achieve is to build a neural network with Keras which will have to stabilize a quadcopter I've built. The network would have three inputs: pitch, roll, yaw, acceleration x, accel y and ...
4
votes
1answer
44 views

Given a Noisy Curve, Write a Function to Output Likely Slopes

I am training a Variational Autoencoder (type of convolutional neural network), and have been plotting cost over time. The result is a noisy curve, shown here: I would like to write a function that ...
1
vote
1answer
173 views

What is the difference between distributed representations and factors of variation in deep learning

In the field of deep learning, people often talk about factors of variation which, in my understanding (in terms of dimensionality reduction), are the latent variable directions capturing variability ...
1
vote
1answer
51 views

4 Neurons to Decide 10 Digits

I am trying to solve this algorithm exercises (http://neuralnetworksanddeeplearning.com/chap1.html#exercise_513527) in Michael Nielson's online book: Neural Networks and Deep Learning (http://...
2
votes
1answer
81 views

What determines the number of inputs and outputs when initialising weights in a convolutional neural network?

Following Deep MNIST for Experts tutorial on Tensorflow, I realize I don't understand where the choice of numbers comes from when initializing weights. In the tutorial, they first show the below ...
1
vote
2answers
743 views

Scoring metric for machine learning method

For a machine learning method X (Deep Neural Nets variant), which performs classification tasks. In the output layer, for every label method, ...
0
votes
0answers
213 views

How are punishment and negative reinforcement handled in reinforcement learning using neural net function approximation?

Punishment is reducing a behaviour due to bad outcome e.g. A cow stops touching an electric fence because it gets a shock. Negative reinforcement is increasing a behaviour that reduces a bad outcome. ...
3
votes
1answer
152 views

Learning rule of multilayer neural networks

Suppose we have a 2-layer neural network completely connected with $d^{(0)}$ input units, $d^{(1)}$ hidden units and $d^{(2)}$ output units. We consider the error function given by $J(w) = \frac{1}{2}\...
0
votes
1answer
294 views

what does activation function become in spiking neural network?

In common ANN, nodes are based on a activation function, like a sigmoid or tanh or ReLU. Updates of synaptic weight are based on the derivative of this function. When node are represented with ...
0
votes
1answer
272 views

Genetic algorithm neural networks converges, but suddenly stops

I'm trying to create a genetic algorithm to train neural networks (because I'm to bad at back-propagation), and it works well until generation 18, where the loss stops to decrease and gets constant. ...
2
votes
0answers
190 views

Multilayer perceptron memory requirements

How much memory do we need to train a multilayer perceptron? I've started to figure this out myself, but I'm stuck. I have one-layer MLP. Each training example is a vector of 100 real numbers in ...
1
vote
2answers
190 views

How to use Neural Network classification if data not same size?

I have data like this. [0 1 0 1 0] [0 1 0 1 0 1 1] [0 1 0 1 ] [0 1 0 1 0 1 1 1 1 0] ... I want to classify with Neural Network but my data different size . I can ...
1
vote
1answer
40 views

Crafting a dataset to train a NN that recognizes image styles

In this paper the authors show how to use two NN that respectively recognize an art style and an image content to apply style filters to photographs. However, I am confused about how would one train ...
3
votes
1answer
3k views

What are the drawbacks of fully-convolutional neural networks?

It is possible to replace the fully-connected layers of a CNN with convolutional layers, making it fully convolutional. Fully-convolutional networks (FCNs) can be applied to inputs of various sizes, ...
1
vote
2answers
259 views

ANN with communication between nodes of same layer

I starting to learn about artificial neural network. I already did some simple things for classification, with different hidden layer. I would like to know : Is there a way to create "...
3
votes
3answers
48 views

Prove a network is a feedforward network if and only if the numbering of its cells satisfy these conditions

Undergraduate math student here attempting to understand neural networks. Picked up a text on sale, "Neural Network Learning and Expert Systems" (Gallant), and I'm just starting on the exercises for ...
-1
votes
1answer
694 views

create a ANN with more than one neuron output

I start to learn artificial neural network and all introduction that I have found on internet present architecture with a number of input variable, so the input layer has the same number of "neurons" ...
0
votes
1answer
70 views

Training a neural network by classifying its own output

The usual way one trains a neural network is to give it some input and provide the correct classification. But what about letting the neural network produce its own inputs, and then classifying those?...
0
votes
2answers
892 views

What are the challenges of using gradient descent on hyper parameters λ and η to find out their optimum values?

A question from chapter 3 of Michael Nielsen's [Neural Networks and Deep Learning]: It's tempting to use gradient descent to try to learn good values for hyper-parameters such as the regularization ...
2
votes
1answer
367 views

How to tackle different sample size in the training set in SVM

I have to train a SVM for a classification problem. I have some strings that are the paths in a deterministic finite automata (DFA). If the alphabet is -01- then possible strings are 011101110 or ...
1
vote
0answers
32 views

Is parameter tying more common in RNNs than in regular NNs?

From what I can see it seems like RNNs favour using a backprop through time method which I haven't seen really applied to other neural networks. Can someone explain the significance of tying the ...
2
votes
2answers
1k views

Measuring difference between two sets of neural network weights?

Suppose that we take a neural network of a given topology, and run it through two training processes, obtaining two different sets of converged weights at the end of the training. What is a good way ...
0
votes
0answers
137 views

Reversible 2D Shape Descriptor

This is a research problem and I am just wondering if there is any already existing answer in any computer vision related paper that may have skipped my notice since this is not my active area of ...
3
votes
1answer
2k views

How to show that cross entropy is minimized?

This Question is taken from the book Neural Networks and DeepLearning by Michael Nielsen The Question: In a single-neuron ,It is argued that the cross-entropy is small if σ(z)≈y for all training ...
0
votes
1answer
381 views

Why is an Artificial Neural Network with high accuracy score giving poor results?

I implemented my ANN using SKlearn module's class MLPClassifier. Fitting it on some data and testing it on a very specific subset of said training data, it gives a score of 1.0, but actually using the ...
2
votes
0answers
114 views

Trying to understand Le's “cat paper”

QV Le et al. show in Building high-level features using large scale unsupervised learning (2012) how to use unsupervised learning of a deep neural net to recognize faces and cats. Working through this ...
1
vote
1answer
170 views

Why is the inner product of -1,+1 binary variables at most $n-2$ and not at most $n-1$?

In short, if $x \neq u_i \in \{\pm1 \}^n$ then why is: $$ \langle x, u_i \rangle \leq n-2 $$ but not: $$ \langle x, u_i \rangle \leq n-1 $$ ? To add context: I was reading understanding machine ...
0
votes
1answer
64 views

Which class of neural network problem is this?

I have managed to describe a problem in quantum computing as the optimization of a function f(graph,vector), over graphs and real vectors. For a given graph, I can ...
1
vote
1answer
108 views

Algorithm Selection for Classification problem

I've been working on a developing a product selection network for my workplace. I work with lots of chemicals and the clients don't always know what they want/need so most of the time I have to ask a ...
2
votes
2answers
144 views

Overfitting in Machine Learning Algorithms

I am new in the ML. I know that overfitting is memorizing the data while training. Like in Neural Network, if we make lots of layers and lots of hidden nodes, we can memorize all the data, but it can ...
3
votes
1answer
657 views

Why does the neural network logistic regression cost function sum for all layers only for lambda?

I'm taking Andrew Ng's machine learning course and week 5 covers the training of neural networks. The modified cost function for neural network training is derived from the logistic regression cost ...
2
votes
0answers
529 views

What are the inputs to an LSTM for Slot Filling Task

I am confused on the inputs of a Long-Short Term Memory (LSTM) for the slot filling task in Spoken Language Understanding. Before I worked on this, I implemented a language model with a Recurrent ...
1
vote
1answer
704 views

Why is backpropagation called backwards propagation of error, when it back propagates error derivatives?

Wikipedia says: The backward propagation of errors or backpropagation, is a common method of training artificial neural networks and used in conjunction with an optimization method such as gradient ...
0
votes
1answer
872 views

Backpropagation in multiple output neural networks

I want to understand how the backpropagation algorithm would work on a neural network with multiple outputs. More specifically, I have a network with 21 binary (0/1) outputs and I want to minimize ...
3
votes
1answer
57 views

Why do people insist to use the term “multilayer perceptron” instead of “multilayer perceptron network”?

The perceptron model describes a linear classifier. Often people use the term "multilayer perceptron" to describe a feedforward neural network that uses perceptrons. This terminology simply sounds ...
2
votes
1answer
42 views

How do we adjust a model to fit our own dataset?

Often, I see people using networks like AlexNet, GoogLeNet, etc to train it on their dataset. These networks are trained or pretrained with a specific dataset, and people fine-tune it to their dataset....
0
votes
0answers
774 views

Can I use machine learning/ANNs to predict the outcome of a simulated horse-racing game based in probability?

I know ML is used in real horse-racing and other sports, where team/player history matters and can be used as a predictor for future games. What about for "simulated" games, where the outcome is ...
0
votes
0answers
124 views

Replace EKF by Neural Networks

I made a SLAM (Simultaneous Localization And Mapping) using Extended Kalman Filter (EKF) and it works really good, but I want to see if it works better using Neural Networks. For the EKF I used an ...
2
votes
1answer
173 views

NEAT algorithm and dynamic topology neural networks

I have recently been investigating dynamic topology neural networks and there is only one problem I have with understanding them. Because a neuron could be inserted at any point, the neurons are no ...