Questions tagged [neural-networks]

Network structure inspired by simplified models of biological neurons (brain cells). Neural networks are trained to "learn" by supervised and unsupervised techniques, and can be used to solve optimization problems, approximation problems, classify patterns, and combinations thereof.

Filter by
Sorted by
Tagged with
0
votes
1answer
45 views

Is it Possible to Get Maximum Weighted Input Value in a Neural Network?

Let's say that I have a standard feedforward neural network which has $M$ inputs, some number of hidden layers $N$, and a single neuron in the output. Is it possible to construct a network such that ...
2
votes
1answer
293 views

Non linear neural networks?

The activation of a perceptron style neuron is: $DotProduct(Inputs, Weights)+Bias > 0$ That is essentially classifying what side of a (hyper)plane a point is on (positive or negative side), like ...
2
votes
1answer
958 views

NeuroEvolution: NEAT algorithm innovation numbers

I have been reading up on the NeuronEvolution of Augmented Topologies and there's this little thing that's been bothering me. While reading Kenneth Stanley's Paper on NEAT I came on this figure here: ...
1
vote
0answers
511 views

Neural Networks: Style transfer for audio

I just found the paper A Neural Algorithm of Artistic Style which seems to be very cool. It describes a way to combine two images: Use the content of image A and the style of image B to create a new ...
5
votes
1answer
2k views

What's the input to the decoder in a sequence to sequence autoencoder?

What's the input to the decoder part of a sequence to sequence autoencoder? I've seen certain examples of such an autoencoder (using LSTM's more often than not) but am still unclear. For example, ...
0
votes
0answers
45 views

What is a Neural attention model and how can it be used for text summarization?

We are doing a research on multi-document (one document max 100 words) text summarization. We are looking into abstractive text summarization methods. I need following things to be clarified. Please ...
1
vote
1answer
67 views

Is learned RNN still universal?

I know it has been shown that RNNs are Turing complete. So for any Turing machine, there exists a configuration of a RNN that is equivalent to it. But I'm wondering, is all those configuration ...
2
votes
0answers
35 views

is mappings optimization problems onto neural networks still a thing?

Is mappings optimization problems onto neural networks still a thing? I only found papers are written prior to 1999. These old papers mostly deal with Hopfield network, which i read is obsolete and ...
3
votes
1answer
84 views

Are Perceptrons the neural network equivalence of Linear and Logistic Regression?

am I right in the assumption that both linear and logistic regression algorithms can be represented as the simplest form of neural networks,a perceptron, which consists of a two layers, an Input and ...
5
votes
1answer
507 views

Confused between turing-completeness and universal approximation - are they related?

I am trying to de-knot a point of confusion in my mind regarding "turing-completeness" and the "universal approximation theorem". The context here is deep neural nets: So, consider two types of ...
1
vote
0answers
206 views

About Barron's theorem in neural nets

I wanted to clarify my understanding that both the theorems of Barron about approximating functions by neural nets are about one hidden layer net. Is this right? Is there any Barron's theorem for ...
1
vote
0answers
454 views

Class activation mapping using inception-v3 network

I am using the technique of this paper: https://arxiv.org/pdf/1604.07953.pdf to localize food objects. The difference is that I am using the next generation of google-net, inception-v3 network. For ...
0
votes
0answers
300 views

RNN input shape for sequence generation on Tensorflow

I would like to train a RNN with LSTM cells in Tensorflow to predict the next word of a sequence. Words are N-length vectors of 0s and 1s. By looking at different tutorials, I saw that the input ...
1
vote
1answer
164 views

Basic queries for Object Detection using deep learning

I have recently started working on the Object detection using deep learning. I have understood the basic steps of object detection are: Input Image -> Object proposals -> Feature Extraction -> ...
0
votes
1answer
599 views

Big number of false positives in binary classification

I am fine-tuning the inception-v3 network(pretrained on ILSVRC) of tensorflow in a binary classification problem. I want to recognize if an image contains food or not. My dataset is imbalanced, the ...
1
vote
1answer
121 views

Why is low-dimensional discrete representation of input space useful?

Self-organizing maps undergo unsupervised training to produce a low-dimensional, discrete representation of the input space. Why is this useful? I think the results of the self-organizing map would ...
1
vote
1answer
96 views

Convolutional Neural Networks - Where should I start from?

I will soon be starting a part-time job within a start-up and I've been assigned to a project where I'll have to build a CNN that recognizes and counts people in various kinds of videos. My CEO knows ...
2
votes
1answer
94 views

Use of Activation Units in Facial Expression Understanding

The topic of extracting the Facial Action Coding System (FACS) Action Units (AUs) [1] from images and it's translation into emotion prediction [2] is pretty well studied, but I'm not clear on how it ...
2
votes
1answer
46 views

How to model a set of categorical values in the input of a neural network

One of the inputs to my neural network is a set. I have a set $S = \{s_0, s_1, ..., s_n\}$ in which all values $s_i$ are constant. An example of such a set could be the set of French wines (...
2
votes
1answer
725 views

Calculating gradient in a neural net using batches

I am a CS student learning about neural nets. Currently I am confused about how to train a neural net in batches. If I calculate error in a batch, I will get a vector of errors e.g. real1 - predicted1,...
3
votes
1answer
1k views

Multiple neural networks or multiple outputs?

Suppose you have data of the form input a matrix A, and output a matrix B, where each row of each is one datapoint. Should you create multiple neural networks, one for each column of B, or one NN with ...
3
votes
1answer
55 views

How can image processing neural networks be effectively trained?

I was just thinking about image processing neural networks and how to effectively train them in regard of the available dataset. Let's say you'd want to build a neural network which can distinguish ...
3
votes
2answers
253 views

Language Classification + AWS ML: what am I doing wrong?

I'm evaluating Amazon's machine learning platform, and thought that I would give it a "simple" classification problem first. As a disclaimer, I am quite new to machine learning (hence my interest in ...
1
vote
1answer
69 views

Multi modal learning from separate modal datasets

Let's say I'm trying to learn emotion given multi-modal sources. In this case, video and audio. However, I only have one dataset for video emotions and one dataset for audio. Is it possible to train a ...
3
votes
0answers
673 views

Square or circle neural network detection

I am trying to use a simple perceptron to recognize if there is a square or a circle on an image. The images I generated are 300x300 px and I am having issues training the network since the images are ...
1
vote
0answers
48 views

Relation between combining factors and disentangling for face recognition

I've read two interesting research papers on face recognition and I'd like to understand the relation between them. "Learning to Disentangle Factors of Variation with Manifold Interaction" by Reed et ...
0
votes
0answers
376 views

What is role of parameter learning rate, lr, and momentum constant, mc in Neural Networks?

can anyone describes the more simplified mathematical formulation of learning rate, lr, and momentum constant, mc in Neural Networks while training the data?
2
votes
1answer
122 views

After the training phase, is it better to run neural networks on a GPU or CPU? [closed]

Sorry if this is the wrong forum for this question. My understanding was GPUs were more efficient for running neural nets, but someone recently suggested GPUs are only needed for the training phase. ...
4
votes
0answers
80 views

What is the practical difference between an neuromorphic processors and GPU's?

IBM is working on a neuromorphic computer, Synapse, intended for computation based on artificial neural networks. My question is, what makes these types of computers more efficient at processing ...
0
votes
1answer
328 views

Architecture of a chatbot - how to organize and fetch possibilities? [closed]

I'm building a chatbot that would respond to text messages. Let's say that my chatbot works for customers of an internet provider and it can respond to the following things: Problems: About payment;...
0
votes
1answer
547 views

Methods to prevent overfitting

I am aware of three approaches to prevent over-fitting of data when trying to model it on a neural net. The first two approaches I know suggest to train on more data and employ bootstrap aggregating. ...
-2
votes
1answer
78 views

Is a Neural Network's weight selection dependent on its architecture? [closed]

Would an optimum combination of weights for a given topology necessarily be the the optimum for a different topology ?
3
votes
1answer
89 views

Confusion regarding terminology surrounding perceptron learning

I am having trouble understanding the terminology with perceptron learning. Is my current understanding correct? Let's say I have some data that classifies what type of flower a particular flower is. ...
2
votes
1answer
500 views

Where I can I find Pre Trained CNN Datasets for Facial Emotion Recognition?

I am working on a project that involves recognising emotion from images. There are two parts in the project: One where I will generate features from images using API's and then classify the emotion ...
4
votes
2answers
1k views

Z-score relations to perceptrons

I am learning about preceptrons and my professor noted that z-scores are a commmon pre-processing step to normalizing input variables. Following this, I am having trouble understanding why z-scores ...
2
votes
1answer
2k views

Difference Between Residual Neural Net and Recurrent Neural Net?

What is the difference between a Residual Neural Net and a Recurrent Neural Net? As I understand, Residual Neural Networks are very deep networks that implement 'shortcut' connections across ...
3
votes
0answers
69 views

Number of parameters to be optimized in Artificial Bee Colony

I was reading this paper - Software defect prediction using cost-sensitive neural network by Ömer Faruk Arara and Kürsat Ayan It uses Artificial Bee Colony algorithm to train the neural network. In ...
3
votes
2answers
104 views

Why neural nets are split into layers?

Why is every artificial neural network layered? Why isn't each node just a separate process?
3
votes
1answer
218 views

What method of collective recogintion to use for digits recognition?

The structure of the question is as follows: at first, I provide the concept of collective recognition, further I provide explanation of the various methods of group classification that I found, in ...
1
vote
1answer
181 views

Back propagation in neural networks

I just finished watching these 3 Coursera videos on back propagation in neural networks. I get the idea of what we're trying to do, but I don't get how we achieve that by calculating error in each ...
0
votes
0answers
32 views

Something wrong when recognizing data set using Binary Artificial Neural Network

I have learnt Neural Network for a couple weeks. I just met a problem when I tried to use ANN to recognize a dataset. So I'm gonna describe the problem: Set A has 15 members which are in set {1, 1.5, ...
0
votes
0answers
34 views

How would physical neural nets learn?

I understand how neural networks that are implemented in software learn. You simply change the synaptic weights in the program. But how would you get a hardware implementation of neural networks to ...
2
votes
1answer
35 views

Change in Training Accuracy

I am solving a problem in the Kaggle learn section : https://www.kaggle.com/c/facial-keypoints-detection The problem involves detecting facial keypoints in a 96x96 image, some 30 features are ...
3
votes
1answer
430 views

Why updating only a part of all neural network weights does not work?

I am having a problem with my program of deep neural network using Theano. In my deep neural network, I have several layers of neural network to predict an output given a certain input. Because of an ...
1
vote
3answers
353 views

Is it possible to add “memory” to a neural network?

Suppose I have a NN with one hidden layer, 10 inputs and 5 outputs, intended to be used as a (for example) game-bot AI. Would it make any sense to add, say, 5 (insert any number here) more inputs and ...
4
votes
1answer
1k views

Why update weights and biases after training a Neural Network on whole set of training samples

I am reading the book Neural Networks and Deep Learning by Micheal Nielsen. In the second chapter of his book, he describes the following algorithm for updating weights and biases for a neural ...
1
vote
1answer
198 views

When a trained RNN is tested, is the number of time-steps same for every input?

I am a beginner in deep learning so bear with me. If I want to unfold the RNN in order to represent the relation of output to the input as a non-recursive functions I would have to know the number of ...
0
votes
0answers
76 views

DFAs can be encoded as input/output for a neural network?

I would encode DFAs (Deterministic Finite State automata) as output (or input) of a neural network for a supervised learning; it is well-known [1] that efficacy of neural network training strongly ...
2
votes
1answer
1k views

Understanding the Broyden–Fletcher–Goldfarb–Shanno Algorithm to Select Weights for Neural Nets

I am trying to train and implement a Neural Network. I was reading a few articles, learning about their principles and the math that goes behind them. However, while I was trying to understand the ...
1
vote
0answers
365 views

Noise injection to fight overfitting

I've been researching the specifics of adding noise injection to my input layer to fight against overfitting. It seems as though most papers recommend a Gaussian noise vector with mean of 0 and a "...