Questions tagged [neural-networks]

Network structure inspired by simplified models of biological neurons (brain cells). Neural networks are trained to "learn" by supervised and unsupervised techniques, and can be used to solve optimization problems, approximation problems, classify patterns, and combinations thereof.

Filter by
Sorted by
Tagged with
2
votes
1answer
22 views

Is it still transfer learning if you consider input as well as output? (neural networks)

I'm new to the CS stack exchange, so a fond hello to you all! I joined since I have a question I've been curious about. I have recently been running some experiments in transfer learning - ...
2
votes
1answer
119 views

Proving Monotonicity of Softmax Layer

In the book here: http://neuralnetworksanddeeplearning.com/chap3.html If you scroll down to Exercise 2 in the Softmax Section, it says Show that $\partial a^L_{j}/\partial z^L_{k}$ is positive if $...
2
votes
1answer
137 views

Capsule networks for classification with limited data

Capsule networks seem to match performance of convolutional neural networks on image classification tasks (more specifically on classification of handwritten digits in the MNIST dataset) 1. I have ...
2
votes
1answer
45 views

Computer vision methods without “pre-training”

I'm new to computer vision and I have a common question that I couldn't figure out with Internet or books. As I understood, in general, there are two main approaches in modern computer vision: neural ...
2
votes
1answer
257 views

How is adversarial autoencoder better than an ordinary autoencoder?

An adversarial autoencoder helps us to impose a prior distribution $p(z)$ on the encoded values of the inputs, or $q(z)$. On the contrary, an ordinary autoencoder (which we train like an ordinary ...
2
votes
1answer
35 views

Detecting face like patterns using CNN based face detector

I have a CNN based object detector trained on WIDER Face dataset. It can successfully detect human faces in a given image. Now, I am trying to detect abstract face and minimalistic face patterns in ...
2
votes
1answer
48 views

Does the specific size of matrices affect the performance of matrix operations?

I was reading DeepMind's paper on I2A's and realized that the sizes of the hidden layers in their model were all like 32, 64, 256, and so on: all powers of 2. I have found the same thing in other ...
2
votes
1answer
273 views

How to calculate the total error from an arbitrary hidden layer in a neural network back propogation?

I'm following the tutorial over at https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/ and so far everything makes sense to me. I am now trying to reason about how these formulas ...
2
votes
1answer
73 views

What determines the number of inputs and outputs when initialising weights in a convolutional neural network?

Following Deep MNIST for Experts tutorial on Tensorflow, I realize I don't understand where the choice of numbers comes from when initializing weights. In the tutorial, they first show the below ...
2
votes
1answer
42 views

How do we adjust a model to fit our own dataset?

Often, I see people using networks like AlexNet, GoogLeNet, etc to train it on their dataset. These networks are trained or pretrained with a specific dataset, and people fine-tune it to their dataset....
2
votes
1answer
168 views

NEAT algorithm and dynamic topology neural networks

I have recently been investigating dynamic topology neural networks and there is only one problem I have with understanding them. Because a neuron could be inserted at any point, the neurons are no ...
2
votes
1answer
122 views

After the training phase, is it better to run neural networks on a GPU or CPU? [closed]

Sorry if this is the wrong forum for this question. My understanding was GPUs were more efficient for running neural nets, but someone recently suggested GPUs are only needed for the training phase. ...
2
votes
1answer
1k views

Difference Between Residual Neural Net and Recurrent Neural Net?

What is the difference between a Residual Neural Net and a Recurrent Neural Net? As I understand, Residual Neural Networks are very deep networks that implement 'shortcut' connections across ...
2
votes
1answer
909 views

Understanding the Broyden–Fletcher–Goldfarb–Shanno Algorithm to Select Weights for Neural Nets

I am trying to train and implement a Neural Network. I was reading a few articles, learning about their principles and the math that goes behind them. However, while I was trying to understand the ...
2
votes
1answer
3k views

How to determine the dimensions of the bias vector in a Convolutional layer (tensor flow)?

In the Deep MNIST Tutorial for Tensor Flow, (and in general), we create a convolutional layer with a weight tensor, say W, of shape (patch dims, #input channels, # output channels). However, when we ...
2
votes
1answer
134 views

Why Isn't This Outlier Score/Reconstruction Error Not Squared?

I was looking through a paper called "AI2 : Training a big data machine to defend", and saw this (http://people.csail.mit.edu/kalyan/AI2_Paper.pdf) $score(X_{i}) = \sum_{j=1}^{p} (|X_{i} − R^{j}_{i}|)...
2
votes
1answer
419 views

Language grammar correction with supervised learning

I want to work on automatic grammar correction using machine learning (possibly using recurrent or deep neural networks). The algorithm will be supplied with both corrected and initial documents for ...
2
votes
1answer
81 views

Updating connections weights in neural networks

I am learning about neural networks and have a couple of things I don't understand. Firstly, in competitive learning I understand that only the neuron with the strongest output is reinforced. That is ...
2
votes
1answer
527 views

weights in a simple neural network

I have seen that in the material made by Andrew Ng about neural networks, he uses the following weights: so when I replace the final values of h_theta(x) in the formula: I got values near 0 and 1. ...
2
votes
1answer
90 views

Neural network: noisy temporal sequence converter (transducer?producer?) on demand?

I start to suspect this problem is very hard now that I cannot find a single relevant literature on the subject, but it's too late to change the class project topics now, so I hope any pointers to a ...
2
votes
1answer
20 views

Neural network game players and incremental updates

Neural networks in recent years have been successfully used for gameplaying. A difference between games and e.g. image processing is that the game boards get updated incrementally. Do any neural ...
2
votes
0answers
28 views

How do convolution layers work?

So let's say I have a 4x4 image with 3 channels. So we have 3 4x4 matrices where each matrix represents a channel. Now let's say I also have a 3x3 kernel. I know that for convolution layers, I have to ...
2
votes
1answer
76 views

Learning the activation function in a neural network?

Neural networks use specific activation functions, commonly used ones are tanh, ReLu. I have seen that people have experimented with continuously parametrices activation functions, for instance here. ...
2
votes
0answers
28 views

Multilayer Perceptrons for solving variational problems

Can we use a multilayer perceptron to solve variational problems? By variational problem I mean something we might encounter in the calculus of variations, for example the geodesic problem: given two ...
2
votes
0answers
79 views

Approximate dot product between neural network output layer's parameter vector and input activations with winner-take-all hashing

In the paper Deep Networks with Large Output Spaces, Vijayanarasimhan et al. describe their approach to approximating the dot product between a neural network's output layer's parameter vector and ...
2
votes
0answers
24 views

How often should I read out information from an echo state recurrent neural network?

Recurrent neural networks makes it possible to implement some kind of memory, which can be very useful for a lot of tasks, incl. (but not limited to) robot control, which I am interested in. For ...
2
votes
0answers
168 views

Multilayer perceptron memory requirements

How much memory do we need to train a multilayer perceptron? I've started to figure this out myself, but I'm stuck. I have one-layer MLP. Each training example is a vector of 100 real numbers in ...
2
votes
0answers
112 views

Trying to understand Le's “cat paper”

QV Le et al. show in Building high-level features using large scale unsupervised learning (2012) how to use unsupervised learning of a deep neural net to recognize faces and cats. Working through this ...
2
votes
0answers
491 views

What are the inputs to an LSTM for Slot Filling Task

I am confused on the inputs of a Long-Short Term Memory (LSTM) for the slot filling task in Spoken Language Understanding. Before I worked on this, I implemented a language model with a Recurrent ...
2
votes
0answers
35 views

is mappings optimization problems onto neural networks still a thing?

Is mappings optimization problems onto neural networks still a thing? I only found papers are written prior to 1999. These old papers mostly deal with Hopfield network, which i read is obsolete and ...
2
votes
0answers
60 views

How to best model multidimensional, continuous, non-convex “shape” as neural network?

I have: set of n-dimensional points that I know are inside of the shape n >= 18, range on all dimensions has upper bound and lower bound (no axis goes to infinity). shape is pretty large in this n-...
2
votes
1answer
43 views

Determining if my artificial neural network needs additional layers

I have implemented a neural network for load forecasting in Microsoft Excel. My structure is very simplistic and involves only 1 hidden layer and 4 neurons. (See picture) I trained my network with a ...
2
votes
0answers
83 views

KDD Machine Learning using K-NN Algorithm Classification Problem

I'm trying to solve a classification problem from the KDD cup archive of 2004. Details can be found here: KDD 2004 Archive I'm only dong the particle physics part. The description of dataset is as ...
2
votes
0answers
12 views

What would happen if a multi-layer perceptron is insufficient to learn a function?

Suppose that something is wrong with its configuration (ie topology), for example there are too few neurons in a certain layer, or not enough layers. I have an intuition that some neurons' "delta" ($\...
2
votes
0answers
66 views

Using the random forest algorithm to predict vectors [duplicate]

I know this might sound like a newbie question, but bear with me. I have read a paper where researchers use a random forest to predict species distribution, but in their study, they only predict a ...
2
votes
0answers
422 views

How to select the maximum weight value for a bias node in a neural network?

I'm programming a neural network. I know that I should initialize the network by picking random weights. How do I pick a random weight for the connections to bias nodes? What distribution should I ...
2
votes
0answers
99 views

How can you use HMMs and ANNs for on-line handwriting recognition?

On-line handwriting recognition is the task of converting a series of $(x(t),y(t))$ coordinates to symbols and words. In contrast to off-line handwriting recognition, where you only have a bitmap of ...
2
votes
1answer
589 views

Making feature vector from Gabor filters for classification

My aim is to classify types of cars (Sedans,SUV,Hatchbacks) and earlier I was using corner features for classification but it didn't work out very well so now I am trying Gabor features. code from ...
1
vote
3answers
319 views

Is it possible to add “memory” to a neural network?

Suppose I have a NN with one hidden layer, 10 inputs and 5 outputs, intended to be used as a (for example) game-bot AI. Would it make any sense to add, say, 5 (insert any number here) more inputs and ...
1
vote
2answers
1k views

Train Neural network with infinite amount of data [closed]

Does a sufficiently complex neural network guarantee to find the optimal solution, given an infinite amount of data and the back propagation technique for training? In other words, given an infinite ...
1
vote
2answers
173 views

Perceptron learning rule doesn't work [closed]

i'm a little bit lost ... can you help me ? So I have this table of date (each row give a point with its group) So i took a random weight let's say : [1, -2] H = 1 if n =< 0 0 otherwise a= H(...
1
vote
1answer
434 views

How are Bayesian Nets, Hidden Markov Chains, Conditional Random Fields and Neural Nets related?

I am having an AI exam in two weeks, and I am still figuring out certain concepts and ideas, related to Bayesian Nets, Hidden Markov Chains, Conditional Random Fields and Neural Nets (yes it is all ...
1
vote
2answers
5k views

What's the difference between a rule based system and an artificial neural network?

I'm currently doing some reading into AI and up to this point couldn't find a satisfying answer to this question: what's the difference between a rule based system and an artificial neural network? ...
1
vote
2answers
244 views

ANN with communication between nodes of same layer

I starting to learn about artificial neural network. I already did some simple things for classification, with different hidden layer. I would like to know : Is there a way to create "...
1
vote
1answer
289 views

error computation in multi layered perceptron

I was reading about Multi Layered Perceptron(MLP) and how can we learn pattern using it. Algorithm was stated as Initiate all weight to small values. Compute activation of each neuron using sigmoid ...
1
vote
3answers
1k views

Are neural networks dynamical systems?

Dynamical systems are those whose evolution can be described by a rule, evolves with time and is deterministic. In this context can I say that Neural networks have a rule of evolution which is the ...
1
vote
1answer
410 views

What factors must one consider choosing an NN structure?

Suppose we have a classification problem and we wish to solve the problem by Neural Network. What factors must one consider choosing an NN structure? e.g Feed Forward, Recurrent and other available ...
1
vote
1answer
66 views

How to train this neural network?

I seek the neural network (NN) which satisfies the 100 equations (i=1,2...100) $\sum_{j=1}^{2000} NN(A_{ij},B_{ij},C_{ij})=Q_i$. Where A,B,C are 100x2000 matrices So I know Q, A, B and C How can ...
1
vote
1answer
1k views

How one epoch completes in Perceptron?

I am confusing on completing one epoch, I am using Single Layer Feed Forward neural Network approach. Lets suppose i have a data of OR Gate: ...
1
vote
1answer
39 views

Crafting a dataset to train a NN that recognizes image styles

In this paper the authors show how to use two NN that respectively recognize an art style and an image content to apply style filters to photographs. However, I am confused about how would one train ...