Questions tagged [neural-networks]

Network structure inspired by simplified models of biological neurons (brain cells). Neural networks are trained to "learn" by supervised and unsupervised techniques, and can be used to solve optimization problems, approximation problems, classify patterns, and combinations thereof.

Filter by
Sorted by
Tagged with
3
votes
1answer
85 views

Classification training data, but regression prediction

Suppose I'm performing machine learning on a simple dataset, and have a bunch of training data of the form: ...
3
votes
1answer
450 views

How does a recurrent connection in a neural network work?

I am reading a very interesting paper on genetic algorithms which define neural networks. I am familiar with how a feedforward neural network operates, but then I came across this: Where node #4 ...
3
votes
1answer
619 views

Role of computational power in recent AI developments

Today Google's AI won its first game of Go against Lee Sedol, one of the best Go players on the planet. Image interpretation and self-driving cars are other recent success stories in machine learning. ...
3
votes
1answer
1k views

Neural Network weight selection using Genetic Algorithm

Hi I want to ask about weight selection in neural network using genetic algorithm. Right now what I understand is Initialize population Encode the weight of the neural network to the chromosome ...
3
votes
1answer
570 views

How to compare the output of a neural network with his target?

I am coding a neural network implementation, but a I have problems in the design. I was wondering about how to compare the output with the target, my neural networks has three outputs ...
3
votes
1answer
168 views

Weird behaviour of softmax derivative?

I have been implementing some neural networks in MATLAB and recently I noticed a weird thing while implementing softmax derivative: Setting the derivative to one, rather than using the actual ...
3
votes
1answer
65 views

Clustered Regions by Each Neuron in Self Organizing Map (SOM)

I was given a question about SOM. There is a SOM which have 4x4 neurons and each neuron's x1 and x2 values (coordinates) given. Also neighborhood function and weight update rule given. How can i ...
3
votes
1answer
153 views

Learning rule of multilayer neural networks

Suppose we have a 2-layer neural network completely connected with $d^{(0)}$ input units, $d^{(1)}$ hidden units and $d^{(2)}$ output units. We consider the error function given by $J(w) = \frac{1}{2}\...
3
votes
1answer
2k views

HOG vs. neural networks for person detection

I am very new to computer vision, (a high school student) and I am working on a project to count the number of people present in a room. I have tried to use the HOGDecriptor for person detection ...
3
votes
1answer
73 views

Training given pairs of similar values, not labels

I have pairs of "similar" values $(x_i, y_i)$ drawn from a space $x_i, y_i \in S$, and want to train a neural network $N$ such that $N(x_i)$ would be "close" to $N(y_i)$ for all $i$, yet, to make it ...
3
votes
0answers
42 views

When is currying more efficient in deep neural nets?

I'm reading a blog post on deep Q-learning, and it contrasts traditional lookup-table-based Q-learning with deep Q-learning: What I wonder about in this picture is: Why does the deep NN not also ...
3
votes
0answers
83 views

Was there a phase in Machine Learning timeline when researchers thought some Neural Networks could not be trained?

I was talking to a professor who made a comment to my question. Me: So much of quality literature around this topic ( IP Protection for Neural Weights) emanated in 1990-1991, I'm truly at loss ...
3
votes
0answers
567 views

What are the inputs to an LSTM for Slot Filling Task

I am confused on the inputs of a Long-Short Term Memory (LSTM) for the slot filling task in Spoken Language Understanding. Before I worked on this, I implemented a language model with a Recurrent ...
3
votes
0answers
690 views

Square or circle neural network detection

I am trying to use a simple perceptron to recognize if there is a square or a circle on an image. The images I generated are 300x300 px and I am having issues training the network since the images are ...
3
votes
0answers
69 views

Number of parameters to be optimized in Artificial Bee Colony

I was reading this paper - Software defect prediction using cost-sensitive neural network by Ömer Faruk Arara and Kürsat Ayan It uses Artificial Bee Colony algorithm to train the neural network. In ...
3
votes
0answers
808 views

The runtime of a neural net with given numbers of observations, features, and neurons

If I have $n$ training observations, $m$ number of features per observation, and my neural network has $x$ neurons in the 1st layer, $y$ neurons in the 2nd layer, and 1 output neuron, what is the ...
3
votes
0answers
429 views

What is the activation function, label and loss function for Hierachical Softmax

Several papers(1 (originator), 2, 3) suggest the use of Hierachical Softmax instead of softmax for classification where the number of classes is large (eg many thousand). I haven't been able to get ...
3
votes
0answers
86 views

How to select topology for neural network?

I was given a target function to design a neural network and train: $y = (x_1 \wedge x_2) \vee (x_3 \wedge x_4)$ The number of inputs and outputs seems obvious (4 and 1). And the training data can ...
3
votes
0answers
119 views

practical use of a Boltzmann machine

I am reading "Neural Networks and Learning Machines" and in Chapter 11 the book covers Boltzman machines and it is stated "the network [Boltzmann machine] can perform pattern completion", but does not ...
3
votes
2answers
332 views

About the behaviour of multi-layer perceptrons

I have a multilayer perceptron. It has an input layer with two neurons, a hidden layer with an arbitrary number of neurons, and an output layer with two neurons. Given that ...
2
votes
2answers
96 views

Being stuck and frustrated with my masters project

I'm doing a masters in CS that requires me to implement from scratch most of the neural network models and because python libraries aren't applicable to what i want. The problem is that i don't feel ...
2
votes
1answer
52 views

Is Artificial General Intelligence possible with our current machine learning models? [closed]

In other words, is artificial human level intelligence not possible yet just because of limitations in processing power and amount of data required to train the models? Or we don't have the knowledge ...
2
votes
2answers
104 views

How deep do neural networks need to be?

My question is a bit on the philosophical side, and there is probably not one single 'correct' answer on this. Nonetheless, I'm curious to hear your opinion... I'm currently designing a convolutional ...
2
votes
1answer
902 views

What are the limitations of RNNs?

For a school project, I'm planning to compare Spiking Neural Networks (SNNs) and Deep Learning recurrent neural networks, such as Long Short Term Memory (LSTMs) networks in learning a time-series. I ...
2
votes
2answers
378 views

What kind of model is used by 20 Questions?

Which kind of machine learning concept / model is used in 20 Questions? Is this kind of thing best solved by a neural network? Where I can read something about it?
2
votes
2answers
112 views

How broad is the meaning of “algorithm”?

This is a purely terminological question. The word algorithm, as I have learnt it refers to something like an "effective method, a sequence of steps, for doing something". There are alternative ...
2
votes
1answer
83 views

Is there a universal learning rate for NeuralNetworks?

I'm currently creating a NeuralNetwork with backpropagation/gradient descent. There is this hyperparameter introduced called "learning rate" (η). Which has to be chosen to guarantee not overshooting ...
2
votes
3answers
672 views

Support Vector Machines as Neural Nets?

This is more of a conceptual question. I have learned about Neural Nets, and I have some clue as to how Support Vector Machines work. I read somewhere however that given the appropriate kernel (is ...
2
votes
1answer
332 views

How can I optimize 3 variables in order to maximize the end product?

I am in the process of making a cryptocurrency trading bot. Currently, I am doing backtesting over a period of 7 months in which I provide a portion of historical data as if it were in real-life. By ...
2
votes
1answer
408 views

How to tackle different sample size in the training set in SVM

I have to train a SVM for a classification problem. I have some strings that are the paths in a deterministic finite automata (DFA). If the alphabet is -01- then possible strings are 011101110 or ...
2
votes
2answers
1k views

Measuring difference between two sets of neural network weights?

Suppose that we take a neural network of a given topology, and run it through two training processes, obtaining two different sets of converged weights at the end of the training. What is a good way ...
2
votes
2answers
147 views

Overfitting in Machine Learning Algorithms

I am new in the ML. I know that overfitting is memorizing the data while training. Like in Neural Network, if we make lots of layers and lots of hidden nodes, we can memorize all the data, but it can ...
2
votes
1answer
171 views

Time Series Prediction with an LSTM

I have a time series that I want to predict with an LSTM. I am able to get very good results using 50 datapoints predicting 51, but I struggle to get any accuracy using something like 200 datapoints ...
2
votes
1answer
113 views

Approximating sign function by tanh in Multi Layer perceptrons

This is an exercise problem from the book "Learning from Data". Given $w_1$ and $\epsilon \gt 0$, find $w_2$ such that, $$\lvert \mbox{sign}(w_1^Tx_n)-\tanh(w_2^Tx_n)\rvert \le \epsilon$$ Where $...
2
votes
1answer
419 views

What kind of Neural Network (if any) could fit two sets of data points?

I have two datasets, one of animal migration patterns (collected over the course of a couple years) that consists of many points on an x, y plane (latitude, longitude), and the other of ocean surface ...
2
votes
1answer
3k views

How to reconstruct the image from a neural network output?

I am trying to use the genetic algorithm to optimise a multi-layered neural network for image classification (i am using a subset of the MNIST handwritten digit data set as my initial dataset, but ...
2
votes
1answer
64 views

Progressive discrete multifunction optimization

I have a set of functions $F$. The functions effectively take a set $S$ that is always a subset of a global set of all possible values $G$, where $|G|>1000$. (alternatively, they could take a $|G|$...
2
votes
2answers
33 views

Zero-Sum Games and Halting Problem

Wikipedia states on the page of the halting problem, "For any program f that might determine if programs halt, a "pathological" program g called with an input can pass its own source and its input to ...
2
votes
1answer
48 views

What are some complexity classes for neural networks?

Turing machines and neural networks are equivalent in their expressive powers, but as models of computation they are different. Turing machines come pre-configured with their transition functions ...
2
votes
1answer
44 views

Energy consumption off CNN models

I want to calculate/estimate the energy consumption for the different convolutional neural networks. Is there any possibility to measure the energy consumed by AlexNet for example with a tool or with ...
2
votes
1answer
31 views

Why do we need to take the derivative of the activation function in backwards propagation?

I was reading this article here: https://towardsdatascience.com/how-does-back-propagation-in-artificial-neural-networks-work-c7cad873ea7. When he gets to the part where he calculates the loss at ...
2
votes
3answers
221 views

Could neural networks be considered metaheuristics?

A metaheuristic is defined as a metaheuristic is a higher-level procedure or heuristic designed to find, generate, or select a heuristic (partial search algorithm) that may provide a sufficiently ...
2
votes
1answer
113 views

How to represent symbolic knowledge using real numbers - theory about neural networks and natural/analog computing?

One can define the semantics of one definite word using the references to real world entities, relationships with the other words and other concepts and represent all this knowledge about this one ...
2
votes
1answer
425 views

Parameter sharing / weight constraints in Neural Networks

I would like to train a neural network whose parameters (alternatively, weights) are subject to linear constraints such as $w_{i,j} = w_{i',j'}$, where $w_{i,j}$ denotes the weight from input node $...
2
votes
1answer
33 views

Neural Network | What is the purpose of hidden layers and how many should I use?

I am pretty new to Neural Networks and I have two questions about hidden layers: 1. What is the purpose of hidden layers? I was wondering this because obviously you can get every result you want with ...
2
votes
1answer
451 views

What can't be done with a neural network

This reference from the german wikipedia article on neural networks states: There are also many other important problems that are so difficult that a neural network will be unable to learn them ...
2
votes
1answer
2k views

Beam size is a parameter in some RNNs like TensorFlow's Magenta. What is beam size?

Magenta's melody_rnn_generate method includes a parameter beam_size. What is it and how does affect the melody?
2
votes
2answers
223 views

What algorithm to use for training combinations

I would be very glad if someone could help me with my machine learning task. I have palettes of 5 colors each (in RGB format), and would like to train the neural network so that I can input a color, ...
2
votes
1answer
320 views

Non linear neural networks?

The activation of a perceptron style neuron is: $DotProduct(Inputs, Weights)+Bias > 0$ That is essentially classifying what side of a (hyper)plane a point is on (positive or negative side), like ...
2
votes
1answer
1k views

NeuroEvolution: NEAT algorithm innovation numbers

I have been reading up on the NeuronEvolution of Augmented Topologies and there's this little thing that's been bothering me. While reading Kenneth Stanley's Paper on NEAT I came on this figure here: ...

1 2
3
4 5
9