Questions tagged [neural-networks]
Network structure inspired by simplified models of biological neurons (brain cells). Neural networks are trained to "learn" by supervised and unsupervised techniques, and can be used to solve optimization problems, approximation problems, classify patterns, and combinations thereof.
125
questions with no upvoted or accepted answers
5
votes
1answer
189 views
Why does the effectiveness of my reinforcement based neural network recede after a while?
I have a reinforcement based neural network training on the OpenAI gym CartPole-v1 environment. For the structure and training algorithm, assume it is the same as the one in this article.
Typically, ...
5
votes
1answer
169 views
Predicting next action to take to reach a final state
Does anyone know of an algorithm that could be used to determine the next action to take to reach a desired state when trained on time-series data?
For example, a robot starts at a certain state, ...
4
votes
0answers
20 views
How to represent sentences with their dependency parses as input to an RNN?
I am working on a task embedding sentences into a lower-dimensional space according to style, both grammatical and lexical. As such, I want to have as input the linear ordering of tokens in each ...
4
votes
0answers
88 views
Clarification in model precision term for Gaussian Processes
Following the blog post, where the dropout in the deep learning models has been approximated to a Gaussian process. Research paper and its appendix.
Looking at the suggestion of the author from the ...
3
votes
0answers
35 views
When is currying more efficient in deep neural nets?
I'm reading a blog post on deep Q-learning, and it contrasts traditional lookup-table-based Q-learning with deep Q-learning:
What I wonder about in this picture is: Why does the deep NN not also ...
3
votes
0answers
80 views
Was there a phase in Machine Learning timeline when researchers thought some Neural Networks could not be trained?
I was talking to a professor who made a comment to my question.
Me: So much of quality literature around this topic ( IP Protection
for Neural Weights) emanated in 1990-1991, I'm truly at loss ...
3
votes
0answers
655 views
Square or circle neural network detection
I am trying to use a simple perceptron to recognize if there is a square or a circle on an image. The images I generated are 300x300 px and I am having issues training the network since the images are ...
3
votes
0answers
79 views
What is the practical difference between an neuromorphic processors and GPU's?
IBM is working on a neuromorphic computer, Synapse, intended for computation based on artificial neural networks.
My question is, what makes these types of computers more efficient at processing ...
3
votes
0answers
67 views
Number of parameters to be optimized in Artificial Bee Colony
I was reading this paper - Software defect prediction using cost-sensitive neural network by Ömer Faruk Arara and Kürsat Ayan
It uses Artificial Bee Colony algorithm to train the neural network. In ...
3
votes
0answers
744 views
The runtime of a neural net with given numbers of observations, features, and neurons
If I have $n$ training observations, $m$ number of features per observation, and my neural network has $x$ neurons in the 1st layer, $y$ neurons in the 2nd layer, and 1 output neuron, what is the ...
3
votes
0answers
421 views
What is the activation function, label and loss function for Hierachical Softmax
Several papers(1 (originator), 2, 3) suggest the use of Hierachical Softmax instead of softmax for classification where the number of classes is large (eg many thousand).
I haven't been able to get ...
3
votes
0answers
85 views
How to select topology for neural network?
I was given a target function to design a neural network and train: $y = (x_1 \wedge x_2) \vee (x_3 \wedge x_4)$
The number of inputs and outputs seems obvious (4 and 1). And the training data can ...
3
votes
0answers
118 views
practical use of a Boltzmann machine
I am reading "Neural Networks and Learning Machines" and in Chapter 11 the book covers Boltzman machines and it is stated "the network [Boltzmann machine] can perform pattern completion", but does not ...
2
votes
1answer
21 views
Neural network game players and incremental updates
Neural networks in recent years have been successfully used for gameplaying. A difference between games and e.g. image processing is that the game boards get updated incrementally. Do any neural ...
2
votes
0answers
30 views
How do convolution layers work?
So let's say I have a 4x4 image with 3 channels. So we have 3 4x4 matrices where each matrix represents a channel. Now let's say I also have a 3x3 kernel. I know that for convolution layers, I have to ...
2
votes
1answer
83 views
Learning the activation function in a neural network?
Neural networks use specific activation functions, commonly used ones are tanh, ReLu.
I have seen that people have experimented with continuously parametrices activation functions, for instance here. ...
2
votes
0answers
28 views
Multilayer Perceptrons for solving variational problems
Can we use a multilayer perceptron to solve variational problems?
By variational problem I mean something we might encounter in the calculus of variations, for example the geodesic problem: given two ...
2
votes
0answers
81 views
Approximate dot product between neural network output layer's parameter vector and input activations with winner-take-all hashing
In the paper Deep Networks with Large Output Spaces, Vijayanarasimhan et al. describe their approach to approximating the dot product between a neural network's output layer's parameter vector and ...
2
votes
0answers
25 views
How often should I read out information from an echo state recurrent neural network?
Recurrent neural networks makes it possible to implement some kind of memory, which can be very useful for a lot of tasks, incl. (but not limited to) robot control, which I am interested in. For ...
2
votes
0answers
174 views
Multilayer perceptron memory requirements
How much memory do we need to train a multilayer perceptron?
I've started to figure this out myself, but I'm stuck.
I have one-layer MLP. Each training example is a vector of 100 real numbers in ...
2
votes
0answers
114 views
Trying to understand Le's “cat paper”
QV Le et al. show in Building high-level features using large scale unsupervised learning (2012) how to use unsupervised learning of a deep neural net to recognize faces and cats. Working through this ...
2
votes
0answers
502 views
What are the inputs to an LSTM for Slot Filling Task
I am confused on the inputs of a Long-Short Term Memory (LSTM) for the slot filling task in Spoken Language Understanding.
Before I worked on this, I implemented a language model with a Recurrent ...
2
votes
0answers
35 views
is mappings optimization problems onto neural networks still a thing?
Is mappings optimization problems onto neural networks still a thing? I only found papers are written prior to 1999. These old papers mostly deal with Hopfield network, which i read is obsolete and ...
2
votes
0answers
61 views
How to best model multidimensional, continuous, non-convex “shape” as neural network?
I have:
set of n-dimensional points that I know are inside of the shape
n >= 18, range on all dimensions has upper bound and lower bound (no axis goes to infinity).
shape is pretty large in this n-...
2
votes
0answers
85 views
KDD Machine Learning using K-NN Algorithm Classification Problem
I'm trying to solve a classification problem from the KDD cup archive of 2004.
Details can be found here: KDD 2004 Archive
I'm only dong the particle physics part.
The description of dataset is as ...
2
votes
0answers
12 views
What would happen if a multi-layer perceptron is insufficient to learn a function?
Suppose that something is wrong with its configuration (ie topology), for example there are too few neurons in a certain layer, or not enough layers.
I have an intuition that some neurons' "delta" ($\...
2
votes
0answers
426 views
How to select the maximum weight value for a bias node in a neural network?
I'm programming a neural network. I know that I should initialize the network by picking random weights. How do I pick a random weight for the connections to bias nodes? What distribution should I ...
2
votes
0answers
99 views
How can you use HMMs and ANNs for on-line handwriting recognition?
On-line handwriting recognition is the task of converting a series of $(x(t),y(t))$ coordinates to symbols and words. In contrast to off-line handwriting recognition, where you only have a bitmap of ...
1
vote
0answers
7 views
Is there a way to connect a deep language model output to input?
In models like GPT-2, TXL and Grover, is there a good way to know which input weights (tokens) resulted in each token of the output?
1
vote
0answers
20 views
Value flow (and economics) in stacked reinforcement learning systems: agent as reinforcement environment for other agents?
There is evolving notion of stacked reinforcement learning systems, e.g. https://www.ijcai.org/proceedings/2018/0103.pdf - where one RL systems executes actions of the second RL system and it itself ...
1
vote
1answer
47 views
What does “Temporal extent” mean?
I am reading Long-term Temporal Convolutions for Action Recognition and under the Section 3.1, I read this:
To investigate the impact of long-term temporal convolutions, we here study network ...
1
vote
0answers
80 views
Converting (reverse-engineering) Turing machine into program or most concise algorithm?
It is known that every program or every algorithm can be converted to Turing machine. But what about the reverse process? Is there algorithm (or research trend that considers such algorithm) to ...
1
vote
2answers
49 views
Learning a perceptron from stream data
I want to train a Perceptron using stochastic gradient rulefrom the stream data. I have very limited amount of memory and i can store only $N$ examples.
Suppose my population consist of point as ...
1
vote
0answers
210 views
Why is the method of im2col with GEMM is more efficient than the method of direction implementation with SIMD in CNN
The convolutional layers are most computationally intense parts of Convolutional neural networks (CNNs).Currently the common approach to impement
convolutional layers is to expand the image into a ...
1
vote
0answers
114 views
Proof of perceptron convergence theorem for ZERO threshold?
The generalized perceptron convergence theorem is for a defined threshold T.
When you do the maths it all comes to an upper bound and a lower bound.
The lower bound looks like this!
Therefore
...
1
vote
0answers
222 views
How can node2vec help find similar “roles” within a graph (nodes whose connections have similar structure within the graph)?
I have a question on the node2vec algorithm described in this paper.
Node2vec is a deep learning algorithm that word2vec to graphs to learn embeddings. The authors claim that it can help find nodes ...
1
vote
0answers
32 views
How would you go about creating a algorithm that should generate a shakespearean sonnet on any given theme
I need to create an algorithm that is going to create a shakespearean sonnet for a specific theme. This theme should be generated out of twitter tweets that have some hashtag.
My current idea goes ...
1
vote
0answers
380 views
Backward mapping with bilinear sampler
I have some experiences with Convolutional Neural Networks before. I have a question regarding the Bilinear Sampler used in "Unsupervised Monocular Depth Estimation With Left-Right Consistency" (the ...
1
vote
0answers
31 views
Does a deep feedforward neural net approximate a (single or multivariate)polynomial?
Does composition of several linear transformation plus non-linear activation function in each layer and different layers (as they are in feedforward neural net) represent or approximate a ...
1
vote
0answers
118 views
Recurrent neural networks (Hopfield-like) with short limit cycles
Standard Hopfield networks exhibit stable patterns (states) which are attractors of a dynamic system. I wonder how to modify standard Hopfield networks such that they exhibit stable limit cycles as ...
1
vote
0answers
21 views
How do RNN's map variable-length sequences to variable-length sequences?
According to Karpathy's blog "The Unreasonable Effectiveness of Recurrent Neural Networks", recurrent neural networks can map variable-length sequences to variable-length sequences, as shown by the ...
1
vote
0answers
199 views
Distributing Neural Network Training World-Wide Using Blockchain?
I just finished reading a white-paper from a recent AI startup. The company, Deep Brain Chain, wants to distribute neural network training over computers worldwide, using blockchain technology. Here ...
1
vote
0answers
98 views
adjusted fitness in NEAT algorithm
I'm learning about NEAT from the following paper: http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf
I'm having trouble understanding how adjusted fitness penalizes large species and prevents ...
1
vote
0answers
43 views
Detecting shapes using CNNs in respect to other shapes in the image
I'm solving a problem of detecting an optic disc in a retina image. As you can see from the image:
the optic disc is the epicentrum of the blood vessels, has an irregular circular shape and has a ...
1
vote
0answers
131 views
Can CNN be used for “unknown correlated” data?
Now the title may be misleading, but I don't know how to give it a better name.
My question is if I have data in a 2d matrix and under the assumption that:
-I know for sure that there are ...
1
vote
0answers
47 views
Finding a line shape objects (bars) in video images using neural networks (instead of Hough transform)
I'm currently trying to learn about computer vision stuff and I want to try it on a simple project of real time tracking of figures of players in a foosball game while the camera can be moving a ...
1
vote
0answers
26 views
Hebbian rule doesn't get to a fixpoint
I'm trying to implement an Hopfield Network for pictures of 32x32 bits either 1 or -1;
I have these 3 pictures and I transform each of them in a vector of 1024 elements.
Then I take the 3 vectors and ...
1
vote
0answers
521 views
Using a neural network to identify patterns in strings
I am currently doing a research on data sensing and I came across the following concept. The requirement for me is to identify different data types using neural networks. Please note that I don't want ...
1
vote
0answers
48 views
Training algorithm for Manual error correction in text
I want to work on a model where I make some manual corrections in my clinical notes data and want the neural network to learn those corrections.The algorithm will be supplied with both corrected and ...
1
vote
0answers
32 views
Is parameter tying more common in RNNs than in regular NNs?
From what I can see it seems like RNNs favour using a backprop through time method which I haven't seen really applied to other neural networks.
Can someone explain the significance of tying the ...
