Podcast #128: We chat with Kent C Dodds about why he loves React and discuss what life was like in the dark days before Git. Listen now.

Questions tagged [neural-networks]

Network structure inspired by simplified models of biological neurons (brain cells). Neural networks are trained to "learn" by supervised and unsupervised techniques, and can be used to solve optimization problems, approximation problems, classify patterns, and combinations thereof.

125 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
5
votes
1answer
189 views

Why does the effectiveness of my reinforcement based neural network recede after a while?

I have a reinforcement based neural network training on the OpenAI gym CartPole-v1 environment. For the structure and training algorithm, assume it is the same as the one in this article. Typically, ...
5
votes
1answer
169 views

Predicting next action to take to reach a final state

Does anyone know of an algorithm that could be used to determine the next action to take to reach a desired state when trained on time-series data? For example, a robot starts at a certain state, ...
4
votes
0answers
20 views

How to represent sentences with their dependency parses as input to an RNN?

I am working on a task embedding sentences into a lower-dimensional space according to style, both grammatical and lexical. As such, I want to have as input the linear ordering of tokens in each ...
4
votes
0answers
88 views

Clarification in model precision term for Gaussian Processes

Following the blog post, where the dropout in the deep learning models has been approximated to a Gaussian process. Research paper and its appendix. Looking at the suggestion of the author from the ...
3
votes
0answers
35 views

When is currying more efficient in deep neural nets?

I'm reading a blog post on deep Q-learning, and it contrasts traditional lookup-table-based Q-learning with deep Q-learning: What I wonder about in this picture is: Why does the deep NN not also ...
3
votes
0answers
80 views

Was there a phase in Machine Learning timeline when researchers thought some Neural Networks could not be trained?

I was talking to a professor who made a comment to my question. Me: So much of quality literature around this topic ( IP Protection for Neural Weights) emanated in 1990-1991, I'm truly at loss ...
3
votes
0answers
655 views

Square or circle neural network detection

I am trying to use a simple perceptron to recognize if there is a square or a circle on an image. The images I generated are 300x300 px and I am having issues training the network since the images are ...
3
votes
0answers
79 views

What is the practical difference between an neuromorphic processors and GPU's?

IBM is working on a neuromorphic computer, Synapse, intended for computation based on artificial neural networks. My question is, what makes these types of computers more efficient at processing ...
3
votes
0answers
67 views

Number of parameters to be optimized in Artificial Bee Colony

I was reading this paper - Software defect prediction using cost-sensitive neural network by Ömer Faruk Arara and Kürsat Ayan It uses Artificial Bee Colony algorithm to train the neural network. In ...
3
votes
0answers
744 views

The runtime of a neural net with given numbers of observations, features, and neurons

If I have $n$ training observations, $m$ number of features per observation, and my neural network has $x$ neurons in the 1st layer, $y$ neurons in the 2nd layer, and 1 output neuron, what is the ...
3
votes
0answers
421 views

What is the activation function, label and loss function for Hierachical Softmax

Several papers(1 (originator), 2, 3) suggest the use of Hierachical Softmax instead of softmax for classification where the number of classes is large (eg many thousand). I haven't been able to get ...
3
votes
0answers
85 views

How to select topology for neural network?

I was given a target function to design a neural network and train: $y = (x_1 \wedge x_2) \vee (x_3 \wedge x_4)$ The number of inputs and outputs seems obvious (4 and 1). And the training data can ...
3
votes
0answers
118 views

practical use of a Boltzmann machine

I am reading "Neural Networks and Learning Machines" and in Chapter 11 the book covers Boltzman machines and it is stated "the network [Boltzmann machine] can perform pattern completion", but does not ...
2
votes
1answer
21 views

Neural network game players and incremental updates

Neural networks in recent years have been successfully used for gameplaying. A difference between games and e.g. image processing is that the game boards get updated incrementally. Do any neural ...
2
votes
0answers
30 views

How do convolution layers work?

So let's say I have a 4x4 image with 3 channels. So we have 3 4x4 matrices where each matrix represents a channel. Now let's say I also have a 3x3 kernel. I know that for convolution layers, I have to ...
2
votes
1answer
83 views

Learning the activation function in a neural network?

Neural networks use specific activation functions, commonly used ones are tanh, ReLu. I have seen that people have experimented with continuously parametrices activation functions, for instance here. ...
2
votes
0answers
28 views

Multilayer Perceptrons for solving variational problems

Can we use a multilayer perceptron to solve variational problems? By variational problem I mean something we might encounter in the calculus of variations, for example the geodesic problem: given two ...
2
votes
0answers
81 views

Approximate dot product between neural network output layer's parameter vector and input activations with winner-take-all hashing

In the paper Deep Networks with Large Output Spaces, Vijayanarasimhan et al. describe their approach to approximating the dot product between a neural network's output layer's parameter vector and ...
2
votes
0answers
25 views

How often should I read out information from an echo state recurrent neural network?

Recurrent neural networks makes it possible to implement some kind of memory, which can be very useful for a lot of tasks, incl. (but not limited to) robot control, which I am interested in. For ...
2
votes
0answers
174 views

Multilayer perceptron memory requirements

How much memory do we need to train a multilayer perceptron? I've started to figure this out myself, but I'm stuck. I have one-layer MLP. Each training example is a vector of 100 real numbers in ...
2
votes
0answers
114 views

Trying to understand Le's “cat paper”

QV Le et al. show in Building high-level features using large scale unsupervised learning (2012) how to use unsupervised learning of a deep neural net to recognize faces and cats. Working through this ...
2
votes
0answers
502 views

What are the inputs to an LSTM for Slot Filling Task

I am confused on the inputs of a Long-Short Term Memory (LSTM) for the slot filling task in Spoken Language Understanding. Before I worked on this, I implemented a language model with a Recurrent ...
2
votes
0answers
35 views

is mappings optimization problems onto neural networks still a thing?

Is mappings optimization problems onto neural networks still a thing? I only found papers are written prior to 1999. These old papers mostly deal with Hopfield network, which i read is obsolete and ...
2
votes
0answers
61 views

How to best model multidimensional, continuous, non-convex “shape” as neural network?

I have: set of n-dimensional points that I know are inside of the shape n >= 18, range on all dimensions has upper bound and lower bound (no axis goes to infinity). shape is pretty large in this n-...
2
votes
0answers
85 views

KDD Machine Learning using K-NN Algorithm Classification Problem

I'm trying to solve a classification problem from the KDD cup archive of 2004. Details can be found here: KDD 2004 Archive I'm only dong the particle physics part. The description of dataset is as ...
2
votes
0answers
12 views

What would happen if a multi-layer perceptron is insufficient to learn a function?

Suppose that something is wrong with its configuration (ie topology), for example there are too few neurons in a certain layer, or not enough layers. I have an intuition that some neurons' "delta" ($\...
2
votes
0answers
426 views

How to select the maximum weight value for a bias node in a neural network?

I'm programming a neural network. I know that I should initialize the network by picking random weights. How do I pick a random weight for the connections to bias nodes? What distribution should I ...
2
votes
0answers
99 views

How can you use HMMs and ANNs for on-line handwriting recognition?

On-line handwriting recognition is the task of converting a series of $(x(t),y(t))$ coordinates to symbols and words. In contrast to off-line handwriting recognition, where you only have a bitmap of ...
1
vote
0answers
7 views

Is there a way to connect a deep language model output to input?

In models like GPT-2, TXL and Grover, is there a good way to know which input weights (tokens) resulted in each token of the output?
1
vote
0answers
20 views

Value flow (and economics) in stacked reinforcement learning systems: agent as reinforcement environment for other agents?

There is evolving notion of stacked reinforcement learning systems, e.g. https://www.ijcai.org/proceedings/2018/0103.pdf - where one RL systems executes actions of the second RL system and it itself ...
1
vote
1answer
47 views

What does “Temporal extent” mean?

I am reading Long-term Temporal Convolutions for Action Recognition and under the Section 3.1, I read this: To investigate the impact of long-term temporal convolutions, we here study network ...
1
vote
0answers
80 views

Converting (reverse-engineering) Turing machine into program or most concise algorithm?

It is known that every program or every algorithm can be converted to Turing machine. But what about the reverse process? Is there algorithm (or research trend that considers such algorithm) to ...
1
vote
2answers
49 views

Learning a perceptron from stream data

I want to train a Perceptron using stochastic gradient rulefrom the stream data. I have very limited amount of memory and i can store only $N$ examples. Suppose my population consist of point as ...
1
vote
0answers
210 views

Why is the method of im2col with GEMM is more efficient than the method of direction implementation with SIMD in CNN

The convolutional layers are most computationally intense parts of Convolutional neural networks (CNNs).Currently the common approach to impement convolutional layers is to expand the image into a ...
1
vote
0answers
114 views

Proof of perceptron convergence theorem for ZERO threshold?

The generalized perceptron convergence theorem is for a defined threshold T. When you do the maths it all comes to an upper bound and a lower bound. The lower bound looks like this! Therefore ...
1
vote
0answers
222 views

How can node2vec help find similar “roles” within a graph (nodes whose connections have similar structure within the graph)?

I have a question on the node2vec algorithm described in this paper. Node2vec is a deep learning algorithm that word2vec to graphs to learn embeddings. The authors claim that it can help find nodes ...
1
vote
0answers
32 views

How would you go about creating a algorithm that should generate a shakespearean sonnet on any given theme

I need to create an algorithm that is going to create a shakespearean sonnet for a specific theme. This theme should be generated out of twitter tweets that have some hashtag. My current idea goes ...
1
vote
0answers
380 views

Backward mapping with bilinear sampler

I have some experiences with Convolutional Neural Networks before. I have a question regarding the Bilinear Sampler used in "Unsupervised Monocular Depth Estimation With Left-Right Consistency" (the ...
1
vote
0answers
31 views

Does a deep feedforward neural net approximate a (single or multivariate)polynomial?

Does composition of several linear transformation plus non-linear activation function in each layer and different layers (as they are in feedforward neural net) represent or approximate a ...
1
vote
0answers
118 views

Recurrent neural networks (Hopfield-like) with short limit cycles

Standard Hopfield networks exhibit stable patterns (states) which are attractors of a dynamic system. I wonder how to modify standard Hopfield networks such that they exhibit stable limit cycles as ...
1
vote
0answers
21 views

How do RNN's map variable-length sequences to variable-length sequences?

According to Karpathy's blog "The Unreasonable Effectiveness of Recurrent Neural Networks", recurrent neural networks can map variable-length sequences to variable-length sequences, as shown by the ...
1
vote
0answers
199 views

Distributing Neural Network Training World-Wide Using Blockchain?

I just finished reading a white-paper from a recent AI startup. The company, Deep Brain Chain, wants to distribute neural network training over computers worldwide, using blockchain technology. Here ...
1
vote
0answers
98 views

adjusted fitness in NEAT algorithm

I'm learning about NEAT from the following paper: http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf I'm having trouble understanding how adjusted fitness penalizes large species and prevents ...
1
vote
0answers
43 views

Detecting shapes using CNNs in respect to other shapes in the image

I'm solving a problem of detecting an optic disc in a retina image. As you can see from the image: the optic disc is the epicentrum of the blood vessels, has an irregular circular shape and has a ...
1
vote
0answers
131 views

Can CNN be used for “unknown correlated” data?

Now the title may be misleading, but I don't know how to give it a better name. My question is if I have data in a 2d matrix and under the assumption that: -I know for sure that there are ...
1
vote
0answers
47 views

Finding a line shape objects (bars) in video images using neural networks (instead of Hough transform)

I'm currently trying to learn about computer vision stuff and I want to try it on a simple project of real time tracking of figures of players in a foosball game while the camera can be moving a ...
1
vote
0answers
26 views

Hebbian rule doesn't get to a fixpoint

I'm trying to implement an Hopfield Network for pictures of 32x32 bits either 1 or -1; I have these 3 pictures and I transform each of them in a vector of 1024 elements. Then I take the 3 vectors and ...
1
vote
0answers
521 views

Using a neural network to identify patterns in strings

I am currently doing a research on data sensing and I came across the following concept. The requirement for me is to identify different data types using neural networks. Please note that I don't want ...
1
vote
0answers
48 views

Training algorithm for Manual error correction in text

I want to work on a model where I make some manual corrections in my clinical notes data and want the neural network to learn those corrections.The algorithm will be supplied with both corrected and ...
1
vote
0answers
32 views

Is parameter tying more common in RNNs than in regular NNs?

From what I can see it seems like RNNs favour using a backprop through time method which I haven't seen really applied to other neural networks. Can someone explain the significance of tying the ...