We’re rewarding the question askers & reputations are being recalculated! Read more.

Questions tagged [neural-networks]

Network structure inspired by simplified models of biological neurons (brain cells). Neural networks are trained to "learn" by supervised and unsupervised techniques, and can be used to solve optimization problems, approximation problems, classify patterns, and combinations thereof.

Filter by
Sorted by
Tagged with
1
vote
0answers
20 views

Value flow (and economics) in stacked reinforcement learning systems: agent as reinforcement environment for other agents?

There is evolving notion of stacked reinforcement learning systems, e.g. https://www.ijcai.org/proceedings/2018/0103.pdf - where one RL systems executes actions of the second RL system and it itself ...
1
vote
0answers
28 views

Solving analytic gradient of loss function for neural networks [closed]

Please note that I am talking in about theory rather than ''what someone would do in a real, practical situation''. Given a multi-layer Perceptron with at least 1 hidden layer, and sigmoid (or other ...
1
vote
1answer
47 views

What does “Temporal extent” mean?

I am reading Long-term Temporal Convolutions for Action Recognition and under the Section 3.1, I read this: To investigate the impact of long-term temporal convolutions, we here study network ...
1
vote
0answers
80 views

Converting (reverse-engineering) Turing machine into program or most concise algorithm?

It is known that every program or every algorithm can be converted to Turing machine. But what about the reverse process? Is there algorithm (or research trend that considers such algorithm) to ...
1
vote
2answers
49 views

Learning a perceptron from stream data

I want to train a Perceptron using stochastic gradient rulefrom the stream data. I have very limited amount of memory and i can store only $N$ examples. Suppose my population consist of point as ...
1
vote
0answers
199 views

Why is the method of im2col with GEMM is more efficient than the method of direction implementation with SIMD in CNN

The convolutional layers are most computationally intense parts of Convolutional neural networks (CNNs).Currently the common approach to impement convolutional layers is to expand the image into a ...
1
vote
0answers
114 views

Proof of perceptron convergence theorem for ZERO threshold?

The generalized perceptron convergence theorem is for a defined threshold T. When you do the maths it all comes to an upper bound and a lower bound. The lower bound looks like this! Therefore ...
1
vote
0answers
219 views

How can node2vec help find similar “roles” within a graph (nodes whose connections have similar structure within the graph)?

I have a question on the node2vec algorithm described in this paper. Node2vec is a deep learning algorithm that word2vec to graphs to learn embeddings. The authors claim that it can help find nodes ...
1
vote
0answers
32 views

How would you go about creating a algorithm that should generate a shakespearean sonnet on any given theme

I need to create an algorithm that is going to create a shakespearean sonnet for a specific theme. This theme should be generated out of twitter tweets that have some hashtag. My current idea goes ...
1
vote
0answers
375 views

Backward mapping with bilinear sampler

I have some experiences with Convolutional Neural Networks before. I have a question regarding the Bilinear Sampler used in "Unsupervised Monocular Depth Estimation With Left-Right Consistency" (the ...
1
vote
0answers
31 views

Does a deep feedforward neural net approximate a (single or multivariate)polynomial?

Does composition of several linear transformation plus non-linear activation function in each layer and different layers (as they are in feedforward neural net) represent or approximate a ...
1
vote
0answers
118 views

Recurrent neural networks (Hopfield-like) with short limit cycles

Standard Hopfield networks exhibit stable patterns (states) which are attractors of a dynamic system. I wonder how to modify standard Hopfield networks such that they exhibit stable limit cycles as ...
1
vote
0answers
21 views

How do RNN's map variable-length sequences to variable-length sequences?

According to Karpathy's blog "The Unreasonable Effectiveness of Recurrent Neural Networks", recurrent neural networks can map variable-length sequences to variable-length sequences, as shown by the ...
1
vote
0answers
199 views

Distributing Neural Network Training World-Wide Using Blockchain?

I just finished reading a white-paper from a recent AI startup. The company, Deep Brain Chain, wants to distribute neural network training over computers worldwide, using blockchain technology. Here ...
1
vote
0answers
98 views

adjusted fitness in NEAT algorithm

I'm learning about NEAT from the following paper: http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf I'm having trouble understanding how adjusted fitness penalizes large species and prevents ...
1
vote
0answers
43 views

Detecting shapes using CNNs in respect to other shapes in the image

I'm solving a problem of detecting an optic disc in a retina image. As you can see from the image: the optic disc is the epicentrum of the blood vessels, has an irregular circular shape and has a ...
1
vote
0answers
130 views

Can CNN be used for “unknown correlated” data?

Now the title may be misleading, but I don't know how to give it a better name. My question is if I have data in a 2d matrix and under the assumption that: -I know for sure that there are ...
1
vote
0answers
182 views

If we can train a ML algorithm to recognize letters with 95% accuracy, why does OCR software still suck? [closed]

So I've seen all these leaps in machine learning alorithms these past years and they've only gotten better and better at recognizing handwritten text. I remember reading once that some algorithms have ...
1
vote
0answers
47 views

Finding a line shape objects (bars) in video images using neural networks (instead of Hough transform)

I'm currently trying to learn about computer vision stuff and I want to try it on a simple project of real time tracking of figures of players in a foosball game while the camera can be moving a ...
1
vote
0answers
26 views

Hebbian rule doesn't get to a fixpoint

I'm trying to implement an Hopfield Network for pictures of 32x32 bits either 1 or -1; I have these 3 pictures and I transform each of them in a vector of 1024 elements. Then I take the 3 vectors and ...
1
vote
0answers
520 views

Using a neural network to identify patterns in strings

I am currently doing a research on data sensing and I came across the following concept. The requirement for me is to identify different data types using neural networks. Please note that I don't want ...
1
vote
0answers
48 views

Training algorithm for Manual error correction in text

I want to work on a model where I make some manual corrections in my clinical notes data and want the neural network to learn those corrections.The algorithm will be supplied with both corrected and ...
1
vote
1answer
168 views

What is the difference between distributed representations and factors of variation in deep learning

In the field of deep learning, people often talk about factors of variation which, in my understanding (in terms of dimensionality reduction), are the latent variable directions capturing variability ...
1
vote
0answers
32 views

Is parameter tying more common in RNNs than in regular NNs?

From what I can see it seems like RNNs favour using a backprop through time method which I haven't seen really applied to other neural networks. Can someone explain the significance of tying the ...
1
vote
0answers
511 views

Neural Networks: Style transfer for audio

I just found the paper A Neural Algorithm of Artistic Style which seems to be very cool. It describes a way to combine two images: Use the content of image A and the style of image B to create a new ...
1
vote
0answers
199 views

About Barron's theorem in neural nets

I wanted to clarify my understanding that both the theorems of Barron about approximating functions by neural nets are about one hidden layer net. Is this right? Is there any Barron's theorem for ...
1
vote
0answers
451 views

Class activation mapping using inception-v3 network

I am using the technique of this paper: https://arxiv.org/pdf/1604.07953.pdf to localize food objects. The difference is that I am using the next generation of google-net, inception-v3 network. For ...
1
vote
0answers
48 views

Relation between combining factors and disentangling for face recognition

I've read two interesting research papers on face recognition and I'd like to understand the relation between them. "Learning to Disentangle Factors of Variation with Manifold Interaction" by Reed et ...
1
vote
0answers
363 views

Noise injection to fight overfitting

I've been researching the specifics of adding noise injection to my input layer to fight against overfitting. It seems as though most papers recommend a Gaussian noise vector with mean of 0 and a "...
1
vote
0answers
57 views

What are the most desirable properties of a neural network? [closed]

I'm trying to compare a custom neural network architecture with other existing ones. I'm quite new to the CS field and I'm looking for desirable properties and/or applications of neural networks(...
1
vote
0answers
524 views

Text data comparison

Okay lets say i have two data structures . two phone data for example containing their Name and spec ( cpu , ram , display etc ) . I want to check if these two phones are the same or not . Their names ...
1
vote
0answers
107 views

Teaching perceptrons colors? [closed]

I am learning about artificial neural networks and I've decided to go with perceptrons. I already made a sample program that can learn based on the learning data, but when I tried to make it recognize ...
1
vote
0answers
61 views

How to calculate activation of hidden nodes in a recurrent neural network?

Usually, when I program recurrent neural networks, I use a loop for each neuron to figure out it's state. What I realized with this is that in this case, no neuron gets any feedback. They just pump ...
1
vote
0answers
301 views

How can neural networks learn to create new things (sentences for example)? [closed]

I have already taken a college course at my uni on machine learning where we implemented all the basic ML programs: linear regression, logistic regression, basic neural network with logistic ...
1
vote
0answers
178 views

Computer vision training procedures: SVM/AdaBoost vs Neural Networks

With SVM, adaboost or similar alogrithms, image training sets must be cropped with specific constraints (keep image cropping ratio the same, have object tightly cropped, same resolution) In general, ...
1
vote
0answers
228 views

Hessian-Free instead of LSTM for Recurrent Net Machine Translation

Last year, Ilya Sutskever and collaborators came out with a paper about a recurrent LSTM net that learns sequence to sequence mappings for machine translation. It's somewhat surprising that the ...
1
vote
0answers
56 views

How do NEF networks differ from ANNs in applications?

As I've described here, the Neural Engineering Framework (NEF) has some functional similarities to Artificial Neural Networks (ANNs). Naturally, there is also an overlap in applications between NEF-...
1
vote
0answers
794 views

Improving MSE as fitness function for a genetic algorithm

I am implementing an autoencoder neural network in matlab, the weights of which are being optimised by a genetic algorithm. At the moment I am working on the first layer, trying to get an improved ...
1
vote
0answers
14 views

What types of images should I use for negative examples in a classification problem? [duplicate]

I am doing a project to recognize a kind of leaf using ANNs with Emgu CV in C#. My project is to get frames from camera then present them to the ANN and have the ANN tell me if that frame contain a ...
1
vote
2answers
574 views

How to determine if a state is a fixed point in a Hopfield network?

I have been reading a lot and I am still unsure of how to determine this. Let's say I have an initial binary state vector (1, 1, 1). How would I go about determining whether (1, 1, 1) is a fixed point ...
0
votes
2answers
56 views

Convolutional Neural Network with constant kernels

I'm starting to learn about CNNs, and I have this question that I haven't been able to answer. Sorry if it is too basic. I know that in a CNN, the network learns to extract relevant features of ...
0
votes
2answers
874 views

What are the challenges of using gradient descent on hyper parameters λ and η to find out their optimum values?

A question from chapter 3 of Michael Nielsen's [Neural Networks and Deep Learning]: It's tempting to use gradient descent to try to learn good values for hyper-parameters such as the regularization ...
0
votes
2answers
93 views

Training a neural net without labels in Reinforcement Learning

I am trying to dig further into machine learning and I am making a program to play a game as a start. I have created a game that is based on the mobile game Flappy Bird and can be generalized to the ...
0
votes
1answer
147 views

max, min gradient and other terms in Neural Network

This link contains a demo that trains a Convolutional Neural Network on the MNIST digits dataset in browser. I am not getting below terms- 1. max, min gradient in each layer. 2.max, min activation ...
0
votes
1answer
376 views

Why is an Artificial Neural Network with high accuracy score giving poor results?

I implemented my ANN using SKlearn module's class MLPClassifier. Fitting it on some data and testing it on a very specific subset of said training data, it gives a score of 1.0, but actually using the ...
0
votes
1answer
296 views

Genetic Algorithm Neural Network- Inputs for evolving creatures

I made a GANN program for evolving creatures. The genes that get put into the GA for each ind are the weights that go into the neural net for each creature. The NN is a basic 1 hidden layer NN with ...
0
votes
2answers
247 views

Trying to Classify MNIST where pixels are shuffled with a seed.Why is it not working?

I transformed the MNIST dataset as follows:(X (70000 x 784) is the training matrix) ...
0
votes
1answer
264 views

Genetic algorithm neural networks converges, but suddenly stops

I'm trying to create a genetic algorithm to train neural networks (because I'm to bad at back-propagation), and it works well until generation 18, where the loss stops to decrease and gets constant. ...
0
votes
1answer
541 views

Methods to prevent overfitting

I am aware of three approaches to prevent over-fitting of data when trying to model it on a neural net. The first two approaches I know suggest to train on more data and employ bootstrap aggregating. ...
0
votes
1answer
224 views

Comparative study between Deep neural nets and Bayesian Networks

Is there any comparative study that showcases the powers of Bayesian Networks and Deep learning in their respective favorable setup and how they compare? I tried to go through blogs but couldn't find ...