Questions tagged [neural-networks]

Network structure inspired by simplified models of biological neurons (brain cells). Neural networks are trained to "learn" by supervised and unsupervised techniques, and can be used to solve optimization problems, approximation problems, classify patterns, and combinations thereof.

Filter by
Sorted by
Tagged with
1
vote
1answer
110 views

Algorithm Selection for Classification problem

I've been working on a developing a product selection network for my workplace. I work with lots of chemicals and the clients don't always know what they want/need so most of the time I have to ask a ...
1
vote
1answer
74 views

Is learned RNN still universal?

I know it has been shown that RNNs are Turing complete. So for any Turing machine, there exists a configuration of a RNN that is equivalent to it. But I'm wondering, is all those configuration ...
1
vote
1answer
122 views

Why is low-dimensional discrete representation of input space useful?

Self-organizing maps undergo unsupervised training to produce a low-dimensional, discrete representation of the input space. Why is this useful? I think the results of the self-organizing map would ...
1
vote
1answer
97 views

Convolutional Neural Networks - Where should I start from?

I will soon be starting a part-time job within a start-up and I've been assigned to a project where I'll have to build a CNN that recognizes and counts people in various kinds of videos. My CEO knows ...
1
vote
1answer
183 views

Back propagation in neural networks

I just finished watching these 3 Coursera videos on back propagation in neural networks. I get the idea of what we're trying to do, but I don't get how we achieve that by calculating error in each ...
1
vote
1answer
210 views

When a trained RNN is tested, is the number of time-steps same for every input?

I am a beginner in deep learning so bear with me. If I want to unfold the RNN in order to represent the relation of output to the input as a non-recursive functions I would have to know the number of ...
1
vote
1answer
992 views

Difference between SNN RL and DNN RL?

In Reinfrocement Learning (RL) in Neural Networks (NNs), I've seen two approaches to Q-learning. The first is to tile the state space with basis functions using Spiking Neural Networks (SNN) to ...
1
vote
1answer
255 views

Using a combination of spatial and non-spatial inputs for convolutional neural networks

I'm working on training a game AI using deep reinforcement learning to achieve specific examples based on pixel input and some additional state information. Naturally, I'm using a convolutional ...
1
vote
1answer
566 views

How is the Delta Rule derived in neural networks and what is the explanation for the algebra?

I am currently trying to learn how the Delta Rule works in a neural network. So far I completely understand the concept of the Delta Rule, but the derivation doesn't make sense. My question is how is ...
1
vote
1answer
63 views

Backpropagation on a matrix of functions

As nicely described in this article, backpropagation for multi-layer perceptrons defines the error term for a neuron in terms of the partial derivative of the weights. It's traditional to represent ...
1
vote
1answer
176 views

Creating a single layer perceptron for the OR problem

I am working on the following problem Find the linear least squares unit weights for the `OR' problem, ie. $v_1^T = (0,0), v_2^T = (1,0), v_3^T = (0,1), v_4^T = (1,1)$ and $u_1 = 0, u_2 = u_3 = u_4 = ...
1
vote
1answer
85 views

backpropagation algorithm seems to be forcing output values to middle than extremes

I have been playing around with artificial neural networks lately, specifically with the prospect of trying to replace the contrastive divergence algorithm with some type of evolutionary metaheuristic ...
1
vote
1answer
937 views

Neural Network Weights per input nodes

Sorry, I'm really new to neural networks and this question is probably pretty obvious. If you have any resource that can help clarify these concepts to me it would be much appreciated. The way I ...
1
vote
1answer
50 views

Matching training with Neural Network

I have a matching algorithm that is based on making an comparison score. This score is divided into parts. Example: 5 - Points for attributes (lets say they have 3 common attributes, would the score ...
1
vote
1answer
71 views

What's the purpose of the “o(1-o)” in the back propagation algorithm

I'm not sure what the purpose of the o(1-o) in the back propagation algorithm achieves? I'm guessing it's related to using the sigmoid function on the output but I'd like to have a proper ...
1
vote
1answer
137 views

Multi-dimensional Neural Network for fingerprint matching

I want to use “Fingerprint matching using multi-dimensional ANN” by Rajesh Kumar and B.R. Deva Vikram [content link] for fingerprint identification. But I have a serious problem understanding what is ...
1
vote
0answers
29 views

How can Kneser-Ney Smoothing be integrated into a neural language model?

I found a paper titled Multimodal representation: Kneser-Ney Smoothing/Skip-Gram based neural language model. I am curious about how the Kneser-Ney Smoothing technique can be integrated into a feed-...
1
vote
0answers
10 views

Why do we need to change the (weight decay) regularization parameter when changing the number of inputs that neural network is being trained with?

I am currently working my way through Michael Nielsen's ebook Neural Networks and Deep Learning and I am reading about overfitting and (L2) regularization. In this subsection, the process of L2 (a.k.a ...
1
vote
1answer
19 views

Learning a specific functional form with machine learning

Suppose I have only three independent features (x, y, z) as the input to some machine learning routine (e.g. neural network). From some domain knowledge, I know ...
1
vote
0answers
7 views

Is there a way to connect a deep language model output to input?

In models like GPT-2, TXL and Grover, is there a good way to know which input weights (tokens) resulted in each token of the output?
1
vote
0answers
22 views

Value flow (and economics) in stacked reinforcement learning systems: agent as reinforcement environment for other agents?

There is evolving notion of stacked reinforcement learning systems, e.g. https://www.ijcai.org/proceedings/2018/0103.pdf - where one RL systems executes actions of the second RL system and it itself ...
1
vote
0answers
42 views

Solving analytic gradient of loss function for neural networks [closed]

Please note that I am talking in about theory rather than ''what someone would do in a real, practical situation''. Given a multi-layer Perceptron with at least 1 hidden layer, and sigmoid (or other ...
1
vote
1answer
111 views

What does “Temporal extent” mean?

I am reading Long-term Temporal Convolutions for Action Recognition and under the Section 3.1, I read this: To investigate the impact of long-term temporal convolutions, we here study network inputs ...
1
vote
0answers
120 views

Converting (reverse-engineering) Turing machine into program or most concise algorithm?

It is known that every program or every algorithm can be converted to Turing machine. But what about the reverse process? Is there algorithm (or research trend that considers such algorithm) to ...
1
vote
0answers
344 views

Why is the method of im2col with GEMM is more efficient than the method of direction implementation with SIMD in CNN

The convolutional layers are most computationally intense parts of Convolutional neural networks (CNNs).Currently the common approach to impement convolutional layers is to expand the image into a ...
1
vote
0answers
143 views

Proof of perceptron convergence theorem for ZERO threshold?

The generalized perceptron convergence theorem is for a defined threshold T. When you do the maths it all comes to an upper bound and a lower bound. The lower bound looks like this! Therefore ...
1
vote
0answers
303 views

How can node2vec help find similar “roles” within a graph (nodes whose connections have similar structure within the graph)?

I have a question on the node2vec algorithm described in this paper. Node2vec is a deep learning algorithm that word2vec to graphs to learn embeddings. The authors claim that it can help find nodes ...
1
vote
0answers
32 views

How would you go about creating a algorithm that should generate a shakespearean sonnet on any given theme

I need to create an algorithm that is going to create a shakespearean sonnet for a specific theme. This theme should be generated out of twitter tweets that have some hashtag. My current idea goes ...
1
vote
0answers
533 views

Backward mapping with bilinear sampler

I have some experiences with Convolutional Neural Networks before. I have a question regarding the Bilinear Sampler used in "Unsupervised Monocular Depth Estimation With Left-Right Consistency" (the ...
1
vote
0answers
38 views

Does a deep feedforward neural net approximate a (single or multivariate)polynomial?

Does composition of several linear transformation plus non-linear activation function in each layer and different layers (as they are in feedforward neural net) represent or approximate a ...
1
vote
0answers
148 views

Recurrent neural networks (Hopfield-like) with short limit cycles

Standard Hopfield networks exhibit stable patterns (states) which are attractors of a dynamic system. I wonder how to modify standard Hopfield networks such that they exhibit stable limit cycles as ...
1
vote
0answers
22 views

How do RNN's map variable-length sequences to variable-length sequences?

According to Karpathy's blog "The Unreasonable Effectiveness of Recurrent Neural Networks", recurrent neural networks can map variable-length sequences to variable-length sequences, as shown by the ...
1
vote
0answers
200 views

Distributing Neural Network Training World-Wide Using Blockchain?

I just finished reading a white-paper from a recent AI startup. The company, Deep Brain Chain, wants to distribute neural network training over computers worldwide, using blockchain technology. Here ...
1
vote
0answers
108 views

adjusted fitness in NEAT algorithm

I'm learning about NEAT from the following paper: http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf I'm having trouble understanding how adjusted fitness penalizes large species and prevents ...
1
vote
0answers
46 views

Detecting shapes using CNNs in respect to other shapes in the image

I'm solving a problem of detecting an optic disc in a retina image. As you can see from the image: the optic disc is the epicentrum of the blood vessels, has an irregular circular shape and has a ...
1
vote
0answers
144 views

Can CNN be used for “unknown correlated” data?

Now the title may be misleading, but I don't know how to give it a better name. My question is if I have data in a 2d matrix and under the assumption that: -I know for sure that there are ...
1
vote
0answers
194 views

If we can train a ML algorithm to recognize letters with 95% accuracy, why does OCR software still suck? [closed]

So I've seen all these leaps in machine learning alorithms these past years and they've only gotten better and better at recognizing handwritten text. I remember reading once that some algorithms have ...
1
vote
0answers
50 views

Finding a line shape objects (bars) in video images using neural networks (instead of Hough transform)

I'm currently trying to learn about computer vision stuff and I want to try it on a simple project of real time tracking of figures of players in a foosball game while the camera can be moving a ...
1
vote
0answers
31 views

Hebbian rule doesn't get to a fixpoint

I'm trying to implement an Hopfield Network for pictures of 32x32 bits either 1 or -1; I have these 3 pictures and I transform each of them in a vector of 1024 elements. Then I take the 3 vectors and ...
1
vote
0answers
569 views

Using a neural network to identify patterns in strings

I am currently doing a research on data sensing and I came across the following concept. The requirement for me is to identify different data types using neural networks. Please note that I don't want ...
1
vote
0answers
53 views

Training algorithm for Manual error correction in text

I want to work on a model where I make some manual corrections in my clinical notes data and want the neural network to learn those corrections.The algorithm will be supplied with both corrected and ...
1
vote
1answer
185 views

What is the difference between distributed representations and factors of variation in deep learning

In the field of deep learning, people often talk about factors of variation which, in my understanding (in terms of dimensionality reduction), are the latent variable directions capturing variability ...
1
vote
0answers
33 views

Is parameter tying more common in RNNs than in regular NNs?

From what I can see it seems like RNNs favour using a backprop through time method which I haven't seen really applied to other neural networks. Can someone explain the significance of tying the ...
1
vote
0answers
511 views

Neural Networks: Style transfer for audio

I just found the paper A Neural Algorithm of Artistic Style which seems to be very cool. It describes a way to combine two images: Use the content of image A and the style of image B to create a new ...
1
vote
0answers
219 views

About Barron's theorem in neural nets

I wanted to clarify my understanding that both the theorems of Barron about approximating functions by neural nets are about one hidden layer net. Is this right? Is there any Barron's theorem for ...
1
vote
0answers
474 views

Class activation mapping using inception-v3 network

I am using the technique of this paper: https://arxiv.org/pdf/1604.07953.pdf to localize food objects. The difference is that I am using the next generation of google-net, inception-v3 network. For ...
1
vote
0answers
48 views

Relation between combining factors and disentangling for face recognition

I've read two interesting research papers on face recognition and I'd like to understand the relation between them. "Learning to Disentangle Factors of Variation with Manifold Interaction" by Reed et ...
1
vote
0answers
367 views

Noise injection to fight overfitting

I've been researching the specifics of adding noise injection to my input layer to fight against overfitting. It seems as though most papers recommend a Gaussian noise vector with mean of 0 and a "...
1
vote
0answers
61 views

What are the most desirable properties of a neural network? [closed]

I'm trying to compare a custom neural network architecture with other existing ones. I'm quite new to the CS field and I'm looking for desirable properties and/or applications of neural networks(...
1
vote
0answers
532 views

Text data comparison

Okay lets say i have two data structures . two phone data for example containing their Name and spec ( cpu , ram , display etc ) . I want to check if these two phones are the same or not . Their names ...

1
5
6
7 8 9