A message from our CEO about the future of Stack Overflow and Stack Exchange. Read now.

Questions tagged [neural-networks]

Network structure inspired by simplified models of biological neurons (brain cells). Neural networks are trained to "learn" by supervised and unsupervised techniques, and can be used to solve optimization problems, approximation problems, classify patterns, and combinations thereof.

Filter by
Sorted by
Tagged with
0
votes
1answer
18 views

Using a point as an input for a perceptron

Can a point be used as the input for a perceptron/neural net? The relationship between the two numbers that make up a 2D point does not affect the output, but does this not matter when the ...
2
votes
0answers
15 views

Information bottleneck function

On the Wikipedia page, the information bottleneck function is defined in the following way. For some random variable $X$ containing information about the relevant variable $Y$, we have some joint ...
0
votes
1answer
57 views

How can we get small test error reducing only train error?

My question is about mathematical part of machine learning algorithms, especially about using it in neural networks. We train network reducing train error and I was thinking about how then test error ...
5
votes
2answers
207 views

Why does the effectiveness of my reinforcement based neural network recede after a while?

I have a reinforcement based neural network training on the OpenAI gym CartPole-v1 environment. For the structure and training algorithm, assume it is the same as the one in this article. Typically, ...
0
votes
0answers
15 views

Training a model from self-play

Say I'd like to build a Go AI. The Go AI takes in the board state and then predicts who's more likely to win from that state. When I want to make a move, I just test every next board state I could ...
1
vote
1answer
15 views

How does the forget layer of an LSTM work?

Can someone explain the mathematical intuition behind the forget layer of an LSTM? So as far as I understand it, the cell state is essentially long term memory embedding (correct me if I'm wrong), ...
1
vote
1answer
32 views

Text detection in computer vision

I'm curious about the way text recognition works in machine learning(or more generally, the question of object vs not object) in computer vision. How are systems trained when the not-object data set ...
1
vote
1answer
25 views

Weighted Average of Neural Networks with Cross Entropy Cost Function

What is the best way to create an ensemble of neural networks utilising weighted averaging when these networks were trained to minimise cross entropy error function? The literature I found (e.g. [1]) ...
2
votes
1answer
44 views

Weighted Average of Multi-Output Neural Networks

[1] discusses how to construct an ensemble of neural networks by giving each network a certain weight $\alpha_i$: \begin{equation} f_\mathrm{GEM}(\boldsymbol{x}) = \sum_{i=1}^N \alpha_i f_i(\...
5
votes
1answer
176 views

Predicting next action to take to reach a final state

Does anyone know of an algorithm that could be used to determine the next action to take to reach a desired state when trained on time-series data? For example, a robot starts at a certain state, ...
0
votes
1answer
33 views

How do we fix area of detectors in object detection?

I have gone through various articles on medium and also some from other sites trying to understand SSD. I am able to figure out most of the things from articles except this one. They always say that ...
0
votes
1answer
1k views

How to feed videos to a neural network

I have been coding and testing Neural Networks for a while but as of now I have only used IMAGE datasets. (i.e. I have M training images and N testing images). Some datasets are video datasets. The ...
1
vote
2answers
57 views

Learning a perceptron from stream data

I want to train a Perceptron using stochastic gradient rulefrom the stream data. I have very limited amount of memory and i can store only $N$ examples. Suppose my population consist of point as ...
2
votes
1answer
27 views

Why do we need to take the derivative of the activation function in backwards propagation?

I was reading this article here: https://towardsdatascience.com/how-does-back-propagation-in-artificial-neural-networks-work-c7cad873ea7. When he gets to the part where he calculates the loss at ...
0
votes
1answer
23 views

How does image reconstruction take place in neural network?

I am reading through and thinking about how neural network works and have been reading about convolutional neural networks (CNN). I am particularly interested in image filtering (or enhancing) using ...
0
votes
0answers
24 views

Are neural network latent representations fitting a Gaussian distribution?

Neural network latent (or pre-activation) representations are the weighted sums of inputs to neurons in hidden layers before applying an activation function. The vector of representations of neurons ...
0
votes
1answer
40 views

Formal definition of loss surface of multi-layered networks

Let $\mathcal{L}$ be a loss function associated with a multi-layered neural network. So it seems almost everyone in AI/ML community is interested in the Hessian $H=\partial^2 \mathcal{L}$ of $\...
0
votes
1answer
47 views

Explain this Neural Network used for Knowledge Graph Embedding

I am interested in better understanding the neural network used in the paper LogicENN A Neural Based Knowledge Graphs Embedding Model with Logical Rules. To my knowledge this is the most advanced and ...
1
vote
1answer
54 views

What does “Temporal extent” mean?

I am reading Long-term Temporal Convolutions for Action Recognition and under the Section 3.1, I read this: To investigate the impact of long-term temporal convolutions, we here study network ...
2
votes
1answer
84 views

Learning the activation function in a neural network?

Neural networks use specific activation functions, commonly used ones are tanh, ReLu. I have seen that people have experimented with continuously parametrices activation functions, for instance here. ...
0
votes
0answers
30 views

Cross entropy minimization

I just read about Cross-entropy cost function for my work. and I see a notation that said "the minimization of cross entropy of some data is equal to maximization of their log likelihood." How can I ...
10
votes
4answers
5k views

Evolving artificial neural networks for solving NP problems

I've recently read a really interesting blog entry from Google Research Blog talking about neural network. Basically they use this neural networks for solving various problems like image recognition. ...
3
votes
2answers
757 views

Why can Multilayer neural networks solve non-linear problems

I understand what a multilayer neural network is, but what about them allows them to solve non-linear problems unlike perceptrons? Is it the fact that they can extend to any number of outputs/hidden ...
0
votes
0answers
28 views

CNN Predicting One Class and Accuracy Getting Stuck

My model is a binary classifier. With the same exact architecture, the model sometimes gets high accuracy (90% etc), other times it predicts only one class (so accuracy is stuck at one number the ...
1
vote
1answer
729 views

Guided Backpropagation in Deep Neural Networks

I am reading about a method called guided backpropagation. https://ramprs.github.io/2017/01/21/Grad-CAM-Making-Off-the-Shelf-Deep-Models-Transparent-through-Visual-Explanations.html#deconv-and-guided-...
1
vote
1answer
32 views

Neural network for PDE: Should we train the PDE using more initial and boundary data at the beginning?

I was trying to solve a partial differential equation (PDE) using a neural network. The solution to the PDE is not unique unless the boundary condition is determined. In my case, the neural network ...
1
vote
0answers
7 views

Is there a way to connect a deep language model output to input?

In models like GPT-2, TXL and Grover, is there a good way to know which input weights (tokens) resulted in each token of the output?
0
votes
0answers
14 views

Risk score from Neural Network classifier (more than 2 categories)

I am trying to use a Neural Network to perform multiclass classification. The classes represent Insurance Risk Level. The most risky level is Level 1, the least risk corresponds to Level 10. The ...
0
votes
0answers
9 views

Deep Learning Technique for Image to Video Conversion

I'm trying to build an engine for the following task: I have n videos, from which I've taken 1 snapshot each. I am trying to train a classification algorithm on these n snapshots. Till now I have ...
1
vote
2answers
79 views

Best practices for normalizing up training, validation, and test sets

I was reading up on how to normalize my training, validation, and test sets for a neural network, when I read this snippet: An important point to make about the preprocessing is that any ...
0
votes
0answers
30 views

Graph neural network

I'm trying to build GNN model that classify images , the first step is to model each image with graph , each node represents one pixel , now how can I define the edges in my case ? does the spatial ...
0
votes
1answer
25 views

Which algorithm for predicting the next word(s) based on previous words, given a sentence?

I want to input some words, and out comes the next word(s). Neural nets are really hot at the moment, and I'm afraid of throwing a neural net at something, when one is not really needed. Or... maybe ...
2
votes
1answer
21 views

Neural network game players and incremental updates

Neural networks in recent years have been successfully used for gameplaying. A difference between games and e.g. image processing is that the game boards get updated incrementally. Do any neural ...
2
votes
2answers
101 views

How deep do neural networks need to be?

My question is a bit on the philosophical side, and there is probably not one single 'correct' answer on this. Nonetheless, I'm curious to hear your opinion... I'm currently designing a convolutional ...
0
votes
1answer
21 views

How to initialize the first h in an RNN?

Take a Vanilla RNN represented by the function $h_t = f(h_{t-1}, x_t)$, how do you determine $h_0$? Edit: This answer over on the stats page has helped.
0
votes
0answers
38 views

On the growth rate of Leela Zero compared to AlphaGoZero

There are not many sources online, but one reference from January says of Leela Zero (LZ) that: The strength depends on the hardware and on thinking time, but from the thread "LeelaZero ...
0
votes
0answers
20 views

How to transform an arbitrary graph into a fixed vector representation?

Actuality I work in computer vision, specifically on a problem known as "scene graph modeling." This problem aims to convert an image $I$ in a graph $G=(V,E)$ where the nodes $V$ represent the objects ...
0
votes
0answers
8 views

train mlp (input x, outpu y) with a blackbox labeling any x-y pair good/bad

Standard MLP (map x to y) is trained with a set of x-y data points. My question: What if there is no train data, the only supervisor lables any x-y pair with 0 or 1. The goal is that the x-y pair ...
0
votes
0answers
11 views

Model suggestion for detection of malware based on multiple api call sequences

I'm trying to build a RNN (LSTM) model for classification of binary as benign/malware. The data structure I've presently looks as follows ...
7
votes
4answers
9k views

Traveling Salesman Problem with Neural Network

I was curious if there were any new developments in solving the traveling salesman problem using something like a Hopfield recurrent neural network. I feel like I saw something about recent research ...
1
vote
0answers
20 views

Value flow (and economics) in stacked reinforcement learning systems: agent as reinforcement environment for other agents?

There is evolving notion of stacked reinforcement learning systems, e.g. https://www.ijcai.org/proceedings/2018/0103.pdf - where one RL systems executes actions of the second RL system and it itself ...
1
vote
1answer
41 views

What is the most efficient way to test whether a set $X \subset \{0, 1\}^n$ and its complement $\{0, 1\}^n \setminus X$ are linearly separable?

I am interested in algorithms that have optimal running time, and ideally which are also very easy to implement. If you can also give some tips on how to implement the algorithm(s) you mention in the ...
0
votes
0answers
7 views

Query about an equation in GAN-NMT paper

So i was studing the paper Adversarial Neural Machine Translation by Lijun Wu1, Yingce Xia2, Li Zhao3, Fei Tian3, Tao Qin3, Jianhuang Lai1,4 and Tie-Yan Liu. The link to the paper is : https://arxiv....
0
votes
1answer
667 views

Single Layer Perceptron vs Multi Layer Perceptron

Why the single layer perceptron has a linear activation function while the Multi Layer Perceptron has a non-linear activation function ? What is the potential of the Multi Layer Perceptron respect of ...
2
votes
1answer
24 views

Is it still transfer learning if you consider input as well as output? (neural networks)

I'm new to the CS stack exchange, so a fond hello to you all! I joined since I have a question I've been curious about. I have recently been running some experiments in transfer learning - ...
0
votes
0answers
7 views

Constructing a 3D virtual map from 2D Depth Maps and $x, y, z, \theta$ coordinates

Context: I intend on building a quadrotor which can generate a 3D virtual map of an area of which it captures photos with a 2D camera. I plan on first training a Convulational NN to return a depth ...
0
votes
0answers
50 views

Training of two 3x3 convolution layers vs training one 5x5 convolution layer

I'm not 100% sure this is the right stackexchange, please feel free to redirect me to another one. I know that two 3x3 convolution layers can be equivalent to one 5x5 convolution layer. I also know ...
1
vote
0answers
32 views

Solving analytic gradient of loss function for neural networks [closed]

Please note that I am talking in about theory rather than ''what someone would do in a real, practical situation''. Given a multi-layer Perceptron with at least 1 hidden layer, and sigmoid (or other ...
0
votes
0answers
18 views

Computing analytic gradients for NN cost function with 1 hidden layer

Given a simple artificial neural network with 1 hidden layer, I want to compute the analytic gradient, to gain a better understanding. Using a simple loss function L such as: $L=(1/N)\sum_{k=1}^N|| \...
0
votes
0answers
75 views

Can someone explain the difference between MCP neurons and Perceptrons?

I am getting confused with the literature. Is a perceptron simply a network of MCP neurons? From what I understand, in 1957 Rosenblatt developed the perceptron based on relaxed constraints from the ...