# Questions tagged [neural-networks]

Network structure inspired by simplified models of biological neurons (brain cells). Neural networks are trained to "learn" by supervised and unsupervised techniques, and can be used to solve optimization problems, approximation problems, classify patterns, and combinations thereof.

492 questions
Filter by
Sorted by
Tagged with
10 views

### Dilated LSTM vs LSTM

I'm currently reading a paper for a literature review related to NLP and its research employs a dilated LSTM model. What is the difference between this and a regular LSTM, or more specifically what ...
1 vote
28 views

### Understanding gradient flow of a linearized wide neural network

I've been trying to fully understand the paper "Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent" (available here), but I'm stuck on the linearization part, ...
1 vote
34 views

### At what precision are GPT's parameters stored?

OpenAI's technical paper on GPT-3 says that GPT-3 has 175 billion parameters. Several sites (here, here) claim that these parameters are stored as single-precision floating-point numbers and so ...
10 views

1 vote
34 views

### Convergence of graph neural networks (GNNs)

I'm new to the area, and we don't have a course on graph neural networks at our university. However, I will still like to know the main theoretical results when considering convergence of graph neural ...
1 vote
39 views

### Can we supervise on the hidden states of RNN?

I'm trying to generate some history-dependent model with machine learning, whose underline physical model has a clear definition of its "internal state variable" (a state derived from ...
287 views

### Is ANN a data structure or an algorithm?

When I read about Artificial Neural Networks (ANN), no one says what ANN is. For instance, Wikipedia says: Artificial neural networks (ANNs), usually simply called neural networks (NNs), are ...
1 vote
239 views

### How many pixels support each neuron in multi-layer CNN?

I'm studying for a computer vision module and I'm on the deep learning topic, in one past paper we have the following question: Given that a convolutional neural network has five convolution layers (...
62 views

### Universal approximation bounds of the form $\|f(x)-\hat{f}(x;w)\|\leq \varepsilon \|f(x)\|$

It is known that for every $\varepsilon>0$ there is an appropriate neural network architecture, such that one can approximate any continuous function $f:[0,1]^n\to[0,1]^m$ by the neural network ...
152 views

### What are open-loop and closed-loop modes of neural networks?

I came across the following line in the book ‘Deep Learning (Ian Goodfellow) 10.2.1, pg 374; The disadvantage of strict teacher forcing arises if the network is going to be later used in an open-loop ...
344 views

### How to transform an Abstract Syntax Tree (AST) to an Abstract Binding Tree (ABT)? (for machine learning fo theorem proving)

I was reading the HOList paper that applies Graph Neural Networks (GNNs) to the HOL Light (HOList) data set and benchmark for ML for theorem proving. They describe their results etc but there is no ...
150 views

### Discrete action space for MADDPG

I am trying to apply MADDPG, a policy gradient algorithm that uses centralized training and decentralized execution, to a project. In the work of Lowe et al., the actor returns a pmf for a discrete ...
72 views

### Raising to the T in machine learning

What does it mean when in a machine learning paper there is $(arg)^{T}$, what does the T does to an arg in this 3b1b video on neural networks he puts the: $(w^{l-1})^{T}$
1 vote
144 views

### Which features can be considered for neural network based SAT solving?

I'm trying to implement SAT solver, based on backtracking algorithm and BCP. This SAT solver is trying to pick one literal from each clause, from 3-CNF SAT instances. I've implemented a neural network ...