Skip to main content

Questions tagged [neural-networks]

Network structure inspired by simplified models of biological neurons (brain cells). Neural networks are trained to "learn" by supervised and unsupervised techniques, and can be used to solve optimization problems, approximation problems, classify patterns, and combinations thereof.

Filter by
Sorted by
Tagged with
0 votes
0 answers
16 views

Proof that the hopfield network decreases the energy for simultaneous updates?

A hopfield network is given by an $N\times N$ matrix $w$ satisfying $w_{ij}=w_{ji}$ and operates on $N$-dimensional bitstrings, which we represent as $\{-1,1\}$. If a hopfield network updates each ...
user56834's user avatar
  • 3,982
0 votes
1 answer
25 views

Advice for better performance neural network

As a beginner in neural networks, I'm currently working on a project for predicting winning fighters in MMA matches. I understand that this is a complex task due to the nature of the sport. Currently, ...
BlizZ's user avatar
  • 1
2 votes
0 answers
25 views

Representational power of Neural Neural Networks without a bias term

In a fully connected Neural Network, each perceptron has it's bias term $b$ which is learnt. Often (example, in Linear/ Logistic Regression), when we don't want to treat this bias term explicitly, we ...
Harry's user avatar
  • 21
0 votes
0 answers
26 views

Explanation for the expression of positional encoding in NeRFs

I was reading the NeRF paper recently (https://arxiv.org/pdf/2003.08934.pdf), and under the positional encoding section, I see that the authors propose usage of the following function for transforming ...
user185887's user avatar
1 vote
1 answer
28 views

Is a Convolutional Neural Net a special case of DNN? If so, how can the convolutional layer be modelled?

In literature, Convolutional Neural Nets (CNNs) are presented as a special case of Deep Neural Nets (DNNs) (e.g., here). I do not understand how the convolutional layer can be implemented through a ...
BlockchainThomas's user avatar
10 votes
3 answers
1k views

Resources for studying the mathematical foundations of machine learning, for someone from a math/physics background

I am a soon-to-be physics graduate student with a background in theoretical and experimental cosmology. In my work, I've often found myself applying machine learning models and techniques for the ...
10GeV's user avatar
  • 201
1 vote
1 answer
132 views

Learning eigenvalue decomposition

How would you build a fully connected neural network that learns eigenvalue decomposition efficiently? I wanted to build NNs that can predict certain properties about matrices which are NP-hard to ...
Arjo's user avatar
  • 113
0 votes
0 answers
35 views

Neural networks as building blocks of a computer

I think I have developed a logic circuit which by using combinational logic and flip flops learns to perform the XNOR logic between 2 bits.It is a kind of state machine. Suppose we built a computer ...
Cerise's user avatar
  • 153
0 votes
0 answers
35 views

Dilated LSTM vs LSTM

I'm currently reading a paper for a literature review related to NLP and its research employs a dilated LSTM model. What is the difference between this and a regular LSTM, or more specifically what ...
paul lacher's user avatar
1 vote
0 answers
33 views

Understanding gradient flow of a linearized wide neural network

I've been trying to fully understand the paper "Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent" (available here), but I'm stuck on the linearization part, ...
user161590's user avatar
1 vote
0 answers
107 views

At what precision are GPT's parameters stored?

OpenAI's technical paper on GPT-3 says that GPT-3 has 175 billion parameters. Several sites (here, here) claim that these parameters are stored as single-precision floating-point numbers and so ...
tparker's user avatar
  • 1,116
0 votes
0 answers
11 views

Neural Net Back-propagation: Question on Wikipedia math

I'm reading the Wikipedia article on Backpropagation and have some questions on the math. $$ \frac{d C}{d a^L}\cdot \frac{d a^L}{d z^L} \cdot \frac{d z^L}{d a^{L-1}} \cdot \frac{d a^{L-1}}{d z^{L-1}}\...
Nick's user avatar
  • 175
1 vote
1 answer
53 views

meaning behind activation function

I have been recently thinking about activation functions and the explainability. For sigmoid and tanh activation functions, I am thinking of them to be similar to logistic regression as the output of ...
zhan danning's user avatar
0 votes
0 answers
99 views

Create a simple Neural Network of n layers in python from scratch with numpy to solve XOR example problem using Batch Gradient Descent

I'm a young programmer that was interested by machine learning. I watched videos and read articles about the theory behind simple neural networks. However, I can't manage to set it up correctly. I've ...
NolanGio's user avatar
0 votes
1 answer
31 views

Calculation in paper of learning rate of neural network

The rate at which a neural network learns something depends on the loss and the backpropagation functions,neural network with different loss and backpropagation functions learn differently at a ...
Volpina's user avatar
  • 97
3 votes
1 answer
1k views

As far as we know, what does GPT-4's training data look like?

I was asking ChatGPT local history questions, because I knew many of the answers and could test the robot. But lots of details were wrong and while it can write citations, it won't disclose its ...
Ethan Miller's user avatar
1 vote
0 answers
76 views

Draw the decision boundary of an neural network

I'm really confused when it comes to what the difference between bias and threshold is. I have read that they are basically the same thing and that they serve the same purpose. Being on opposite sides ...
kim120's user avatar
  • 21
2 votes
0 answers
17 views

Finding Large Subset of Variables Approximately Equal, Low-Cost with Neural Network

I was reading David J.C. MacKay's Information Theory, Inference and Learning Algorithms(pdf available in the link), and in section 16.4 of Chapter 16: Message Passing, he writes about how only ...
Micah's user avatar
  • 21
1 vote
0 answers
143 views

Tabular Meta-Learning in RL

There are various meta-learning algorithms in RL that are proposed for settings when we have a (deep) neural network and the policy (or the value function) are parameterized as such. Can these methods ...
Perissiane's user avatar
0 votes
1 answer
177 views

How sparsity term in loss function for sparse autoencoder is making hidden units inactive?

I am working on a Sparse Autoencoder but Andrew NG's notes are hard to understand. My question is about the following equation: Loss Function. In sparse autoencoder, the goal is to inactive some ...
p200401Samuel's user avatar
2 votes
3 answers
179 views

Does ChatGPT use specific sub-programs

I have had a somewhat hard time trying to understand how ChatGPT can "solve" some tasks that cannot be entirely cast as language-model-based rephrasing of textual subsets of the internet ...
Radio Controlled's user avatar
0 votes
1 answer
31 views

Equivalence of echo state networks and DFAs/NFAs

Echo state networks are theoretically equivalent to DFAs/NFAs, but how would you use an ESN to parse a regular language? Would you just feed many different input strings, some from the language and ...
Gabriel's user avatar
  • 11
0 votes
1 answer
5k views

What are the 175 billion parameters used in the GPT-3 language model?

I am currently working my way through Language Models are Few-Shot Learners , the initial 75-page paper about GPT-3, the language learning model spawning off into ChatGTP. In it, they mention several ...
Lance's user avatar
  • 2,243
-1 votes
1 answer
53 views

Convolutional neural network for non-image data

I am working on a project for the prediction of preeclampsia, and one of the algorithms to be used is the convolutional neural network. I am a beginner in machine learning, and I am struggling to ...
HANEEN Mohammed's user avatar
0 votes
0 answers
17 views

Computer Vision techniques to find the slope of the buildings with respect to the ground?

I want to find the slope of the building with respect to the ground which means that I need to find how tilted is the building with respect to the ground through computer vision or any other ...
Jaskirat Singh's user avatar
1 vote
1 answer
41 views

Are there already Neural Network AIs that can play object-finding games?

Neural networks are good at image analysis. They are also being used in video games which could outperform professional human players. I wonder if they can also play a purely object-finding game where ...
Software Carpenter's user avatar
0 votes
0 answers
22 views

Link prediction in a network using GNNs versus Matrix Completion and other methods

I am looking at some social network analysis and I need to do link prediction. By link prediction I mean that I only have a subset of the relationships in the network, and I need to make predictions ...
krishnab's user avatar
  • 171
2 votes
1 answer
379 views

What are common ways to distinguish between local minima and long run-time in hyperparamteer optimization

I'm using Bayesian optimization for a pretty cost expensive function in the context of neural networks in order to optimize hyperparameters for the neural net. Is there a general, quantitative way to ...
Skeddaddleman's user avatar
1 vote
1 answer
198 views

does local search's State-Flipping Algorithm always terminate?

I am looking at the state flipping algorithm, an algorithm that tries to find a stable configuration in a Hopfield network. The algorithm simply flips the state of unsatisfied nodes as long as the the ...
user206904's user avatar
1 vote
1 answer
43 views

Convolutional Neural Network : how to ignore swaths of uninteresting areas? Or make it focus to interesting regions only?

Imagine you trained a text-from-image reader, similarly to the one described in this article https://medium.com/analytics-vidhya/image-text-recognition-738a368368f5 Now you want to port the trained ...
JohnDeeDoe's user avatar
1 vote
1 answer
71 views

Why does Calibration of NN not lead to higher accuracy?

I have a question about the use of calibration in neural networks and its relation to accuracy. In Guo et al. it says: "Because the parameter T does not change the maximum of the softmax function,...
JamesThomas's user avatar
1 vote
0 answers
43 views

Long and short memory in reinforcement learning Connect 4 AI

I'm writing an AI based on reinforcement learning to play Connect 4. That's my second bot and attempt to RNN and AI (first was copy a code of snake RNN AI from youtube) and I'm looking for some ...
Saguro's user avatar
  • 11
1 vote
1 answer
51 views

Which artificial neural network model can I use when trying to predict gene expression?

I need help choosing a model for my data. I have 50 genes that I need to test for their expression levels. I was thinking of implementing either an SVM or a multilayer perceptron model where the ...
Cassy's user avatar
  • 13
1 vote
1 answer
152 views

Is Self-Modifying Turing Machine equivalent to NTM or TM?

Let SMTM be Turing Machine, but the commands recorded in which can change to others in some random way (for example, choose with a 50/50 probability the command to move to the right or move to the ...
Oleg Chaika's user avatar
0 votes
0 answers
12 views

Reusability of neural network architecture

I have created a neural network architecture by considering the data for one study area. Have performed grid-search to tune almost all the hyperparameters of the model. I want to use the same network ...
Badal's user avatar
  • 101
1 vote
1 answer
37 views

Function or signal as machine learning model output

I am new to machine learning. I've seen that machine learning models such as ANN or SVM can be used to predict scalar outputs taking scalar inputs. I am trying to create a model that has a number of ...
James Craft's user avatar
1 vote
0 answers
95 views

RMSProp Momentum and Decay

I'm making an application of MobileNetV2 and according to their article: We train our models using TensorFlow. We use the standard RMSPropOptimizer with both decay and momentum set to 0.9. We use ...
Ricardo's user avatar
  • 11
0 votes
1 answer
39 views

How can I modify this detail in the article ""Extending Multi-Sense Word Embedding to Phrases and Sentences for Unsupervised Semantic Applications"?"

This question is about de paper Extending Multi-Sense Word Embedding to Phrases and Sentences for Unsupervised Semantic Applications. I am interested in the transformer part of the paper and the main ...
R. S.'s user avatar
  • 177
1 vote
2 answers
48 views

Are the weights between hidden layers changed in backwards propogation? If so, how

If so, I don't understand how, as a large part of back propogation is knowing what the 'real' answer is in comparison to 'predicted' answer. With 1 neuron in a hidden layer, we do not know the 'real ...
Sergei Kiselev's user avatar
1 vote
1 answer
179 views

Predicting continuous variables using image and text input to neural network

I am looking to create a neural network architecture that takes two kinds of inputs: image and text, and outputs a predicted continuous variable. More specifically, I will have multiple images of a ...
Crop's user avatar
  • 11
1 vote
0 answers
204 views

How should I create a word-embedding an NLP model recognizing HTML elements?

I'm currently doing a small-time project where I have to deliver a model which can classify specific elements on a web-page using the HTML code. For this, I have considered using the HTML tags for ...
GoldenChicken's user avatar
0 votes
3 answers
153 views

Mutual Friends in a Network?

I always seem to have trouble finding a formal way to analyze this (be through proofs or whatever). The problem statement is as such: If A and B are friends, and B and C are friends, then A and C are ...
stephen081's user avatar
4 votes
1 answer
39 views

What is the primary reference for the observation/discussion of how neural networks struggle with ambiguous training datasets?

It is known that neural networks, such as convolutional neural networks, struggle with pattern recognition if training sets contain ambiguities (i.e. several labels can correspond to one and the same ...
Capo Mestre's user avatar
1 vote
0 answers
12 views

How long can the short memory last in the RNN?

For a recurrent neural network, the LSTM was a model of how the network worked. However, consider the case where an input was a long paragraph or even an article. $$c_1c_2...c_n$$ where $c_i$ were ...
ShoutOutAndCalculate's user avatar
1 vote
1 answer
40 views

Convergence of graph neural networks (GNNs)

I'm new to the area, and we don't have a course on graph neural networks at our university. However, I will still like to know the main theoretical results when considering convergence of graph neural ...
nir shahar's user avatar
  • 11.6k
1 vote
2 answers
56 views

Can we supervise on the hidden states of RNN?

I'm trying to generate some history-dependent model with machine learning, whose underline physical model has a clear definition of its "internal state variable" (a state derived from ...
mian xiao's user avatar
2 votes
1 answer
429 views

Is ANN a data structure or an algorithm?

When I read about Artificial Neural Networks (ANN), no one says what ANN is. For instance, Wikipedia says: Artificial neural networks (ANNs), usually simply called neural networks (NNs), are ...
user366312's user avatar
2 votes
1 answer
316 views

How many pixels support each neuron in multi-layer CNN?

I'm studying for a computer vision module and I'm on the deep learning topic, in one past paper we have the following question: Given that a convolutional neural network has five convolution layers (...
Yallooss's user avatar
4 votes
0 answers
63 views

Universal approximation bounds of the form $\|f(x)-\hat{f}(x;w)\|\leq \varepsilon \|f(x)\|$

It is known that for every $\varepsilon>0$ there is an appropriate neural network architecture, such that one can approximate any continuous function $f:[0,1]^n\to[0,1]^m$ by the neural network ...
FeedbackLooper's user avatar
2 votes
0 answers
209 views

What are open-loop and closed-loop modes of neural networks?

I came across the following line in the book ‘Deep Learning (Ian Goodfellow) 10.2.1, pg 374; The disadvantage of strict teacher forcing arises if the network is going to be later used in an open-loop ...
Aether's user avatar
  • 141

1
2 3 4 5
10