Questions tagged [neural-networks]

Network structure inspired by simplified models of biological neurons (brain cells). Neural networks are trained to "learn" by supervised and unsupervised techniques, and can be used to solve optimization problems, approximation problems, classify patterns, and combinations thereof.

Filter by
Sorted by
Tagged with
0
votes
0answers
24 views

Papers on depth separations in neural networks

I am new to the domain of Machine learning. I have been asked to present a paper related to the mathematics behind the depth separations in Neural Networks (by Itay Safran, Ronen Eldan and Ohad Shamir)...
0
votes
2answers
25 views

Conv Net dimensions misunderstanding?

I was asked this question: Given an image with shape [1,28,28], what will be the shape of the output of a convolution layer with 10 5x5 kernels (filters) without padding? Now, are the shape dimensions ...
0
votes
0answers
23 views

Good Articles/book on neural networks

I search good articles or books on neural network. unfortunately i don't a lot of time and only now i started to research this field. so i actually search a good source that will give the fundamental (...
0
votes
1answer
21 views

What is the time complexity of the SGVB estimator?

In the context of variational autoencoders, we want to maximize the evidence lower bound and this is typically done using Stochastic Gradient Variational Bayes (SGVB). I was curious if there is any ...
1
vote
1answer
16 views

which neural network is good for predecting the value of strings

I have a dataset that contains some strings. A numeric value is assigned to each string. I want to develop a machine learning (deep learning) model to get a string and predict its value. What neural ...
0
votes
1answer
24 views

How to estimate the computational cost in a neural network?

Given a neural network(assuming no regularisation/dropout), I want to determine the computational cost of doing a forward and a backward pass of a datapoint. I want the measure to be of independent of ...
1
vote
1answer
30 views

Expressivity of neural networks, how much information can be stored

I want to know whether a given neural network (with a finite number of nodes) is able to store all injective maps f: D -> C, where D has cardinality k and C has cardinality N (so the number of maps ...
1
vote
7answers
3k views

Why can't we say that a Neural Network is a NP problem solver?

From this video lecture from MIT https://youtu.be/moPtwq_cVH8?t=1229 there is mention how NP complexity works with finding a "lucky" algorithm and luck can never be accounted for. The ...
0
votes
1answer
11 views

Signal translation with Seq2Seq model

I'm currently doing some research on signal processing and I got a dataset which includes the signal in itself and its "translation". So I want to use a Many-to-Many RNN to translate the ...
2
votes
1answer
33 views

object image segmentation

I have 2 different datasets with similar objects, one where each object is 50 pixels wide and the other where they are 150 pixels. Each photo is 512x512 for both datasets. These two datasets have the ...
0
votes
1answer
23 views

What Happens if I swap the forget gate and update gate in LSTM model?

Consider the following eqautions used in LSTM ( taken from Andrew ng's course on Sequential model) In an LSTM model, LSTM Cell has three inputs at any time step t Input($X_t , a^{(t-1)}, C^{(t-1)})$, ...
1
vote
1answer
24 views

How can one measure the time dependency of an RNN?

Most of the discussion about RNN and LSTM alludes to the varying ability of different RNNs to capture "long term dependency". However, most demonstrations use generated text to show the ...
1
vote
1answer
36 views

Why isn't there just one “keystone” activation function in Neural Networks?

This article says the following: Deciding between the sigmoid or tanh will depend on your requirement of gradient strength. I have seen (so far in my learning) 7 activation functions/curves. Each ...
0
votes
2answers
49 views

Why don't they use all kinds of non-linear functions in Neural Network Activation Functions? [duplicate]

Pardon my ignorance, but after just learning about Sigmoid and Tanh activation functions (and a few others), I am wondering why they choose functions that always go up and to the right? Why not use ...
1
vote
1answer
38 views

How was the σ function chosen to extend the perceptron?

I am just reading about perceptrons in more depth, and now onto Sigmoid Neurons. Some quotes: A small change in the weights or bias of any single perceptron in the network can sometimes cause the ...
1
vote
0answers
35 views

In a machine learning system, why use differentially private SGD if our input data is already perturbed by a DP mechanism?

I'm trying to implement my own version of a deep neural network with differential privacy to preserve the privacy of the parties involved in the training dataset. I'm using the method by Abadi et al. ...
0
votes
0answers
16 views

Using the features embedding of the output from a transformers to represent probabilities of categorical data

I was considering using a transformer, on input data which can be represented as an embedding, so I can use the attention mechanism in the transformer architecture. As my data is of variable input and ...
1
vote
0answers
33 views

How can Kneser-Ney Smoothing be integrated into a neural language model?

I found a paper titled Multimodal representation: Kneser-Ney Smoothing/Skip-Gram based neural language model. I am curious about how the Kneser-Ney Smoothing technique can be integrated into a feed-...
1
vote
2answers
41 views

Why doesn't relu cause back propegation to get stuck

Say you have a neural net that is being trained using back propagation and you are using relu activation. Say the input to a node is a weighted sum of the previous layer with a bias term and say for a ...
4
votes
1answer
24 views

Are non-ordered (Slabbed) Neural Networks in wide modern use and what does Fiesler (1994) mean by Clamping Function and Ontogenic function?

In Fiesler (1994) Neural Network Classification and Formalization, he talks a lot about a more general version of neural networks, one that is not ordered into layers, but rather the network is called ...
0
votes
0answers
35 views

Delta rule for binary step function

I have a question of understanding about the Delta Rule: $\Delta w_i = (y - \hat{y}) \times x_i$ Why does $x$ have to be multiplied again after the difference? If the input is $0$, the product of $w$...
1
vote
1answer
44 views

Can you apply neural networks to design algorithms?

I’m kind of a newbie to neural networks (and CS in general) but I was wondering if there are any methods to apply them in such a way with the aim of producing algorithms that solve difficult math ...
2
votes
2answers
48 views

Zero-Sum Games and Halting Problem

Wikipedia states on the page of the halting problem, "For any program f that might determine if programs halt, a "pathological" program g called with an input can pass its own source and its input to ...
0
votes
1answer
34 views

Performing semantic segmentation on images on word documents

At the moment, I'm making a comparative analysis between two different neural networks for a class. The problem they are both trying to solve is detecting tables on documents. Below is an example of ...
5
votes
5answers
285 views

fast and stable x * tanh(log1pexp(x)) computation

$$f(x) = x \tanh(\log(1 + e^x))$$ The function (mish activation) can be easily implemented using a stable log1pexp without any significant loss of precision. Unfortunately, this is computationally ...
0
votes
0answers
32 views

How to find clusters of a set of points in n-dimensional space that are furthest from unwanted points

I have a list of 25 points and their coordinates in a 512-dimensional space. I have 8 target points and 17 points I need to avoid (the 17 points to avoid also have differences in priority of how ...
2
votes
1answer
72 views

What are some complexity classes for neural networks?

Turing machines and neural networks are equivalent in their expressive powers, but as models of computation they are different. Turing machines come pre-configured with their transition functions ...
1
vote
0answers
16 views

Why do we need to change the (weight decay) regularization parameter when changing the number of inputs that neural network is being trained with?

I am currently working my way through Michael Nielsen's ebook Neural Networks and Deep Learning and I am reading about overfitting and (L2) regularization. In this subsection, the process of L2 (a.k.a ...
2
votes
0answers
24 views

Simple back propagation example

Sorry if this is too simplistic of a question, but over the last couple of months I have been working through the course mathematical foundations of machine learning at my college. I think I am really ...
1
vote
1answer
22 views

Learning a specific functional form with machine learning

Suppose I have only three independent features (x, y, z) as the input to some machine learning routine (e.g. neural network). From some domain knowledge, I know ...
0
votes
1answer
10 views

Intersection of two generative neural networks

I have an input x (a word), and I have a neural network Y such that Y(x), which is an image always satisfies F(x), and another neural network Z such that Z(x), which is also an image, always satisfies ...
0
votes
0answers
24 views

Artificial Neuron: question on definition and inhibited signals

This is an artificial neuron of a NOT function from Rojas's ML book. I have a question on it's behavior. It's my understanding that the neuron produces a signal if it's inputs $x_1$ summed are $\geq ...
0
votes
1answer
34 views

Mcculloch and Pitts Neurons: Is there errata in this article?

The paper A LOGICAL CALCULUS OF THE IDEAS IMMANENT IN NERVOUS ACTIVITY has the following description of neurons, which I question if there is errata or perhaps I need more information to understand ...
0
votes
1answer
64 views

Approximating Deep Neural Networks (DNNs) with Binarized Neural Networks (BNNs)

I am working currently as a research intern on Binarized Neural Networks where the weights and the activations of the network are binary. The architecture of this type of networks makes them memory ...
1
vote
2answers
74 views

Can neural network process randomness?

So the question is : Is it theoretically possible to feed a neural network with some random values to expect an output since randomness is a lack of knowledge in most case. For this question, I've ...
2
votes
2answers
99 views

Being stuck and frustrated with my masters project

I'm doing a masters in CS that requires me to implement from scratch most of the neural network models and because python libraries aren't applicable to what i want. The problem is that i don't feel ...
2
votes
1answer
53 views

Energy consumption off CNN models

I want to calculate/estimate the energy consumption for the different convolutional neural networks. Is there any possibility to measure the energy consumed by AlexNet for example with a tool or with ...
0
votes
1answer
53 views

Understanding how convolution of images work in cnn

I'd like to understand convolutional neural network. Consider this quote from Stanford CS notes on Convolutional Neural Networks for Visual Recognition (CNNs/ConvNets): Example Architecture: ...
0
votes
0answers
7 views

Learning rule of hopfield network with sigmoidal activation function

I am going to build a hopfield network with continuous-valued units. In this article it is said that hopfield network units can have sigmoidal activation function (soft-limiting). If units can have ...
0
votes
1answer
25 views

Using a point as an input for a perceptron

Can a point be used as the input for a perceptron/neural net? The relationship between the two numbers that make up a 2D point does not affect the output, but does this not matter when the ...
0
votes
0answers
20 views

Training a model from self-play

Say I'd like to build a Go AI. The Go AI takes in the board state and then predicts who's more likely to win from that state. When I want to make a move, I just test every next board state I could ...
1
vote
1answer
39 views

How does the forget layer of an LSTM work?

Can someone explain the mathematical intuition behind the forget layer of an LSTM? So as far as I understand it, the cell state is essentially long term memory embedding (correct me if I'm wrong), ...
1
vote
1answer
40 views

Text detection in computer vision

I'm curious about the way text recognition works in machine learning(or more generally, the question of object vs not object) in computer vision. How are systems trained when the not-object data set ...
1
vote
1answer
56 views

Weighted Average of Neural Networks with Cross Entropy Cost Function

What is the best way to create an ensemble of neural networks utilising weighted averaging when these networks were trained to minimise cross entropy error function? The literature I found (e.g. [1]) ...
2
votes
1answer
58 views

Weighted Average of Multi-Output Neural Networks

[1] discusses how to construct an ensemble of neural networks by giving each network a certain weight $\alpha_i$: \begin{equation} f_\mathrm{GEM}(\boldsymbol{x}) = \sum_{i=1}^N \alpha_i f_i(\...
2
votes
1answer
35 views

Why do we need to take the derivative of the activation function in backwards propagation?

I was reading this article here: https://towardsdatascience.com/how-does-back-propagation-in-artificial-neural-networks-work-c7cad873ea7. When he gets to the part where he calculates the loss at ...
0
votes
0answers
45 views

Are neural network latent representations fitting a Gaussian distribution?

Neural network latent (or pre-activation) representations are the weighted sums of inputs to neurons in hidden layers before applying an activation function. The vector of representations of neurons ...
0
votes
0answers
32 views

Cross entropy minimization

I just read about Cross-entropy cost function for my work. and I see a notation that said "the minimization of cross entropy of some data is equal to maximization of their log likelihood." How can I ...
0
votes
1answer
50 views

How does image reconstruction take place in neural network?

I am reading through and thinking about how neural network works and have been reading about convolutional neural networks (CNN). I am particularly interested in image filtering (or enhancing) using ...
0
votes
1answer
75 views

Explain this Neural Network used for Knowledge Graph Embedding

I am interested in better understanding the neural network used in the paper LogicENN A Neural Based Knowledge Graphs Embedding Model with Logical Rules. To my knowledge this is the most advanced and ...

1
2 3 4 5
9