Questions tagged [neural-networks]

Network structure inspired by simplified models of biological neurons (brain cells). Neural networks are trained to "learn" by supervised and unsupervised techniques, and can be used to solve optimization problems, approximation problems, classify patterns, and combinations thereof.

Filter by
Sorted by
Tagged with
0
votes
0answers
38 views

How to calculate the number of weights for a CNN?

Can somebody help me with the formula needed to calculate the number of weights for a CNN, using the following sample question as the basis for it? Suppose we have a convolutional neural network with ...
0
votes
0answers
5 views

Which model to apply on such panel data with so may rows but for each unique id rows are 6-8 rows per unique id?

I am new to such panel data where I have multiple observation for same ID in different Quarter and I am not sure what kind of machine learning algorithm I can apply. I have data from Q1-18 till Q4-...
0
votes
0answers
6 views

How is the dimensionality of the volume in ConvNets determined for a general case?

In ConvNets, I understand how the dimensionality of a flat image changes after convolving it with a single filter. For example, if you convolve a P x P x 1 image (assume no padding) with a filter with ...
0
votes
0answers
11 views

Calculating error in the Input to a convolutional layer

Say I have a incredibly simple input to a convolutional layer: [In1] <----- 1*1 input matrix I have two filters applied this input:[F1] and [F2] They give the results: [F1 * In1] and [F2 * In1] ...
0
votes
0answers
23 views

Looking for references for real-world scenarios of data-poisoning attack on labels while doing supervised learning

Consider the following mathematical model of training a neural net : Suppose $f_{w} : \mathbb{R}^n \rightarrow \mathbb{R}$ is a neural net whose weights are $w$. Suppose during the training the ...
1
vote
0answers
11 views

What is spatial information in neural network? [duplicate]

The architecture of the CNN model contains several convolutional layers, non-linear activations, batch normalization, and pooling layers. The initial layers learn the low-level concepts such as edges ...
0
votes
0answers
16 views

Kohonen, 1 dimension, SOM, puzzle

We consider training one-dimensional open map of Kohonen with neurons in one-dimensional input space. We assume it is completed the phase of the device and the weights wi, i = 1,2, ..., N, are placed ...
1
vote
1answer
20 views

Is transfer learning applied on only similar datasets only?

I am trying to make a CNN model on different brands of logos . Firstly , I wrote a CNN from scratch and trained it on which I got 70% accuracy, I have total 40 classes and each class has 100 images . ...
1
vote
1answer
46 views

What is the Number of epochs with no improvement after which training will be stopped.?

I am trying to make a Convolutional neural network. Training the images of different brands of Logos. Have 100 images per class and there are 40 classes. I have trained the model now want to check ...
0
votes
1answer
33 views

How does CNN deal with rotation invariant pictures?

I am trying to make a CNN model . Training the image . Want to know that When we apply kernel on image and take out the features of images. That features are rotation invariant or we have to apply ...
2
votes
0answers
38 views

Efficient top eigenvalue computation for Hessian of neural networks

I train a neural network - one of the Resnet variations ($\approx 10^7$ parameters) on the CIFAR-10 dataset - and after each epoch, I would like to find the smallest/largest eigenvalues of its Hessian....
0
votes
0answers
12 views

How do I choose the right model for production?

I am trying to build a classification algorithm having 28 classes. These classes consists of Logo of companies like adidas , Nike etc. I have very low dataset below than 100 images and greater than 70 ...
1
vote
1answer
28 views

Which pretrained model will be best for my dataset?

I am trying to build a classification algorithm having 28 classes. These classes consists of Logo of companies like adidas , Nike etc. I have very low dataset below than 100 images and greater than 70 ...
0
votes
0answers
17 views

Testing correctness of feedforward neural network implementation

So I'm currently reading about neuroevolution (NEAT, WANN) and trying to make my own implementation just as an exercise. Now I want to test if my feedforward implementation gives the expected output. ...
1
vote
1answer
60 views

Binary cross Entropy derivative?

I am just learning backpropagation algorithm for NN and currently I am stuck with the right derivative of Binary Cross Entropy as loss function. Here it is: ...
0
votes
1answer
64 views

If i use other peoples code for my final year project does that come under Plagiarism?

I want to make a facial recognition project but my coding ability is very limited so I am going to be relying on other peoples code but if i use their code even if i change the variables and make ...
2
votes
0answers
51 views

any hope to solve np hard problem using deep learning? [duplicate]

I know some basic machine learning and deep learning. Now a days deep learning solve many types of problem. I working working optimization problem like np, np hard problem. Is there any hope to solve ...
0
votes
0answers
10 views

New Neural Network Pruning Algorithm

I developed an algorithm that can prune down neural networks while retaining most of the accuracy. For example, it can take a trained neural network (relu activation functions) with a hidden layer of ...
0
votes
0answers
17 views

Designing a neural network with LSTM and feedforward NN combination

Currently, I'm designing a neural network that works with reinforcement learning. In summary, the agent takes in information about itself and nearby agents and, in conjunction with global world ...
0
votes
0answers
27 views

Stuck on Ejaz and Islam's paper about Masked Face Recognition

I'm relatively new in machine learning and I am trying to put together my undergraduate thesis on masked face recognition. I've read Ejaz and Islam's paper (available at https://www.researchgate.net/...
1
vote
1answer
18 views

What does non-linearity mean in Neural Networks? Why is it necessary?

ReLU units are said to be necessary in CNNs to introduce non-linearity which convolution does not involve. This is needed, because many real-world forms of data are non-linear. What does non-linearity ...
3
votes
1answer
25 views

CNNs: Why do the filters identify more complex features as the model goes on?

I have a feeling that it's related to how different output layers are stacked up against one another? However, there's a missing link in that argument that I can't completely get to.
0
votes
0answers
26 views

Papers on depth separations in neural networks

I am new to the domain of Machine learning. I have been asked to present a paper related to the mathematics behind the depth separations in Neural Networks (by Itay Safran, Ronen Eldan and Ohad Shamir)...
0
votes
2answers
35 views

Conv Net dimensions misunderstanding?

I was asked this question: Given an image with shape [1,28,28], what will be the shape of the output of a convolution layer with 10 5x5 kernels (filters) without padding? Now, are the shape dimensions ...
0
votes
0answers
27 views

Good Articles/book on neural networks

I search good articles or books on neural network. unfortunately i don't a lot of time and only now i started to research this field. so i actually search a good source that will give the fundamental (...
0
votes
1answer
30 views

What is the time complexity of the SGVB estimator?

In the context of variational autoencoders, we want to maximize the evidence lower bound and this is typically done using Stochastic Gradient Variational Bayes (SGVB). I was curious if there is any ...
1
vote
1answer
20 views

which neural network is good for predecting the value of strings

I have a dataset that contains some strings. A numeric value is assigned to each string. I want to develop a machine learning (deep learning) model to get a string and predict its value. What neural ...
1
vote
1answer
35 views

How to estimate the computational cost in a neural network?

Given a neural network(assuming no regularisation/dropout), I want to determine the computational cost of doing a forward and a backward pass of a datapoint. I want the measure to be of independent of ...
1
vote
1answer
33 views

Expressivity of neural networks, how much information can be stored

I want to know whether a given neural network (with a finite number of nodes) is able to store all injective maps f: D -> C, where D has cardinality k and C has cardinality N (so the number of maps ...
1
vote
7answers
3k views

Why can't we say that a Neural Network is a NP problem solver?

From this video lecture from MIT https://youtu.be/moPtwq_cVH8?t=1229 there is mention how NP complexity works with finding a "lucky" algorithm and luck can never be accounted for. The ...
0
votes
1answer
16 views

Signal translation with Seq2Seq model

I'm currently doing some research on signal processing and I got a dataset which includes the signal in itself and its "translation". So I want to use a Many-to-Many RNN to translate the ...
2
votes
1answer
34 views

object image segmentation

I have 2 different datasets with similar objects, one where each object is 50 pixels wide and the other where they are 150 pixels. Each photo is 512x512 for both datasets. These two datasets have the ...
0
votes
1answer
28 views

What Happens if I swap the forget gate and update gate in LSTM model?

Consider the following eqautions used in LSTM ( taken from Andrew ng's course on Sequential model) In an LSTM model, LSTM Cell has three inputs at any time step t Input($X_t , a^{(t-1)}, C^{(t-1)})$, ...
1
vote
1answer
24 views

How can one measure the time dependency of an RNN?

Most of the discussion about RNN and LSTM alludes to the varying ability of different RNNs to capture "long term dependency". However, most demonstrations use generated text to show the ...
1
vote
1answer
36 views

Why isn't there just one “keystone” activation function in Neural Networks?

This article says the following: Deciding between the sigmoid or tanh will depend on your requirement of gradient strength. I have seen (so far in my learning) 7 activation functions/curves. Each ...
0
votes
2answers
51 views

Why don't they use all kinds of non-linear functions in Neural Network Activation Functions? [duplicate]

Pardon my ignorance, but after just learning about Sigmoid and Tanh activation functions (and a few others), I am wondering why they choose functions that always go up and to the right? Why not use ...
1
vote
1answer
42 views

How was the σ function chosen to extend the perceptron?

I am just reading about perceptrons in more depth, and now onto Sigmoid Neurons. Some quotes: A small change in the weights or bias of any single perceptron in the network can sometimes cause the ...
1
vote
0answers
38 views

In a machine learning system, why use differentially private SGD if our input data is already perturbed by a DP mechanism?

I'm trying to implement my own version of a deep neural network with differential privacy to preserve the privacy of the parties involved in the training dataset. I'm using the method by Abadi et al. ...
0
votes
0answers
24 views

Using the features embedding of the output from a transformers to represent probabilities of categorical data

I was considering using a transformer, on input data which can be represented as an embedding, so I can use the attention mechanism in the transformer architecture. As my data is of variable input and ...
1
vote
0answers
33 views

How can Kneser-Ney Smoothing be integrated into a neural language model?

I found a paper titled Multimodal representation: Kneser-Ney Smoothing/Skip-Gram based neural language model. I am curious about how the Kneser-Ney Smoothing technique can be integrated into a feed-...
1
vote
2answers
47 views

Why doesn't relu cause back propegation to get stuck

Say you have a neural net that is being trained using back propagation and you are using relu activation. Say the input to a node is a weighted sum of the previous layer with a bias term and say for a ...
4
votes
1answer
24 views

Are non-ordered (Slabbed) Neural Networks in wide modern use and what does Fiesler (1994) mean by Clamping Function and Ontogenic function?

In Fiesler (1994) Neural Network Classification and Formalization, he talks a lot about a more general version of neural networks, one that is not ordered into layers, but rather the network is called ...
0
votes
0answers
44 views

Delta rule for binary step function

I have a question of understanding about the Delta Rule: $\Delta w_i = (y - \hat{y}) \times x_i$ Why does $x$ have to be multiplied again after the difference? If the input is $0$, the product of $w$...
2
votes
1answer
54 views

Can you apply neural networks to design algorithms?

I’m kind of a newbie to neural networks (and CS in general) but I was wondering if there are any methods to apply them in such a way with the aim of producing algorithms that solve difficult math ...
2
votes
2answers
49 views

Zero-Sum Games and Halting Problem

Wikipedia states on the page of the halting problem, "For any program f that might determine if programs halt, a "pathological" program g called with an input can pass its own source and its input to ...
0
votes
1answer
44 views

Performing semantic segmentation on images on word documents

At the moment, I'm making a comparative analysis between two different neural networks for a class. The problem they are both trying to solve is detecting tables on documents. Below is an example of ...
5
votes
5answers
306 views

fast and stable x * tanh(log1pexp(x)) computation

$$f(x) = x \tanh(\log(1 + e^x))$$ The function (mish activation) can be easily implemented using a stable log1pexp without any significant loss of precision. Unfortunately, this is computationally ...
0
votes
0answers
35 views

How to find clusters of a set of points in n-dimensional space that are furthest from unwanted points

I have a list of 25 points and their coordinates in a 512-dimensional space. I have 8 target points and 17 points I need to avoid (the 17 points to avoid also have differences in priority of how ...
2
votes
1answer
77 views

What are some complexity classes for neural networks?

Turing machines and neural networks are equivalent in their expressive powers, but as models of computation they are different. Turing machines come pre-configured with their transition functions ...
1
vote
0answers
16 views

Why do we need to change the (weight decay) regularization parameter when changing the number of inputs that neural network is being trained with?

I am currently working my way through Michael Nielsen's ebook Neural Networks and Deep Learning and I am reading about overfitting and (L2) regularization. In this subsection, the process of L2 (a.k.a ...

1
2 3 4 5
9