Questions tagged [neural-networks]

Network structure inspired by simplified models of biological neurons (brain cells). Neural networks are trained to "learn" by supervised and unsupervised techniques, and can be used to solve optimization problems, approximation problems, classify patterns, and combinations thereof.

Filter by
Sorted by
Tagged with
1
vote
1answer
17 views

Which pretrained model will be best for my dataset?

I am trying to build a classification algorithm having 28 classes. These classes consists of Logo of companies like adidas , Nike etc. I have very low dataset below than 100 images and greater than 70 ...
0
votes
0answers
11 views

Testing correctness of feedforward neural network implementation

So I'm currently reading about neuroevolution (NEAT, WANN) and trying to make my own implementation just as an exercise. Now I want to test if my feedforward implementation gives the expected output. ...
1
vote
1answer
12 views

Binary cross Entropy derivative?

I am just learning backpropagation algorithm for NN and currently I am stuck with the right derivative of Binary Cross Entropy as loss function. Here it is: ...
0
votes
1answer
57 views

If i use other peoples code for my final year project does that come under Plagiarism?

I want to make a facial recognition project but my coding ability is very limited so I am going to be relying on other peoples code but if i use their code even if i change the variables and make ...
2
votes
0answers
47 views

any hope to solve np hard problem using deep learning? [duplicate]

I know some basic machine learning and deep learning. Now a days deep learning solve many types of problem. I working working optimization problem like np, np hard problem. Is there any hope to solve ...
0
votes
0answers
10 views

New Neural Network Pruning Algorithm

I developed an algorithm that can prune down neural networks while retaining most of the accuracy. For example, it can take a trained neural network (relu activation functions) with a hidden layer of ...
0
votes
0answers
14 views

Designing a neural network with LSTM and feedforward NN combination

Currently, I'm designing a neural network that works with reinforcement learning. In summary, the agent takes in information about itself and nearby agents and, in conjunction with global world ...
0
votes
0answers
21 views

Stuck on Ejaz and Islam's paper about Masked Face Recognition

I'm relatively new in machine learning and I am trying to put together my undergraduate thesis on masked face recognition. I've read Ejaz and Islam's paper (available at https://www.researchgate.net/...
1
vote
1answer
14 views

What does non-linearity mean in Neural Networks? Why is it necessary?

ReLU units are said to be necessary in CNNs to introduce non-linearity which convolution does not involve. This is needed, because many real-world forms of data are non-linear. What does non-linearity ...
3
votes
1answer
25 views

CNNs: Why do the filters identify more complex features as the model goes on?

I have a feeling that it's related to how different output layers are stacked up against one another? However, there's a missing link in that argument that I can't completely get to.
0
votes
0answers
25 views

Papers on depth separations in neural networks

I am new to the domain of Machine learning. I have been asked to present a paper related to the mathematics behind the depth separations in Neural Networks (by Itay Safran, Ronen Eldan and Ohad Shamir)...
0
votes
2answers
34 views

Conv Net dimensions misunderstanding?

I was asked this question: Given an image with shape [1,28,28], what will be the shape of the output of a convolution layer with 10 5x5 kernels (filters) without padding? Now, are the shape dimensions ...
0
votes
0answers
26 views

Good Articles/book on neural networks

I search good articles or books on neural network. unfortunately i don't a lot of time and only now i started to research this field. so i actually search a good source that will give the fundamental (...
0
votes
1answer
24 views

What is the time complexity of the SGVB estimator?

In the context of variational autoencoders, we want to maximize the evidence lower bound and this is typically done using Stochastic Gradient Variational Bayes (SGVB). I was curious if there is any ...
1
vote
1answer
19 views

which neural network is good for predecting the value of strings

I have a dataset that contains some strings. A numeric value is assigned to each string. I want to develop a machine learning (deep learning) model to get a string and predict its value. What neural ...
1
vote
1answer
32 views

How to estimate the computational cost in a neural network?

Given a neural network(assuming no regularisation/dropout), I want to determine the computational cost of doing a forward and a backward pass of a datapoint. I want the measure to be of independent of ...
1
vote
1answer
31 views

Expressivity of neural networks, how much information can be stored

I want to know whether a given neural network (with a finite number of nodes) is able to store all injective maps f: D -> C, where D has cardinality k and C has cardinality N (so the number of maps ...
1
vote
7answers
3k views

Why can't we say that a Neural Network is a NP problem solver?

From this video lecture from MIT https://youtu.be/moPtwq_cVH8?t=1229 there is mention how NP complexity works with finding a "lucky" algorithm and luck can never be accounted for. The ...
0
votes
1answer
12 views

Signal translation with Seq2Seq model

I'm currently doing some research on signal processing and I got a dataset which includes the signal in itself and its "translation". So I want to use a Many-to-Many RNN to translate the ...
2
votes
1answer
33 views

object image segmentation

I have 2 different datasets with similar objects, one where each object is 50 pixels wide and the other where they are 150 pixels. Each photo is 512x512 for both datasets. These two datasets have the ...
0
votes
1answer
23 views

What Happens if I swap the forget gate and update gate in LSTM model?

Consider the following eqautions used in LSTM ( taken from Andrew ng's course on Sequential model) In an LSTM model, LSTM Cell has three inputs at any time step t Input($X_t , a^{(t-1)}, C^{(t-1)})$, ...
1
vote
1answer
24 views

How can one measure the time dependency of an RNN?

Most of the discussion about RNN and LSTM alludes to the varying ability of different RNNs to capture "long term dependency". However, most demonstrations use generated text to show the ...
1
vote
1answer
36 views

Why isn't there just one “keystone” activation function in Neural Networks?

This article says the following: Deciding between the sigmoid or tanh will depend on your requirement of gradient strength. I have seen (so far in my learning) 7 activation functions/curves. Each ...
0
votes
2answers
51 views

Why don't they use all kinds of non-linear functions in Neural Network Activation Functions? [duplicate]

Pardon my ignorance, but after just learning about Sigmoid and Tanh activation functions (and a few others), I am wondering why they choose functions that always go up and to the right? Why not use ...
1
vote
1answer
40 views

How was the σ function chosen to extend the perceptron?

I am just reading about perceptrons in more depth, and now onto Sigmoid Neurons. Some quotes: A small change in the weights or bias of any single perceptron in the network can sometimes cause the ...
1
vote
0answers
37 views

In a machine learning system, why use differentially private SGD if our input data is already perturbed by a DP mechanism?

I'm trying to implement my own version of a deep neural network with differential privacy to preserve the privacy of the parties involved in the training dataset. I'm using the method by Abadi et al. ...
0
votes
0answers
21 views

Using the features embedding of the output from a transformers to represent probabilities of categorical data

I was considering using a transformer, on input data which can be represented as an embedding, so I can use the attention mechanism in the transformer architecture. As my data is of variable input and ...
1
vote
0answers
33 views

How can Kneser-Ney Smoothing be integrated into a neural language model?

I found a paper titled Multimodal representation: Kneser-Ney Smoothing/Skip-Gram based neural language model. I am curious about how the Kneser-Ney Smoothing technique can be integrated into a feed-...
1
vote
2answers
44 views

Why doesn't relu cause back propegation to get stuck

Say you have a neural net that is being trained using back propagation and you are using relu activation. Say the input to a node is a weighted sum of the previous layer with a bias term and say for a ...
4
votes
1answer
24 views

Are non-ordered (Slabbed) Neural Networks in wide modern use and what does Fiesler (1994) mean by Clamping Function and Ontogenic function?

In Fiesler (1994) Neural Network Classification and Formalization, he talks a lot about a more general version of neural networks, one that is not ordered into layers, but rather the network is called ...
0
votes
0answers
41 views

Delta rule for binary step function

I have a question of understanding about the Delta Rule: $\Delta w_i = (y - \hat{y}) \times x_i$ Why does $x$ have to be multiplied again after the difference? If the input is $0$, the product of $w$...
1
vote
1answer
50 views

Can you apply neural networks to design algorithms?

I’m kind of a newbie to neural networks (and CS in general) but I was wondering if there are any methods to apply them in such a way with the aim of producing algorithms that solve difficult math ...
2
votes
2answers
49 views

Zero-Sum Games and Halting Problem

Wikipedia states on the page of the halting problem, "For any program f that might determine if programs halt, a "pathological" program g called with an input can pass its own source and its input to ...
0
votes
1answer
40 views

Performing semantic segmentation on images on word documents

At the moment, I'm making a comparative analysis between two different neural networks for a class. The problem they are both trying to solve is detecting tables on documents. Below is an example of ...
5
votes
5answers
297 views

fast and stable x * tanh(log1pexp(x)) computation

$$f(x) = x \tanh(\log(1 + e^x))$$ The function (mish activation) can be easily implemented using a stable log1pexp without any significant loss of precision. Unfortunately, this is computationally ...
0
votes
0answers
34 views

How to find clusters of a set of points in n-dimensional space that are furthest from unwanted points

I have a list of 25 points and their coordinates in a 512-dimensional space. I have 8 target points and 17 points I need to avoid (the 17 points to avoid also have differences in priority of how ...
2
votes
1answer
74 views

What are some complexity classes for neural networks?

Turing machines and neural networks are equivalent in their expressive powers, but as models of computation they are different. Turing machines come pre-configured with their transition functions ...
1
vote
0answers
16 views

Why do we need to change the (weight decay) regularization parameter when changing the number of inputs that neural network is being trained with?

I am currently working my way through Michael Nielsen's ebook Neural Networks and Deep Learning and I am reading about overfitting and (L2) regularization. In this subsection, the process of L2 (a.k.a ...
2
votes
0answers
24 views

Simple back propagation example

Sorry if this is too simplistic of a question, but over the last couple of months I have been working through the course mathematical foundations of machine learning at my college. I think I am really ...
1
vote
1answer
23 views

Learning a specific functional form with machine learning

Suppose I have only three independent features (x, y, z) as the input to some machine learning routine (e.g. neural network). From some domain knowledge, I know ...
0
votes
1answer
10 views

Intersection of two generative neural networks

I have an input x (a word), and I have a neural network Y such that Y(x), which is an image always satisfies F(x), and another neural network Z such that Z(x), which is also an image, always satisfies ...
0
votes
0answers
25 views

Artificial Neuron: question on definition and inhibited signals

This is an artificial neuron of a NOT function from Rojas's ML book. I have a question on it's behavior. It's my understanding that the neuron produces a signal if it's inputs $x_1$ summed are $\geq ...
0
votes
1answer
41 views

Mcculloch and Pitts Neurons: Is there errata in this article?

The paper A LOGICAL CALCULUS OF THE IDEAS IMMANENT IN NERVOUS ACTIVITY has the following description of neurons, which I question if there is errata or perhaps I need more information to understand ...
0
votes
1answer
68 views

Approximating Deep Neural Networks (DNNs) with Binarized Neural Networks (BNNs)

I am working currently as a research intern on Binarized Neural Networks where the weights and the activations of the network are binary. The architecture of this type of networks makes them memory ...
1
vote
2answers
77 views

Can neural network process randomness?

So the question is : Is it theoretically possible to feed a neural network with some random values to expect an output since randomness is a lack of knowledge in most case. For this question, I've ...
2
votes
2answers
102 views

Being stuck and frustrated with my masters project

I'm doing a masters in CS that requires me to implement from scratch most of the neural network models and because python libraries aren't applicable to what i want. The problem is that i don't feel ...
2
votes
1answer
54 views

Energy consumption off CNN models

I want to calculate/estimate the energy consumption for the different convolutional neural networks. Is there any possibility to measure the energy consumed by AlexNet for example with a tool or with ...
0
votes
1answer
54 views

Understanding how convolution of images work in cnn

I'd like to understand convolutional neural network. Consider this quote from Stanford CS notes on Convolutional Neural Networks for Visual Recognition (CNNs/ConvNets): Example Architecture: ...
0
votes
0answers
7 views

Learning rule of hopfield network with sigmoidal activation function

I am going to build a hopfield network with continuous-valued units. In this article it is said that hopfield network units can have sigmoidal activation function (soft-limiting). If units can have ...
0
votes
1answer
25 views

Using a point as an input for a perceptron

Can a point be used as the input for a perceptron/neural net? The relationship between the two numbers that make up a 2D point does not affect the output, but does this not matter when the ...

1
2 3 4 5
9