Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [neural-networks]

Network structure inspired by simplified models of biological neurons (brain cells). Neural networks are trained to "learn" by supervised and unsupervised techniques, and can be used to solve optimization problems, approximation problems, classify patterns, and combinations thereof.

0
votes
0answers
5 views

What is the practical limit to how many object classes you can detect with Faster RCNN?

I am trying to follow this tutorial where the Faster-RCNN-Inception-V2-COCO model from TensorFlow's model zoo is used to detect playing cards. I was wondering what is the practical limit to the number ...
0
votes
0answers
19 views

Can a Neural Network be Created to Identify the Composer of a Single Song with Python and Tensorflow? [on hold]

I would like to propose a theoretical question. Recently, I have been attempting to complete an experiment involving a neural network. Here are the parameters for my experiment and the specifications ...
2
votes
0answers
19 views

Multilayer Perceptrons for solving variational problems

Can we use a multilayer perceptron to solve variational problems? By variational problem I mean something we might encounter in the calculus of variations, for example the geodesic problem: given two ...
0
votes
0answers
10 views

What difference does tanh and relu activation functions makes when used in the hidden layers of an autoencoder?

Please explain that what difference these activation functions can make to the output of an autoencoder.
0
votes
0answers
14 views

How should I represent an unordered set of dense vectors as input to a neural network?

My dataset consists of 10000 (x,y) pairs where x is an unordered set of 27-dimensional vectors with a total of 8 elements, and y is a different unordered set of 27-dimensional vectors with a total of ...
0
votes
0answers
6 views

About cross-entropy loss

Is there a classification scenario with cross-entropy loss such that the loss as a function of the predictor/neural net's parameters is a function s.t it satisfies the properties of (a) having a ...
1
vote
1answer
31 views

Linear Regression using a Neural Network

I am trying to create a regression model using a Neural Network. I am currently learning how to work with neural networks (deeplearning.ai) and so the model is not implemented using any existing ...
1
vote
1answer
70 views

(OCR ) How to Recognise Handwritten fractional numbers using Neural networks

I want to be able to recognise handwritten math numbers using images of the numbers , i was able to do create a ANN model for recognising simple decimal numbers , but i have no idea on how to ...
1
vote
0answers
31 views

Why is the method of im2col with GEMM is more efficient than the method of direction implementation with SIMD in CNN

The convolutional layers are most computationally intense parts of Convolutional neural networks (CNNs).Currently the common approach to impement convolutional layers is to expand the image into a ...
2
votes
0answers
18 views

When is currying more efficient in deep neural nets?

I'm reading a blog post on deep Q-learning, and it contrasts traditional lookup-table-based Q-learning with deep Q-learning: What I wonder about in this picture is: Why does the deep NN not also ...
2
votes
1answer
46 views

Analog circuits for neural networks?

Neural networks in machine learning are inherently a continuous model of computation. Yet we use digital logic circuits with floating point numbers to "emulate" this continuity. I am wondering: is ...
0
votes
0answers
12 views

Input Features to ANN

I want to classify different flower species. It's required that features computed from GLCM and color moments should be used as input features to neural network. I have some doubt whether it is ...
0
votes
0answers
12 views

Is the optimization of the Gaussian VAE well-posed?

In a Variational Autoencoder (VAE), given some data $x$ and latent variables $t$ with prior distribution $p(t) = \mathcal{N}(t | 0, I)$, the encoder aims to learn a distribution $q_{\phi}(t)$ that ...
1
vote
0answers
30 views

Proof of perceptron convergence theorem for ZERO threshold?

The generalized perceptron convergence theorem is for a defined threshold T. When you do the maths it all comes to an upper bound and a lower bound. The lower bound looks like this! Therefore ...
1
vote
1answer
30 views

Can we use Convolutional Neural Network for dataset containing numeric data? [closed]

I am working on a project of " Fraud detection using deep learning" . For that I have a dataset containing some numerical attributes . Now the task is to use CNN for the above purpose. Please guide me ...
2
votes
1answer
36 views

Is Artificial General Intelligence possible with our current machine learning models? [closed]

In other words, is artificial human level intelligence not possible yet just because of limitations in processing power and amount of data required to train the models? Or we don't have the knowledge ...
0
votes
0answers
14 views

how many hidden layers in an n-layer neural network

Simple terminology question: (should be easy to answer) How many hidden layers does an n-layer neural network have? I believe the answer is n-1. For example a single layer perceptron has no hidden ...
2
votes
1answer
85 views

How to represent symbolic knowledge using real numbers - theory about neural networks and natural/analog computing?

One can define the semantics of one definite word using the references to real world entities, relationships with the other words and other concepts and represent all this knowledge about this one ...
1
vote
1answer
31 views

Proving Monotonicity of Softmax Layer

In the book here: http://neuralnetworksanddeeplearning.com/chap3.html If you scroll down to Exercise 2 in the Softmax Section, it says Show that $\partial a^L_{j}/\partial z^L_{k}$ is positive if $...
4
votes
0answers
18 views

How to represent sentences with their dependency parses as input to an RNN?

I am working on a task embedding sentences into a lower-dimensional space according to style, both grammatical and lexical. As such, I want to have as input the linear ordering of tokens in each ...
3
votes
1answer
34 views

Weird behaviour of softmax derivative?

I have been implementing some neural networks in MATLAB and recently I noticed a weird thing while implementing softmax derivative: Setting the derivative to one, rather than using the actual ...
0
votes
1answer
25 views

How to calculate the weight between neurons in ANN?

I am currently learning Supervised ANN training using Backpropogation and I am stuck in this exercise. I calculated the δA using the equation at the bottom of the screenshot, however, I am unable to ...
0
votes
1answer
15 views

Correct cost function of multi classification problem using neural network?

I am going through machine learning course on coursera. While going through the section on neural networks I came across the cost function for multi - classification problem using neural networks ( ...
1
vote
1answer
43 views

Is deep learning appropriate to approximate dynamic programming problems?

I have a problem which can be completely solved using dynamic programming, but in a very intractable way (On^4, where n is around 1000). I won't get into the details of the problem since it's a bit ...
0
votes
0answers
17 views

Implementing recurrent neural networks - matrix dimensions

This may be potentially better suited as a Linear Algebra question. I'm trying to implement the forward pass update rules for an LSTM unit. Following this definition: The problem is it is unclear ...
1
vote
0answers
78 views

How can node2vec help find similar “roles” within a graph (nodes whose connections have similar structure within the graph)?

I have a question on the node2vec algorithm described in this paper. Node2vec is a deep learning algorithm that word2vec to graphs to learn embeddings. The authors claim that it can help find nodes ...
0
votes
1answer
21 views

MLP and backpropagation limitations?

I have heard a colleague of mine giving the following statements to a student, but I am not quite sure if he is right. The statements were about Multi Layer Perceptron and the Backpropagation ...
1
vote
1answer
36 views

Parameter sharing / weight constraints in Neural Networks

I would like to train a neural network whose parameters (alternatively, weights) are subject to linear constraints such as $w_{i,j} = w_{i',j'}$, where $w_{i,j}$ denotes the weight from input node $...
2
votes
1answer
62 views

Is there a universal learning rate for NeuralNetworks?

I'm currently creating a NeuralNetwork with backpropagation/gradient descent. There is this hyperparameter introduced called "learning rate" (η). Which has to be chosen to guarantee not overshooting ...
2
votes
1answer
325 views

How can I optimize 3 variables in order to maximize the end product?

I am in the process of making a cryptocurrency trading bot. Currently, I am doing backtesting over a period of 7 months in which I provide a portion of historical data as if it were in real-life. By ...
0
votes
1answer
25 views

Formal definition of loss surface of multi-layered networks

Let $\mathcal{L}$ be a loss function associated with a multi-layered neural network. So it seems almost everyone in AI/ML community is interested in the Hessian $H=\partial^2 \mathcal{L}$ of $\...
0
votes
0answers
34 views

Hessian in reinforcement learning

The Hessian of multi-layered network exhibits known behaviour at critical points as shown in [1]. The tools of random matrix theory allow [2] to deduce the asymptotic distribution of the eigenvalues ...
0
votes
0answers
16 views

Reward surface in reinforcement learning

There is a remarkable paper [1] which explores geometry of neural network. I believe this information is quite helpful in plethora of optimization methods. In reinforcement learning, the optimization ...
0
votes
0answers
22 views

Adding and removing output layer units of a neural network

I'm fairly new to deep learning, so if terminology makes no sense, please let me know so I can clarify what I mean. We're working with a neural network for applying classes to inputs. That is, each ...
3
votes
1answer
68 views

How do you protect an AI from a human doing “illogical” moves?

Using a monte carlo approach and evalutation function. Some moves will deemed to be more advantageous than others. As a computer plays itself, it will generally go for the best moves possible. And ...
2
votes
1answer
67 views

Capsule networks for classification with limited data

Capsule networks seem to match performance of convolutional neural networks on image classification tasks (more specifically on classification of handwritten digits in the MNIST dataset) 1. I have ...
1
vote
1answer
14 views

CNN/Neural Network: Can I still estimate 3 parameters if my input data has insufficient parameter labels?

I am trying to simplify a CNN model. Currently, I need to train 3 different models (with the same architecture) to estimate each parameter. I am just wondering if there is a way to just train one ...
2
votes
0answers
37 views

Approximate dot product between neural network output layer's parameter vector and input activations with winner-take-all hashing

In the paper Deep Networks with Large Output Spaces, Vijayanarasimhan et al. describe their approach to approximating the dot product between a neural network's output layer's parameter vector and ...
0
votes
1answer
56 views

textbook on the human brain for computer scientists/machine learning professionals?

The human brain as studied by neuroscientists, or neurobiologists, generally focus on what I would call implementation details (how specifically does information transport between synaptic channels, ...
0
votes
1answer
24 views

Matrix multiplication in recurrent neural networks

I was looking at a tutorial for recurrent neural networks in Python, and I have a question in regards to multiplying matrices of different sizes. Specifically, why does S[t] have 100 elements in it? ...
0
votes
0answers
24 views

Classify manifolds with neural networks

Can a neural network be used to find the genus of a 2-manifold given for instance as a CW complex?
0
votes
0answers
90 views

Reverse engineering a neural network

Suppose we have a neural network (such as google uses for instance) which detects an object in images, which could be a cat or car. Suppose that it is instead an alien artifact that we dont have ...
1
vote
1answer
32 views

How does pre-training help with semantic segmentation with U-net

I am working with a u-net, a variation of the more commonly known fully convolutional network for semantic segmentation. For training a u-net, I was given the suggestion that I should use a pre-...
1
vote
0answers
24 views

How would you go about creating a algorithm that should generate a shakespearean sonnet on any given theme

I need to create an algorithm that is going to create a shakespearean sonnet for a specific theme. This theme should be generated out of twitter tweets that have some hashtag. My current idea goes ...
1
vote
0answers
126 views

Backward mapping with bilinear sampler

I have some experiences with Convolutional Neural Networks before. I have a question regarding the Bilinear Sampler used in "Unsupervised Monocular Depth Estimation With Left-Right Consistency" (the ...
1
vote
0answers
20 views

Does a deep feedforward neural net approximate a (single or multivariate)polynomial?

Does composition of several linear transformation plus non-linear activation function in each layer and different layers (as they are in feedforward neural net) represent or approximate a ...
1
vote
2answers
72 views

Is there a difference between computational graphs and neural networks?

According to this post, "neural networks are a special form [of a computational graph]". I think that one can infer from that that all neural networks are computational graphs. My question then would ...
0
votes
0answers
16 views

Simple Memory Mechanism in a highly customized recurrent neural network

I am currently implementing the experiment described in that paper: "Emergence of grounded compositional language in Multi-Agent population" by Igor Mordatch and Pieter Abbeel, available here: https://...
0
votes
0answers
12 views

Deep neural networks and deep learning generate Markov chain of representation

When and why do deep neural networks and deep learning generate a Markov chain of representations of input patterns? How can be understood why the representations of the input patterns generated by ...
4
votes
2answers
298 views

What exactly representational bottleneck in InceptionV3 means?

I am trying to understand the concepts behind the InceptionNet V3 and got confused with the meaning of representational bottleneck. They said One should avoid bottlenecks with extreme compression. ...