Questions tagged [neural-networks]

Network structure inspired by simplified models of biological neurons (brain cells). Neural networks are trained to "learn" by supervised and unsupervised techniques, and can be used to solve optimization problems, approximation problems, classify patterns, and combinations thereof.

-2
votes
0answers
35 views

Does this increase the power of a neural network

If we use a stack along with a DFA (Finite Automata), that is, a PDA (Push down automata) at each node of a neural network, can we solve problems which cannot be solved by a normal neural network?
0
votes
0answers
5 views

Constructing a 3D virtual map from 2D Depth Maps and $x, y, z, \theta$ coordinates

Context: I intend on building a quadrotor which can generate a 3D virtual map of an area of which it captures photos with a 2D camera. I plan on first training a Convulational NN to return a depth ...
0
votes
0answers
9 views

Training of two 3x3 convolution layers vs training one 5x5 convolution layer

I'm not 100% sure this is the right stackexchange, please feel free to redirect me to another one. I know that two 3x3 convolution layers can be equivalent to one 5x5 convolution layer. I also know ...
1
vote
0answers
21 views

Solving analytic gradient of loss function for neural networks [closed]

Please note that I am talking in about theory rather than ''what someone would do in a real, practical situation''. Given a multi-layer Perceptron with at least 1 hidden layer, and sigmoid (or other ...
0
votes
0answers
13 views

Computing analytic gradients for NN cost function with 1 hidden layer

Given a simple artificial neural network with 1 hidden layer, I want to compute the analytic gradient, to gain a better understanding. Using a simple loss function L such as: $L=(1/N)\sum_{k=1}^N|| \...
0
votes
0answers
13 views

Can someone explain the difference between MCP neurons and Perceptrons?

I am getting confused with the literature. Is a perceptron simply a network of MCP neurons? From what I understand, in 1957 Rosenblatt developed the perceptron based on relaxed constraints from the ...
0
votes
0answers
28 views

Guided Backpropagation in Deep Neural Networks

I am reading about a method called guided backpropagation. https://ramprs.github.io/2017/01/21/Grad-CAM-Making-Off-the-Shelf-Deep-Models-Transparent-through-Visual-Explanations.html#deconv-and-guided-...
0
votes
0answers
25 views

Interpretation of statement over probability distributions and functions

Consider the following paragraph from this research paper: To learn the generator’s distribution $p_g$ over data $x$, we define a prior on input noise variables $p_z(z)$, then represent a mapping ...
0
votes
0answers
24 views

Question about implementing GAN(Generative Adversial network)?

so most examples I've seen create two NN-s, train them, then they stack them, make the discriminator part untrainable and then train this stacked NN, why do we do this? So loss, that is calculated on ...
0
votes
0answers
8 views

Higher order reinforcement learning - that discovers new states and new actions (possibly in lifelong learning setting)?

Is there higher order reinforcement learning, that can not only find rewards (and hence optimal policy), bet that can also find the necessity to introduce new states and actions to better model the ...
1
vote
1answer
33 views

What is the most efficient way to test whether a set $X \subset \{0, 1\}^n$ and its complement $\{0, 1\}^n \setminus X$ are linearly separable?

I am interested in algorithms that have optimal running time, and ideally which are also very easy to implement. If you can also give some tips on how to implement the algorithm(s) you mention in the ...
0
votes
0answers
21 views

How to compute the derivative using chain rule of hidden layer (more than 5 neurons for hidden layer) with bias

In the given problem having 8 inputs with 5 hidden layers and 3 output layers and bias(b1) on hidden layers and bias(b2) on ...
0
votes
0answers
26 views

Sorting using AI / neural net

I have a search operation taking place on a server that essentially queries images using OpenCV against other images from a database. Since each image query ...
1
vote
1answer
21 views

List of all possible reasoning tasks - satisfiability and theorem proving only?

What is the exhaustive list of reasoning tasks? As far as I can understand, then any logical reasoning reduces to 2 tasks only: 1) satisfiability problem (finding the assignment of the variables) and ...
0
votes
1answer
40 views

Is it possible to detect cats from dogs in image with single layer perceptron?

I want to make a simple application that input is an image and output must be 0 if image is dog and 1 if image is cat. Is it possible to detect cats from dogs in image with single layer perceptron?
2
votes
3answers
210 views

Could neural networks be considered metaheuristics?

A metaheuristic is defined as a metaheuristic is a higher-level procedure or heuristic designed to find, generate, or select a heuristic (partial search algorithm) that may provide a sufficiently ...
0
votes
1answer
40 views

How can we get small test error reducing only train error?

My question is about mathematical part of machine learning algorithms, especially about using it in neural networks. We train network reducing train error and I was thinking about how then test error ...
0
votes
0answers
15 views

Loss function for spurious data

I am trying to train an autoencoding neural network (autoencoder) to reconstruct seismograms. Previous studies employing this technique (e.g. Valentine & Trampert, 2012) used an L2 (mean squared ...
2
votes
0answers
26 views

How do convolution layers work?

So let's say I have a 4x4 image with 3 channels. So we have 3 4x4 matrices where each matrix represents a channel. Now let's say I also have a 3x3 kernel. I know that for convolution layers, I have to ...
1
vote
1answer
60 views

How to train this neural network?

I seek the neural network (NN) which satisfies the 100 equations (i=1,2...100) $\sum_{j=1}^{2000} NN(A_{ij},B_{ij},C_{ij})=Q_i$. Where A,B,C are 100x2000 matrices So I know Q, A, B and C How can ...
1
vote
1answer
35 views

What does “Temporal extent” mean?

I am reading Long-term Temporal Convolutions for Action Recognition and under the Section 3.1, I read this: To investigate the impact of long-term temporal convolutions, we here study network ...
2
votes
1answer
68 views

Learning the activation function in a neural network?

Neural networks use specific activation functions, commonly used ones are tanh, ReLu. I have seen that people have experimented with continuously parametrices activation functions, for instance here. ...
0
votes
1answer
25 views

How do we fix area of detectors in object detection?

I have gone through various articles on medium and also some from other sites trying to understand SSD. I am able to figure out most of the things from articles except this one. They always say that ...
0
votes
0answers
10 views

Architecture of ANN to combine images

I am interested in ANNs that combines two images in single as it shown below: But I don't know how it's called and can't find any papers or tutorials about the architecture of these ANNs. Can ...
1
vote
0answers
32 views

Converting (reverse-engineering) Turing machine into program or most concise algorithm?

It is known that every program or every algorithm can be converted to Turing machine. But what about the reverse process? Is there algorithm (or research trend that considers such algorithm) to ...
1
vote
1answer
33 views

Learning a perceptron from stream data

I want to train a Perceptron using stochastic gradient rulefrom the stream data. I have very limited amount of memory and i can store only $N$ examples. Suppose my population consist of point as ...
2
votes
1answer
20 views

Is it still transfer learning if you consider input as well as output? (neural networks)

I'm new to the CS stack exchange, so a fond hello to you all! I joined since I have a question I've been curious about. I have recently been running some experiments in transfer learning - ...
0
votes
0answers
46 views

Crossover topologically identical neural networks

I have recently learned about artificial neural networks (very interesting) and genetic algorithms (also very interesting). I have read some suggestions concerning how to crossover two parent neural ...
0
votes
1answer
448 views

Single Layer Perceptron vs Multi Layer Perceptron

Why the single layer perceptron has a linear activation function while the Multi Layer Perceptron has a non-linear activation function ? What is the potential of the Multi Layer Perceptron respect of ...
0
votes
0answers
22 views

Finding subgraph in multigraph using deeplearning

I have big multigraph each node represent entity with 0..n attributes(e.g. name, address, salary). My problem is: I will get for example 10 subgraph selected from user and these subgraphs represent ...
0
votes
1answer
73 views

How many neurons is needed for UAT to hold?

The universal approximation theorem states that a feed-forward network with a single hidden layer containing a finite number of neurons can approximate continuous functions on compact subsets of Rn, ...
4
votes
2answers
209 views

Why is the manifold hypothesis true?

The manifold hypothesis is the statement that real-world high dimensional data (such as images) lie on low-dimensional manifolds embedded in the high-dimensional space. It has been tested to be true ...
0
votes
0answers
32 views

What is the practical limit to how many object classes you can detect with Faster RCNN?

I am trying to follow this tutorial where the Faster-RCNN-Inception-V2-COCO model from TensorFlow's model zoo is used to detect playing cards. I was wondering what is the practical limit to the number ...
2
votes
2answers
107 views

How broad is the meaning of “algorithm”?

This is a purely terminological question. The word algorithm, as I have learnt it refers to something like an "effective method, a sequence of steps, for doing something". There are alternative ...
2
votes
0answers
27 views

Multilayer Perceptrons for solving variational problems

Can we use a multilayer perceptron to solve variational problems? By variational problem I mean something we might encounter in the calculus of variations, for example the geodesic problem: given two ...
0
votes
0answers
8 views

About cross-entropy loss

Is there a classification scenario with cross-entropy loss such that the loss as a function of the predictor/neural net's parameters is a function s.t it satisfies the properties of (a) having a ...
1
vote
1answer
41 views

Linear Regression using a Neural Network

I am trying to create a regression model using a Neural Network. I am currently learning how to work with neural networks (deeplearning.ai) and so the model is not implemented using any existing ...
1
vote
1answer
90 views

(OCR ) How to Recognise Handwritten fractional numbers using Neural networks

I want to be able to recognise handwritten math numbers using images of the numbers , i was able to do create a ANN model for recognising simple decimal numbers , but i have no idea on how to ...
1
vote
0answers
120 views

Why is the method of im2col with GEMM is more efficient than the method of direction implementation with SIMD in CNN

The convolutional layers are most computationally intense parts of Convolutional neural networks (CNNs).Currently the common approach to impement convolutional layers is to expand the image into a ...
3
votes
0answers
23 views

When is currying more efficient in deep neural nets?

I'm reading a blog post on deep Q-learning, and it contrasts traditional lookup-table-based Q-learning with deep Q-learning: What I wonder about in this picture is: Why does the deep NN not also ...
2
votes
1answer
262 views

Analog circuits for neural networks?

Neural networks in machine learning are inherently a continuous model of computation. Yet we use digital logic circuits with floating point numbers to "emulate" this continuity. I am wondering: is ...
1
vote
0answers
82 views

Proof of perceptron convergence theorem for ZERO threshold?

The generalized perceptron convergence theorem is for a defined threshold T. When you do the maths it all comes to an upper bound and a lower bound. The lower bound looks like this! Therefore ...
1
vote
1answer
151 views

Can we use Convolutional Neural Network for dataset containing numeric data? [closed]

I am working on a project of " Fraud detection using deep learning" . For that I have a dataset containing some numerical attributes . Now the task is to use CNN for the above purpose. Please guide me ...
2
votes
1answer
42 views

Is Artificial General Intelligence possible with our current machine learning models? [closed]

In other words, is artificial human level intelligence not possible yet just because of limitations in processing power and amount of data required to train the models? Or we don't have the knowledge ...
2
votes
1answer
96 views

How to represent symbolic knowledge using real numbers - theory about neural networks and natural/analog computing?

One can define the semantics of one definite word using the references to real world entities, relationships with the other words and other concepts and represent all this knowledge about this one ...
1
vote
1answer
73 views

Proving Monotonicity of Softmax Layer

In the book here: http://neuralnetworksanddeeplearning.com/chap3.html If you scroll down to Exercise 2 in the Softmax Section, it says Show that $\partial a^L_{j}/\partial z^L_{k}$ is positive if $...
4
votes
0answers
20 views

How to represent sentences with their dependency parses as input to an RNN?

I am working on a task embedding sentences into a lower-dimensional space according to style, both grammatical and lexical. As such, I want to have as input the linear ordering of tokens in each ...
3
votes
1answer
87 views

Weird behaviour of softmax derivative?

I have been implementing some neural networks in MATLAB and recently I noticed a weird thing while implementing softmax derivative: Setting the derivative to one, rather than using the actual ...
0
votes
1answer
60 views

How to calculate the weight between neurons in ANN?

I am currently learning Supervised ANN training using Backpropogation and I am stuck in this exercise. I calculated the δA using the equation at the bottom of the screenshot, however, I am unable to ...
0
votes
1answer
20 views

Correct cost function of multi classification problem using neural network?

I am going through machine learning course on coursera. While going through the section on neural networks I came across the cost function for multi - classification problem using neural networks ( ...