I starting to learn about artificial neural network.

I already did some simple things for classification, with different hidden layer.

I would like to know :

Is there a way to create "communication" (synapse) between neurons of the same layer ? If yes how will work the back-propagation ?

Thanks a lot


Yes! That is definitely possible. You should know that not all network topologies are layered. You could also have a network which consists of loose neurons which are connected to one another in a more liquid way:

enter image description here

The reason we have 'layers' in networks is not because they add functionality, but because they make coding easier and more overseeable. When you have for example this network:

input: [neuron1, neuron2]
hiddenlayer: [neuron3, neuron4, neuron5]
output: [neuron6]

that is exactly the same as:

[neuron1, neuron2, neuron3, neuron4, neuron5, neuron6]

Because all neurons get activated sequentially.

So what kind of synapses do you want to create inside a layer? Recurrent or feedforward?

  • $\begingroup$ Thanks, I would like to create something like this (feedforward) but I don't understand how to make a back-propagation in this case (or is there an other method ?), I only learn using layer, and matrix and thing are unclear for me.... what would be the general method to make an algorithm of this kind of network ? $\endgroup$ – Dadep May 8 '17 at 17:37

Keep in mind that the training is about optimizing your cost function given the weights of your connections and the activation functions chosen. As long as you calculate the derivatives correctly(depending on your architecture, coding may become hard) taking care with your weights, everything may work. Depending on the architecture there may be changes in performance, which you would need to compare with other architectures.

  • $\begingroup$ Thanks a lot, you resume my problem saying : "depending on your architecture, coding may become hard"... My real problem could be formulated this way : if I have random connections how do I (back)propagate the error ? The node should receive the "error" from node that the error had already been calculated ...? no ? $\endgroup$ – Dadep May 11 '17 at 11:46
  • $\begingroup$ On the layered case, the derivative of each layer $N$ uses the layer $N+1$ because the connections goes to the layer $N$ to the layer $N+1$. If you have connections going from layer $N$ to itself, you just need to adapt the derivatives. Taking for instance the wikipedia page for backprop: en.wikipedia.org/wiki/Backpropagation. In the section "Finding the derivative of the error" see how the derivatives are taken with respect to $w_{ij}$. In your case you would need to use the $w_{ij}$, where $i$ and $j$ are in the same layer. $\endgroup$ – Victor Hugo May 11 '17 at 12:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.