I have recently been investigating dynamic topology neural networks and there is only one problem I have with understanding them. Because a neuron could be inserted at any point, the neurons are no longer organised into layers, so how is it possible to ensure that all of the inputs for Neuron A at position X have been computed, prior to the computation of Neuron A's output, especially when each neurons output is calculated linearly? I am by no means experienced in the topic, just curious so any information would be helpful, Thank you.
Not an expert in neural net. But I recently read and answer a question on NEAT. From there, I would assume that direction of edges are already determined in each gene. Because of the way operations on genome was designed, I could see that you can ensure the genome is always in `topological order'.
Alternatively, looking at heuristic algorithms that iteratively update graph's weight as it learns (e.g. pagerank and ACO), it should be fine to assume initial weight and just let the iteration stabilise. One could imagine this as every edge being directed both ways --- every link is both input and output.