About the behaviour of multi-layer perceptrons

Question

I have a multilayer perceptron. It has an input layer with two neurons, a hidden layer with an arbitrary number of neurons, and an output layer with two neurons.

Given that randomboolean and targetboolean are random boolean values, and the network operates as such:

input(randomboolean); //Set the input neurons to reflect the random boolean
propagateforwards(); //Perform standard forward propagation
outputboolean = output(); //To get the networks output
ideal(targetboolean); //Performs connection updating via back-prop

Is it possible to get the network to map the randomboolean value to the targetboolean value in such a way as the the outputboolean value will correctly match the targetboolean while running in an 'on-line' (where prediction occurs along with continued learning) mode after some arbitrary number of training cycles.

I hear that the network needs to be recurrent to process this as it may be temporal behaviour, however the MLP is a universal computing platform and I assume it should be able to approximate the temporal behaviour needed for this task.

Answer 1

The answer is no. What you want to do is to predict randomness. The perceptron network takes randomboolean(true/false) and it outputs outputboolean(not random!!). The random generation of targetboolean is independent from the generation of outputboolean.

Perceptrons generally learn functions. If you have $f(A)=B$ and $f(A)=C$ and $B\neq C$, then $f$ is not a function.

EDIT: To predict temporal behavior you should add some time dependent variable in the input of the network.

Answer 2

The question isn't very clear, but it seems that you are trying to use a neural network to analyze a pseudo random number generator (PRNG). The idea being that if the values are properly random, the network won't learn anything, and if it will, then the random number generator is flawed.

If this is the case, I don't expect that you'll have much luck. To be fair, many PRNGs have exploitable flaws, but these are very deep and highly specific. Basic statistical tests aren't enough, and I don't expect that a simple neural network will do any better.

Still, it never hurts to try. If you are going to perform this experiment, you should define very clearly where in the sequence of random bits your input and output value come from. No system is going to predict any bit from an PRNG from just a given other bit (unless the relative frequencies of 0s and 1s aren't uniform, but that would make it a very bad PRNG). The best strategy, I think, would be to use the most recent N bits in the bitstring as inputs and the next bit as the target. You can then slide a window over your PRNG's stream of random bits to generate a training set.

Finally, as mentioned, your neural network is trying to learn a probability distribution over outputs rather than a function. The best option here is to have two output nodes, one for 0 and one for 1 whose activations sum to one (and represent the probabilities per bit). If they predict the behavior of the PRNG better than chance, you've shown a weakness in the PRNG. To make sure that the activations sum to one, you can use the softmax activation function for the output layer.

I recommend trying some obviously flawed PRNGs to see where the method works, and at what level of sophistication it breaks down.