# Neural Network Design Challenge

i'm studying for PHD Entrance Exam on Stanford. one of previous material exam designed very challenging.

i want to design a NN for classifying following 2-class problem.

1) output should be -1 or 1.

2) output should be 0 or 1. for sove this question i think using XOR neural network for following problem for above dataset? i'm not sure because one point under the X-axis in this picture and above data all is above X-axis? can anyone help me ? i want to draw a NN for two case mentioned above and find minimum number of node (without input node). for example i think for 0 or 1 output we have following NN with (5 nodes and 2 input node) ## 1 Answer

Artificial Neural Network design is as much art as it is science. Art in the sense that many problems have varying approaches and designs which may not be uniquely characterized.

After this simple prologue, this is a generalisation of the ANN XOR problem which multi-level ANNs have been shown to solve eficiently.

It can also be seen as transformation of the original XOR problem (simply rotate the data by $\pi/4$) and explosion by a factor.

Usually ANNs employ 3 layers (1-input 1 -intermediate and 1-output), of course more layers can be added. This simply illustrates a trade-off between computational power versus computational complexity.

By the universal approximation theorem arbitrary (compact) mappings can be approximated by a (sufficiently complex) MLP ANN .

A further approach is to use a hidden layer with $2n$ or $2^n$ neurons (by symmetry of the problem) and experiment with that.

UPDATE:

As per the OP's further comments regarding transformations between variables $\in \{0,1\}$ to variables $\in \{-1,1\}$, a link for an MLP ANN for the XOR problem using -1,1 values

• thanks Nikos. Dear Nickos, can i say we can using a ANN with 3 node at minimum? (2 input + 3 other nodes) we can correctly classify such data? – user17973 Jun 15 '14 at 18:57
• @user17973, unfortunately i cannot tell, i will have to try it specifically, but i have the feeling it will need an even number of hidden neurons to achieve the $\pi/4$ rotation.. hope this helps you – Nikos M. Jun 15 '14 at 19:01
• @user17973 On the other hand it is known that the XOR problem uses an even number of hidden neurons, so since the data here can be pre-rotated and then fed into an XOR MLP, same conjecture follows – Nikos M. Jun 15 '14 at 19:04
• infact i want to find minimum number of node (wihtout input node) for implementation :) 1) for -1 or 1 output we have (2 input nodes + 3 another nodes) 2) for 0 or 1 output we have (2 input + 5 output ). i dont know am i right ? – user17973 Jun 15 '14 at 19:08
• @user17973, sorry dont think i follow, what exactly do you mean by "nodes", the number of nodes/neurons on the hidden layer? if yes my previous comments tried to answer that, maybe sth else? (note encoding is not so much of an issue as far as i can tell, since any bias term can be added in the output node) – Nikos M. Jun 15 '14 at 19:11