8
$\begingroup$

I am studying for an AI exam and I'm looking for a better way of solving the following problem:

Graph shows a classification problem in the unit square $[0,1]^2$, where Class A is denoted by the grey area in the figure (not including the points on the dotted lines), and Class B occurs otherwise.

Graph shows a classification problem in the unit square [0,1]^2

To solve this problem, there is a need of constructing 2 perceptrons: both perceptrons output 1 if the input lies in the grey area, and at least one of the perceptrons outputs 0 otherwise.

There are 3 ways of finding appropriate weights for inputs and biases that I know of:

  1. Trial and error
  2. Perceptron learning algorithm (involving random weights, learning rate, multiple epochs)
  3. Geometrical way of finding the vectors of weights (involving finding an orthogonal line to boundary function)

All of them are quite time consuming when you do it with just pen and paper.

Is there a simple way of calculating/finding perceptron weights based on a classification graph?

$\endgroup$
2
$\begingroup$

think that you may be somewhat misinterpreting the point of the question. the intent of the exam question seems to be, "what good perceptron weights would result for this classification problem". the reason for this is that one cannot go through all the steps of a long-running algorithm during an exam or even homework.

so going back to perceptron theory [which is several decades old] the point is that its a classifier based on linearly separable functions, ie linear inequalities. for the 2d case this results in simple algebraic geometry. in other words the perceptron can implement some number of linear inequalities to attempt to create the classification. here it is given as $n=2$. so the question probably intended is, "what are two linear inequalities that a good perceptron algorithm would converge to for this problem".

so it seems to reduce to find the coefficients of:

$y < m_1 x + b_2$

$y > m_2 x + b_2$

so then use basic geometric algebra to determine the coefficients $m_1, m_2, b_1, b_2$ that work for the above diagram. based on the diagram there is not a perfect solution, but a "good" one. (also note that the equations only need to work within $[0,1]^2$.)

intuitively, its also somewhat implied from this analysis why perceptrons are so limited. because its easy even in 2d to draw classification problems for which there is no linearly separable solution (concave shapes, shapes with holes in them, etcetera— some hint of this is seen in noting the above shape is slightly concave). this observation, fleshed out in a more sophisticated/formal/mathematical/rigorous way in [1], caused a lot of controversy at the time & was an important debate in the history of AI, and even apparently discouraged some scientists from investigating more sophisticated simulated neural network models for some time, although Minsky emphatically denied that was his intent and said his work was misinterpreted.

[1] Perceptrons, by Minsky & Papert

$\endgroup$
  • $\begingroup$ ps this technique does seem to correspond to point (3) above, finding the weights via geometrical techniques, but its not really "time consuming" unless its a complex object (eg a many sided polygon etc) $\endgroup$ – vzn Sep 21 '12 at 20:06
  • $\begingroup$ in other words, finding geometrical orthogonal boundary lines is indeed basically "simple" in the 2d case because it reduces to algebraic geometry, but not in higher dimensions. $\endgroup$ – vzn Sep 22 '12 at 17:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.