Confusion regarding terminology surrounding perceptron learning

Question

I am having trouble understanding the terminology with perceptron learning. Is my current understanding correct? Let's say I have some data that classifies what type of flower a particular flower is. And let's say the factors taken into consideration are petal size, petal coloring, and leaf size. So my current understanding is that we take every pair of inputs and make them the axis of a graph(e.g. leaf size vs petal coloring). So in this case, we would have 3 graphs. Now, we plot the data points and see if it is linearly separable. That is, we can draw a line called a "decision boundary" that separates the data into two regions to be able to differentiate which inputs correlate with which outputs. This line is defined by $w^T \cdot x = y$. However, my first confusion is the following. How are 3 different graphs each with pairs of inputs (the 2 axis) represented using a equation of a single line that is used as the input layer of the neural network?

My second confusion is, is the objective function what defines how the $w^T \cdot x$ is to be interpreted? For instance, something like $y = 1 $ if $w^T \cdot x \geqslant 0$ and else $y = 0$. Also, as I have understood, the learning rule is supposed to be how to update consecutive weights.

Can someone please explain the situation I have laid out. I am reading Bishop's book on Neural Networks. But I am confused by the text.

Answer 1

In machine learning, the perceptron is an algorithm for supervised learning of binary classifiers.

Its much complex than just drawing some graph and fitting a line to divide these data. I should tell you that you gone entirely wrong about the core concept.

Our brain are capable of solving complex problem, few of which are impossible for a computer to figure out. Thus scientist studies our brain and tried to mimic its biological structure to achieve its capability in computers. One of the initial steps towards it was perceptron.

Two types of perceptron are:

1. Single Layer perceptron - works only in linearly separable data.
2. Multi Layer perceptron - work with non linear data too.

Perceptron have the capability of learning from a given (training) data and implement that knowledge into another (new) data as we desire.

Your problem: (desired output)

classify what type of flower a particular flower is?

There are thousands and thousands of types of flowers so, for simplicity I'm modifying you question as "classify if this flower is jasmine or not"

Input you fed to the perceptron are: (features)

petal size, petal coloring, and leaf size

If you construct a perceptron (network) it will be:

$x_1, x_2, x_3$ are input neuron, $w_1, w_2, w_3$ are weights (mimicking the strength between biological neurons) and $Y$ is the output neuron.

You may have a question "Why 3 nodes (or neurons) in input and 1 node in output layer?" so I'm explaining that below:

3 neuron in the input layer as we have 3 features(or inputs).

1 neuron in the output layer as we have 1 class (jasmine or not).

Input layer have 3 nodes which will carry you features to output neuron which will classify if its jasmine or not.

Unlike other classification methods, here we should teach the system (like a human brain learns when we first see a jasmine flower).

How will a computer come to know if its a jasmine flower? Answer is, by showing sample features (petal size, petal coloring, and leaf size) of many jasmine to it.

Note: In biology neural network, the neuron get activated when electric impulses exceeds a threshold. Mimicking it, we use a threshold function to active(output one) or not(output zero) a artificial neuron[ threshold value = $\theta$ ].

If the input is a jasmine flower, output neuron will output '1' else '0'.

$ Y = \begin{cases} \text{1,} &\quad\text{if $\sum_{}^{} w_ix_i>\theta $}\\ \text{0,} &\quad\text{else}\\ \end{cases} $

Note: When we consider $w_i$ and $x_i$ as vector ($W$ and $X$ respectively) its can be written as $Y=W^tX$ (you mentioned it as objective function).

We give sample features (petal size, petal coloring, and leaf size) of jasmine to input neurons $x_1,x_2,x_3$ respectively and tell its a jasmine(set output as one). Meanwhile, the weights $w_1, w_2, w_3$ take some random (initial) values and output $Y$ is found. As we have given a jasmine data to input neuron we expect the output to be one but as weights are taken in random, the output wont be one and an error $E$ is formed.

$E =|ExpectedOutput - ActualOutput|$

This error is propagated back, so that according to the error the weights $w_1, w_2, w_3$ can be updated such that next time error will be much lesser or zero.

This process is called Training, the data used to train the perceptron is called Training data and the network after training is called trained network. The training is called back-propagation algorithm.

It learns from the training data we provide and this trained network can easily classify a new flower(can tell if its jasmine or not).

Read these : Biological neural networks, NPTEL Artificial intelligence, MIT Introduction: The Perceptron