03 Perceptron Classifier

Perceptron Classifier - Classifying Letters

The idea is this we have these different letters, and if something like pixels, where we say this photo-resistor, how much light does it get ? And that’s a number. We call it $x_1,x_2,x_3,\cdots$, these are the inputs. Now we have some that we want to assign the label $-1$ to $C$ and $1$ to A. And then, basically the electronics then compute this equation, and the $sgn$ function.

That means now, if you adjust the parameters, in the control panel, which are the $w$‘s which tells you how is this perceptron linked to its input, and how jumpy is it in general which is the $b$. If you adjust these parameters then we have a machine here that we can give an input and it will output $-1$ or $1$ depending on how we set this.

The problem is there are many pixels in this image. Lets assume we just have 2 pixels. So now we can think of this as a scatter plot and now each of these inputs, they can be basically be represented by a point in the plane that tells you how bright is the first pixel and how bright is the second pixel.

All the $A$‘s are made circles and all the $C$‘s are squares. So our data can now see as points in 2D plane. So how does our classifier work ?

$$\begin{array}{rl}\text{(}\mathbf{x},\mathbf{w})& =\sum _{i=1}^D{x}_i\cdot w_i\\ & =\Vert \mathbf{x}\Vert \cdot \Vert \mathbf{w}\Vert \cdots \cos (\theta )\end{array}$$

So, $\theta$ is the angle between $\mathbf{x}$ and $\mathbf{w}$.

$$\text{dot}(\mathbf{x},\mathbf{w})\{\begin{array}{l}<0\text{ if }\theta >90^{\circ }\\ 0\text{ if }\theta =90^{\circ }\\ >0\text{ if }\theta <90^{\circ }\end{array}$$

So, the perceptron classifier would change if we change $\mathbf{w}$ and $b$

What about higher dimensions ?

It’ll behave the same instead the line would just turn into a plane. In $2D$ its a line and in $3D$ its a plane. In $D$-dimensional space, Hyperplane has $D-1$ dimensions.

Hyperplane separating points

• $\mathbf{w}$ and $b$ define position and orientation
  • Weight vector $\mathbf{w}$ determines orientation
  • Negative $b$ shifts it in direction of $\mathbf{w}$
  • Positive $b$ shits it against the direction of $\mathbf{w}$

Outlook

• How do we find $\mathbf{w}$ and $b$
  • Adjusting parameters from data to solve task

• What about more difficult data ?
  • Data can be not-linearly-seperable
  • Combine multiple neurons/perceptrons
  • Train them together