pathterminuspages/notes/aboutcontactabout me

Perceptrons

25.05.2020

A perceptron is a type of artificial neuron. It consists of a series of inputs, $x_i$, and a single output. Each input is a binary value of $0$ or $1$. The output is binary too. To determine the output a threshold is used along a weight for each input. Given weights, input variables and threshold $t$ we calculate the output as $$ \sum_{i} w_i x_i \leq t : 0 $$ and $$ \sum_{i} w_i x_i \gt t : 1 $$ We can extend on this by adding the negative value of $t$ to the sum. Instead of threshold we call it bias. That is let $b = - threshold$, and we get: $$ \sum_{i} w_i x_i + b \leq 0 : 0 $$ and $$ \sum_{i} w_i x_i + b \gt 0 : 1 $$

We can extend further on the notation, we can just write $w \cdot x$ for the dot product of the two vectors $w$ and $x$. And we get $$ output = \begin{cases} 0 & w \cdot x + b \leq 0 \\ 1 & w \cdot x + b \gt 0 \end{cases} $$ Now the bias is a measure of how easy it is for the perceptron to output 1. Or said in another way: how wasy it is to get the perceptron to fire.

Example #1

Given three inputs $x_1,x_2,x_3$, a threshold of say $t = 4$ and weights $w_1 = 5, w_2 = 3, w_3 = 2$ we can see that if we have $x_1 = 0$, then we need $x_2 = x_3 = 1$ in order for the output to be 1. On the other hand if we do have $x_1 = 1$, the output will be 1. That is in order for the output to be 1 we need the following to be true: $$ x_1 = 1 \lor (x_2 = 1 \land x_3 = 1) $$

Chaining/Layers

We can build layers of perceptrons. That is we route the output of a layer of perceptrons to the input of the next layer. The further we go down the layers, the more complex the decisions of each perceptron. Furthermore we can route the output of one perceptron to several inputs. The output is still well defined, it is just used more than once.

Perceptrons as logical gates/connectives

We can quite easily model logical gates using perceptrons with two inputs for binary connectives, and one for unary connectives. For an or-gate we just set $-(b + 1) = w_1 = w_2$. For example $b = -1, w_1 = w_2 = 2$. Now the perceptron fires if $x_1 = 1 \lor x_2 = 1$. For an and-gate we can set $w_1 = w_2 = 2$ and $b = -3$, and the perceptron fires only if $x_1 = 1 \land x_2 = 1$. Ultimately we can build a nand-gate. That is a gate that acts as the formula $\neg (x_1 \land x_2)$ which is logical equivalent to $\neg x_1 \lor \neg x_2$. We set $b = 2$ and $w_1 = w_2 = -1$. Now $$ x_1 = x_2 = 1 \Rightarrow w_1 \cdot x_1 + w_2 \cdot x_2 + 2 = 0 $$ and in all other cases the perceptron will fire. The nand-gate is ultimate in the sense that it is functional complete. All other logical functions can be made from this one gate. Hence everything computable can be made of nand-gates. Thus we can build any program using perceptrons.

CommentsGuest Name:Comment: