Consider binary classification, two possible labels y∈{−1,1}y \in \{-1,1\}, with binary output according to

g(z)={−1if z<0+1if z≥0g(z) = \begin{cases} -1 \quad \text{if } z < 0 \\ +1 \quad \text{if } z \geq 0 \end{cases}

where z=f(x,w)z= f(x,w) is a model's predication.

Assume a linear model f(x,w)=wTxf(x,w) = w^T x.

A perceptron is a linear classifier which uses the loss:

L(y,f(x,w))=1n∑i∈{misclassified}−yif(xi,w)=1n∑i∈{misclassified}−yiwTxiL(y,f(x,w)) = \frac{1}{n} \sum_{i \in \{\text{misclassified}\}} -y_i f(x_i, w) = \frac{1}{n} \sum_{i \in \{\text{misclassified}\}} -y_i w^T x_i

where i∈{misclassified}⟺{i:yixiTw≤0}i \in \{\text{misclassified}\} \iff \{ i : y_i x_i^T w \leq 0 \}.

References

  1. Anna Choromanska. ECE-GY 7143 course slides, lecture 2.