Hinge Loss

Hinge loss (SVM loss)

1. Definition: The correct score should be larger than a margin (1, and the 1 here actually can be arbitrary) than wrong scores

2. Min 0, Max infinity

3. At the initial time, every class scores should be similar and the expected loss is C-1 ( each wrong class has loss 1, and there are C - 1 wrong classes)

4. Squared hinge loss is different from hinge loss, squared loss focusing more on the bad cases, a small difference may have huge loss value at the end.