> For the complete documentation index, see [llms.txt](https://sisyphus.gitbook.io/project/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://sisyphus.gitbook.io/project/machine-learning/svm.md).

# SVM

Maximize Margin

![](https://443921002-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LGHUhl6VYqrZm4Re77O%2F-LTKxy37hNQ68Ltk2sPK%2F-LTL5esSbAUvPj9BQPhK%2FScreen%20Shot%202018-12-09%20at%207.12.48%20PM.png?alt=media\&token=b84adf4b-565f-4d0e-9041-ad3cc7f2c596)

## Hinge loss (SVM loss)

1. **Definition:** The correct score should be larger than a margin (1, and the 1 here actually can be arbitrary) than wrong scores 
2. **Min** 0, **Max** infinity
3. **At the initial time**, every class scores should be similar and the expected loss is C-1 ( each wrong class has loss 1, and there are C - 1 wrong classes)
4. **Squared hinge loss** is different from hinge loss, squared loss focusing more on the bad cases, a small difference may have huge loss value at the end.

![](https://443921002-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LGHUhl6VYqrZm4Re77O%2F-LP8-kKpmTZDLmcaKLID%2F-LP81K9GYzvQS2CN4oEW%2FScreen%20Shot%202018-10-18%20at%202.16.00%20PM.png?alt=media\&token=43914d4a-59d4-496e-b650-d3fb5d4ad7b1)

### SVM Loss vs Cross Entropy Loss

![Differences](https://443921002-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LGHUhl6VYqrZm4Re77O%2F-LGHYwvncirBGkeyLS8z%2F-LGHa7nGk0U1Ch6QxT1i%2FScreen%20Shot%202018-06-29%20at%204.04.45%20PM.png?alt=media\&token=8e7530ab-4947-4562-abe3-f76f5bde71f3)

### Kernels

#### Gaussian Kernels

![](https://443921002-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LGHUhl6VYqrZm4Re77O%2F-LTKxy37hNQ68Ltk2sPK%2F-LTLIM1K-szIVvLi706-%2FScreen%20Shot%202018-12-09%20at%208.06.14%20PM.png?alt=media\&token=25c39f7b-8b46-4efd-80ab-a2dc7faf4f3b)

#### Need to do feature normalization before using multivariate SVM

![](https://443921002-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LGHUhl6VYqrZm4Re77O%2F-LTKxy37hNQ68Ltk2sPK%2F-LTLINvZ4RtqvxCG0gZq%2FScreen%20Shot%202018-12-09%20at%208.07.00%20PM.png?alt=media\&token=9238617a-f806-4358-8119-ff627da1510e)

#### Other Kernels

![](https://443921002-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LGHUhl6VYqrZm4Re77O%2F-LTKxy37hNQ68Ltk2sPK%2F-LTLIOVHEdHiRi3m1aRF%2FScreen%20Shot%202018-12-09%20at%208.07.22%20PM.png?alt=media\&token=fcdaf9ae-9dba-46f5-af3c-9061f4a340af)