# Model Ensemble

1. Train ten models and ensemble them.
2. Keep good snapshots and ensemble them
3. Fancier version of snapshot ensemble: **crazy high learning rate scheduler**. &#x20;

<https://arxiv.org/pdf/1608.03983.pdf>

<https://arxiv.org/pdf/1704.00109.pdf>

&#x20;   4\. **Moving average of the parameter vector.**

![](https://443921002-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LGHUhl6VYqrZm4Re77O%2F-LLHYf3G_tCf2iN5nTwz%2F-LLHYw1-C9vG9hc4bXiI%2FScreen%20Shot%202018-08-31%20at%205.36.26%20PM.png?alt=media\&token=f6bdd849-b95d-46fa-90ac-f63dfc50d534)

&#x20;<http://www.meyn.ece.ufl.edu/archive/spm_files/Courses/ECE555-2011/555media/poljud92.pdf>
