# Model Ensemble

1. Train ten models and ensemble them.
2. Keep good snapshots and ensemble them
3. Fancier version of snapshot ensemble: **crazy high learning rate scheduler**. &#x20;

[https://arxiv.org/pdf/1608.03983.pdf](https://arxiv.org/pdf/1608.03983.pdf)

[https://arxiv.org/pdf/1704.00109.pdf](https://arxiv.org/pdf/1704.00109.pdf)

&#x20;   4\. **Moving average of the parameter vector.**

![](https://443921002-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LGHUhl6VYqrZm4Re77O%2F-LLHYf3G_tCf2iN5nTwz%2F-LLHYw1-C9vG9hc4bXiI%2FScreen%20Shot%202018-08-31%20at%205.36.26%20PM.png?alt=media\&token=f6bdd849-b95d-46fa-90ac-f63dfc50d534)

&#x20;[http://www.meyn.ece.ufl.edu/archive/spm\_files/Courses/ECE555-2011/555media/poljud92.pdf](http://www.meyn.ece.ufl.edu/archive/spm_files/Courses/ECE555-2011/555media/poljud92.pdf)

