regularization

Mar 07

Dropout as Shared-Parameter Bagging

Alex Leung's Blog

A concrete way to understand dropout in deep learning: sampled subnetworks with shared weights, standing in for a much more expensive ensemble.

deep learning book notes regularization

Jan 01

Robustness, interpretability, and scaling of eigenvalue models

Alex Shtoff

Intro We all want our models to perform well. But some of us would also like our models to be efficient, robust, or interpretable. So in this post we will discuss some mathematical properties of these models that are related to these three pillars. Robustness and interpretability may mean...

machine learning regularization interpretability adversarial robustness spectral norm eigenvalue models operator norm