bg de en es fr it nl pl pt sv tr zh

polynomials

Paying attention to feature distribution alignment

Alex Shtoff

Intro Yes, I’m making a joke of the tendency to put the words “attention” and “alignment” in any ML paper 😎. Now let’s see how this provocative title is related to our adventures in the land of polynomial features. The Legendre polynomial basis serverd us well in recent posts about polynomial...

machine learning feature scaling correlation analysis polynomials polynomial regression orthogonal polynomials quantile transformer california housing dataset feature engineering legendre polynomials

Off with the polynomial’s tail!

Alex Shtoff

Intro Last time we did a small curve fitting exercise - we fit high degree polynomials represented in the Legendre basis to a curve, and observe the celebrated “double descent” phenomenon: after crossing the memorization threshold, the generalization error improves as the degree increases. Then,...

machine learning model pruning scikit-learn double descent feature selection polynomials polynomial regression california housing dataset feature engineering legendre polynomials

Let the polynomial monster free

Alex Shtoff

Intro In a recent post by Ben Recht, titled Though Shalt Not Overfit, Ben claims that overfitting in the way that it is colloquially described in data science and machine learning, doesn’t exist. Indeed, there is the famous double descent: trained neural networks that have much more parameters...

machine learning generalization double descent fourier features polynomials overparameterization polynomial regression chebyshev polynomials feature engineering legendre polynomials