boosting

Definition

A family of methods that build a strong predictor by adding weak learners in sequence, each fitted to the errors the ensemble has made so far. Combined with shallow decision trees this gives gradient boosting, still the top performer on most tabular problems. Contrast bagging, as in random forests, where learners are trained independently and averaged; boosting is inherently sequential and overfits without shrinkage.

Explainers using this term

• Decision Trees and Gradient Boosting — Still the Champion on Tabular DataXGBoost: A Scalable Tree Boosting System