#gradient-boosting

2 articles

01 ·Machine Learning Basics·FREE·PAPER·12 min read Decision Trees and Gradient Boosting — Still the Champion on Tabular Data On data made of rows and columns, ensembles of decision trees are still the first thing to reach for. From the intuition behind a single split to boosting, what XGBoost and LightGBM actually do, and why neural networks have not taken this territory — built up from nothing. 02 ·Time Series·FREE·PAPER·11 min read Do Transformers Actually Work on Time Series? — The Argument and the Practical Answer In 2022 a single linear layer beat the whole crop of time-series Transformers on the standard benchmarks. The culprit turned out to be tokenization, not attention — a diagnosis PatchTST fixed by patching and iTransformer by transposing the axes. And yet gradient boosting keeps winning the practitioner competitions, for reasons that come down to the shape of real data.