#bayesian-optimization
4 articles
01
·Probability & Statistics·★ MEMBER·PAPER·11 min read
Kernel Methods and Gaussian Processes — The Champions Before Neural Nets
Before deep learning took over, kernel SVMs held the crown in classification. This article builds the whole story from zero: the kernel trick that makes lifting into high dimensions free, Gaussian processes that put a probability distribution over functions, and Bayesian optimization that turns uncertainty into a weapon.
02
·Deep Learning Basics·★ MEMBER·PAPER·12 min read
Hyperparameter Search — Hunches, Grids, and Bayesian Optimization
Gradients tell you nothing about the learning rate, so you have to go looking. Why grid search is weak, why search spaces should be carved on a log scale, what a Bayesian acquisition function is actually counting, and why early stopping beats a cleverer search algorithm — with Optuna code and the traps that bite in production.
03
·Paper Deep-Dives·★ MEMBER·PAPER·13 min read
Paper Deep Dive — Large Discovery Models: giving an LLM a value signal for what to try next
An LLM can propose candidates but cannot reliably grade its own proposals. Large Discovery Models tilts the LLM's proposal distribution by an acquisition value drawn from a Gaussian process surrogate, so search keeps working when every evaluation is expensive. From the equations to the reported numbers.
04
·Probability & Statistics·★ MEMBER·PAPER·10 min read
Bayes' Theorem in AI — Priors, Posteriors, and Uncertainty
One line of math — Bayes' theorem — turns into three workhorse tools in real AI systems: probability calibration, active learning, and Bayesian optimization. Starting from a positive medical test, we build up priors, posteriors, and uncertainty with zero background assumed.