#contrastive-learning
5 articles
01
·★ MEMBER·PAPER·9 min read
Paper Walkthrough: The Design Fundamentals of Pixel Text Representation Learning
An encoder that reads meaning straight off the pixels, never converting glyphs to character codes. This EMNLP 2026 paper argues that what decides its quality is not data volume but four design choices — explained from zero.
02
·Machine Learning Basics·★ MEMBER·PAPER·11 min read
Self-Supervised Learning — The Day Unlabeled Data Became an Asset
Nobody has to label the data — the data can write its own exam. A ground-up tour of the two families (masked prediction and contrastive learning), from the intuition through the InfoNCE loss, an interactive figure, PyTorch code, and finally why LLM pretraining is the largest self-supervised system ever run.
03
·Information Theory·★ MEMBER·PAPER·11 min read
Mutual Information — Putting a Number on What You Know
How much does knowing one thing shrink your uncertainty about another? Mutual information turns that into a single number. We build it from entropy subtraction, work through conditional MI, the data processing inequality and why estimation is hard, and end at why the InfoNCE loss behind modern contrastive learning is called a lower bound on mutual information.
04
·Audio & Speech·★ MEMBER·PAPER·9 min read
Paper Walkthrough: Interpretable MEG Decoding of Perceived Speech — Reading the Decoder's Weights as a Brain Map
From three seconds of magnetoencephalography, this decoder picks the audio a listener was hearing out of 1005 candidates with 39.75% Top-1 accuracy — and, unlike its predecessors, its trained weights translate into cortical locations and rhythms, while paired occlusion experiments reveal which speech features it actually relies on.
05
·VLMs & Multimodal·★ MEMBER·PAPER·7 min read
Paper Deep Dive — CLIP: Putting Words and Images on One Map
A reading of the CLIP paper (Radford et al., 2021) grounded strictly in its own text: how the in-batch positives and negatives work, why zero-shot classification is really a hypernetwork trick, the five points that prompt wording is worth, and the long list of limits the authors wrote themselves.