#speech

7 articles

01 ·★ MEMBER·PAPER·7 min read Paper Explained: Realtime-Venus — A Voice-and-Video Assistant That Keeps Talking While Work Runs in the Background Listening, speaking, and background delegation all share one clock ticking once per second. We walk through the 9B models and the harness that let an assistant hand off tool calls without pausing the conversation, from analogy to equations to pseudocode. 02 ·Audio & Speech·★ MEMBER·PAPER·9 min read Paper Walkthrough: AuK — Folding Speech Generation and Editing Into One Model AuK is an open-source foundation model that puts speech generation, editing, enhancement and separation behind one door: a natural-language instruction plus optional audio. A ground-up walkthrough of its data design, MMDiT backbone, rectified flow, human-preference and GRPO post-training, and four-step distillation. 03 ·Audio & Speech·★ MEMBER·PAPER·14 min read Paper Walkthrough: VoiceMem — A Left Brain and a Right Brain for Voice Agents, at Zero Added Latency A from-scratch walkthrough of VoiceMem, a memory system for real-time speech interaction. A factual 'left brain' and an affective 'right brain' run in parallel, and the whole retrieval is hidden inside the silence a VAD already waits out — which is how it wins at a top-5 budget. 04 ·Audio & Speech·FREE·PAPER·14 min read Representing Sound — Mel Spectrograms and Audio Tokens Why speech models never eat raw waveforms, and what they eat instead: the chain from short-time Fourier transform to the mel scale to the log to discrete tokens. Covers the window-length tradeoff, why MFCCs dropped the DCT, how acoustic and semantic tokens differ, and the config mismatches that silently wreck audio in production. 05 ·Audio & Speech·FREE·PAPER·14 min read Representing Sound — Mel Spectrograms and Audio Tokens Why speech models never eat raw waveforms, and what they eat instead: the chain from short-time Fourier transform to the mel scale to the log to discrete tokens. Covers the window-length tradeoff, why MFCCs dropped the DCT, how acoustic and semantic tokens differ, and the config mismatches that silently wreck audio in production. 06 ·Audio & Speech·FREE·PAPER·12 min read Speech Synthesis from Scratch — From Text to a Voice Speech synthesis invents a waveform tens of thousands of times longer than the handful of characters it starts from. This walks through why naive regression fails (one-to-many and phase), why text → mel spectrogram → waveform became the standard split, and how a few seconds of reference audio is now enough to carry a voice. 07 ·Audio & Speech·FREE·11 min read Speech Recognition from Scratch — From Waveform to Text How a stream of numbers from a microphone becomes words, starting from zero: spectrogram features, the alignment problem that CTC solved, autoregressive encoder-decoder models, and Whisper — in the order history solved them.