#tts

3 articles

01 ·Audio & Speech·★ MEMBER·PAPER·9 min read Paper Walkthrough: AuK — Folding Speech Generation and Editing Into One Model AuK is an open-source foundation model that puts speech generation, editing, enhancement and separation behind one door: a natural-language instruction plus optional audio. A ground-up walkthrough of its data design, MMDiT backbone, rectified flow, human-preference and GRPO post-training, and four-step distillation. 02 ·Audio & Speech·FREE·PAPER·12 min read Speech Synthesis from Scratch — From Text to a Voice Speech synthesis invents a waveform tens of thousands of times longer than the handful of characters it starts from. This walks through why naive regression fails (one-to-many and phase), why text → mel spectrogram → waveform became the standard split, and how a few seconds of reference audio is now enough to carry a voice. 03 ·Audio & Speech·★ MEMBER·PAPER·10 min read Paper Walkthrough: SwanTale — Designing Voices from Words Alone, with Speech and Sound in One Waveform ByteDance's SwanTale handles two jobs in one model: cloning a voice from reference audio (zero-shot) and building voices, ambience, and sound effects purely from a natural-language caption (instruct). We walk through the paper, from the 70M-caption data pipeline to Unified MoE routing and GRPO post-training.