omni-modal

Definition

Handling text, images, audio, and video within a single model, on both the input and output side. The term is a marketing-adjacent escalation of multimodal, used to claim coverage beyond the usual image-plus-text pairing. In practice the meaningful distinction is usually speech: whether audio is processed end to end within the model, enabling low-latency spoken interaction, or bolted on through separate transcription and synthesis components.

Explainers using this term

• Paper Walkthrough: TLive-Omni — An Omni-Modal Model That Watches, Listens, and Tells You the TimestampTLive-Omni: An Omni-Modal Understanding Model for E-Commerce Live Streaming
• Paper Walkthrough: OmniScientist — An AI Scientist That Actually Looks at the Raw DataOmniScientist: An Omni-Modal Omni-Discipline AI Scientist