omni-modal

Definition

Handling text, images, audio, and video within a single model, on both the input and output side. The term is a marketing-adjacent escalation of multimodal, used to claim coverage beyond the usual image-plus-text pairing. In practice the meaningful distinction is usually speech: whether audio is processed end to end within the model, enabling low-latency spoken interaction, or bolted on through separate transcription and synthesis components.

Explainers using this term

• Paper Explained: Can MiniMax-H3 Reason About the Physical World? Testing an Omni-Modal Generator on What the Prompt Leaves OutCan MiniMax-H3 Reason About the Physical World? An Evaluation of Omni-Modal Generative Model
• Paper Explained: Ovis-Embedding — One Ruler for Text, Images, Video, and AudioOvis-Embedding: Pushing the Frontiers of Universal Omni-Modal Embeddings
• Paper Walkthrough: TLive-Omni — An Omni-Modal Model That Watches, Listens, and Tells You the TimestampTLive-Omni: An Omni-Modal Understanding Model for E-Commerce Live Streaming