omnimodal

Definition

Claims that one model handles every modality — text, images, audio, video — on both the input and the output side. It is multimodality pushed further: instead of bolting a converter onto each stream, the design maps them all into a shared representation. The word has no agreed definition, so check which modalities are actually supported in which direction; many "omni" systems are far stronger in some than in others.

Explainers using this term

• Paper Walkthrough: EchoWM — A World You Can Walk Into, and Hear, Driven by Camera IntentEchoWM: Open and Enterable Omnimodal World Models