self-hosted

Definition

Running the model or service on infrastructure you control rather than calling a vendor API. The attractions are that data never leaves your boundary, cost is tied to hardware rather than per-token billing, and the model version stays put until you decide to change it. The cost is that capacity planning, serving stack, and upgrades become your problem. It is the main practical reason organisations reach for open-weight models.

Explainers using this term

• Paper Walkthrough: From Production Traffic to Post-Training — Folding 200 Internal Apps Into One Self-Hosted LLMFrom Production Traffic to Post-Training: Building a Self-Hosted LLM That Covers the Corporate Request Mix