The Llama Family from Scratch — The Main Line of Open LLMs

How Meta's Llama became the main line of open LLMs. The contrarian bet behind Llama 1, the lineage running through Llama 3.x, the llama.cpp / GGUF / Ollama ecosystem that grew around it, and the licensing traps to read before you ship commercially — starting from no prior knowledge.

The restaurant that published its recipes

A closed LLM like ChatGPT is a fine restaurant. You get to eat the food — the answers — but the recipe, the model's insides, never leaves the kitchen. You cannot borrow the kitchen, and you cannot adjust the seasoning to your own taste.

An open LLM hands out the recipe instead — or more precisely, the weights: the billions of parameter values a neural network ends up holding once training is done. Get the weights and everything else opens up. You can run the model on your own laptop or server, keep training it on in-house data, or take it apart to study how it works.

Since 2023, the centre of that open world has been Meta's Llama family. Get this lineage straight once and news about the other open models — Mistral, Qwen, Gemma — starts reading easily too, because most of them take Llama's design as their starting point.

Llama 1 (February 2023) — the contrarian bet: build it small, train it long

The received wisdom at the time was that capability follows parameter count. GPT-3 had 175 billion parameters, and every lab was racing to build something bigger. Then DeepMind's Chinchilla work (2022) showed that for a fixed compute budget you come out ahead by shrinking the model and feeding it more training data.

Llama 1 took that one step further. The stance: accept some inefficiency in training cost in exchange for a smaller model that is cheap to run at inference time, and train it on as many tokens as you possibly can. Four sizes — 7B, 13B, 33B and 65B — trained on up to 1.4 trillion tokens. The paper reported that the 13B model beat GPT-3, more than ten times its size, on many benchmarks. That was the moment "bigger is not everything" stopped being an argument and became a result.

The license was research-only and gated behind an application, but the weight files leaked onto the internet not long after release. Ironically, it was that leak that moved history. Within weeks came derivatives — Stanford's Alpaca (Llama 7B trained further on instruction-response data), then Vicuna, then more. What formed there is the prototype of today's open-LLM culture: as long as somebody builds the foundation, seasoning it — fine-tuning — takes very little compute.

Before the lineage: an LLM is a machine that outputs the probability of the next word

There is one operating principle shared by every Llama that is worth pinning down first. An LLM is a machine that outputs a probability distribution over what the next token — a fragment of a word — will be as the text continues. And the dial that controls how sharply peaked that distribution is is the temperature $T$.

Read it in words: take the raw score $z_i$ the model assigned to each candidate token $i$ — called the logit — divide it by the temperature $T$, then convert it into a probability $p_i$ with a softmax, the normalisation that makes everything sum to 1. Turn $T$ down and probability piles onto the top candidate, so the output goes rigid and stable; turn it up and the gaps between candidates flatten out, so the output goes varied and random.

The --temp option in the local-inference tools coming up is handing you that $T$ directly.

A look inside: a conservatively improved Transformer

Llama's architecture is a decoder-only Transformer — attention stacked many layers deep. There is almost nothing newly invented in it; what characterises it is that it soberly loads in every improvement already shown to work by that point. Three of them stand out.

The first is RMSNorm (pre-normalisation). Before a vector enters a layer, divide it by its own "typical magnitude" to put everything on a common scale.

Read it in words: take the $d$-dimensional input $x$, divide it by the root of the mean of its squared components — that root is the vector's typical magnitude — to equalise the length, then scale it back component by component with a learnable gain $g$ ($\epsilon$ is a tiny constant that prevents division by zero, and $\odot$ is the element-wise product). Because it never subtracts the mean, it costs less to compute than a standard LayerNorm.

The second is SwiGLU, an improvement around the activation, empirically known to squeeze more capability out of the same model size than the ReLU family. The third is RoPE (rotary position embedding), which encodes a word's position by rotating vectors. Later generations were able to stretch the context length so far precisely because this design had good bones.

That combination — a decoder-only Transformer with RMSNorm + SwiGLU + RoPE — became the de facto standard shape for the open LLMs that followed, Mistral and Qwen among them. This is why learning Llama translates directly into understanding the rest.

Llama 2 (July 2023) — commercial use unlocked, and chat arrives

Two things changed with Llama 2. First, commercial use was permitted, making it the first Llama a company could build into its own product. Second, alongside the bare foundation model it shipped Llama 2-Chat, already tuned for dialogue. That tuning runs through the stages that polish responses with human feedback — SFT and RLHF — and those stages are taken apart in the article on instruction tuning and RLHF.

The sizes were 7B, 13B and 70B, the training data grew to 2 trillion tokens, and the context length doubled to 4096. The largest model, 70B, introduced GQA (grouped-query attention) — cutting the number of Key and Value heads to save memory at inference time.

Llama 3.x (2024) — data taken to its limit, and the lineage completed

Llama 3 (April 2024, 8B and 70B) had two headline design changes: a new tokenizer with the vocabulary expanded to roughly 128,000 entries — which expresses the same text in fewer tokens, so the same context length effectively reads further — and GQA adopted across every size. Training ran past 15 trillion tokens, more than seven times Llama 2. That is the philosophy Llama 1 started with, a smallish model on an enormous pile of data, carried through to its conclusion. From there the updates came in quick succession.

• 3.1 (July): adds 405B, among the largest openly released weights, and extends the context to 128K
• 3.2 (September): adds 1B and 3B for devices, plus 11B and 90B that can also read images (vision)
• 3.3 (December): an improved 70B, tuned to reach large-model usability at 70B's inference cost

A 128K context is not free, though. The reason is that attention's compute grows with the square of the sequence length.

Llama 4, which followed in 2025, steered toward an MoE (mixture-of-experts) structure. The Llama 1 → 3.x lineage of dense Transformers that this article covers can fairly be said to have reached a kind of completion at 3.3.

The ecosystem: llama.cpp, GGUF, Ollama

Llama's influence rests as much on the tooling that grew up around it as on the models themselves. At the centre sits llama.cpp, the C/C++ inference engine Georgi Gerganov released in March 2023, which changed the current by showing that Llama would run on a laptop with no GPU at all. The key to that is quantization — compressing weights from 16 bits down to around 4 to cut memory use to a fraction of what it was — and the mechanics are explained in the article on quantization.

GGUF, the file format llama.cpp defined, became the de facto standard for distributing quantized models. Search Hugging Face for "GGUF" and you will turn up volunteer-converted quantized builds of just about every model going.

# Running it with Ollama (a friendly wrapper around llama.cpp-style engines)
ollama run llama3.2

# Using llama.cpp itself directly
./llama-cli -m Llama-3.2-3B-Instruct-Q4_K_M.gguf \
  -c 8192 --temp 0.7 -ngl 99

In the filename, Q4_K_M is the name of the quantization scheme (roughly 4 bits per weight); -c is the context length, -ngl is how many layers to place on the GPU, and --temp is the temperature from earlier in this article.

Licensing: "open," but not "open source"

This is the caveat that matters most. Llama's license — the Llama Community License — is Meta's own, and it does not meet the conditions the Open Source Initiative (OSI) sets for "open source." Here are the clauses that bite in practice.

1. A company whose product has more than 700 million monthly active users must obtain a separate license from Meta (in effect, a clause that carves out the giant tech companies)
2. Products and services with Llama built into them carry an obligation to display "Built with Llama"
3. If you use Llama to build another AI model, you are obliged to put "Llama" at the front of that model's name
4. The Acceptable Use Policy restricts what you may use it for
5. The terms differ from version to version. Llama 2, for instance, restricted using outputs to improve other models, while Llama 3.1 relaxed that conditionally — the details keep moving

"Free to download" is not the same as "do whatever you like." Before you build it into a commercial product, always read the license text for the exact version you are using.

How this plays out in practice

Who reaches for it, and when. ML engineers and backend engineers on projects where the data cannot leave the building — healthcare, finance, search and summarisation over internal documents — find that this article becomes their job description the moment they decide "an open model, not an API." The order of operations is: choose the model → choose the quantization level → design the serving setup.

What you actually touch. Model selection starts by estimating memory from parameter count × quantization bits (at Q4 the weights alone come to roughly the parameter count × 0.5 bytes, with the context sitting on top of that). For trying things out locally, llama.cpp or Ollama gives you --temp (temperature), top_p (sample only from the top slice of the probability mass), -c / num_ctx (context length) and -ngl (GPU layers). For production serving, a server such as vLLM is the standard choice.

Pitfalls that turn into incidents.

• Chat template mismatch. The dialogue format — the arrangement of special tokens — differs from generation to generation. Getting it wrong raises no error; the model just keeps running in a vaguely-poor-quality state, which is the worst possible failure mode, and the investigation dissolves days at a time.
• Quantizing too hard. Down to around 4 bits is usually practical, but cut further and the smaller the model, the more easily it breaks. Verify on your own task, never on public benchmarks alone.
• Skipping the license check. The naming obligation on derivative models and the "Built with Llama" notice are easy to forget, and they lead to incidents where you are forced to change a product name or a screen after release.

If it comes up in an interview. "Why did you pick 8B rather than 70B?" — if you can answer that with the allocation between data volume and model size that traces back to Chinchilla, plus a rough estimate of inference cost in memory and response latency, this article has done its job.

Wrapping up

• Llama's essence is the product of two things: a contrarian allocation of compute — a smallish model on an enormous number of tokens — and a distribution strategy of simply handing out the weights
• The architecture (RMSNorm + SwiGLU + RoPE) became the standard shape for open LLMs as a whole, and the lineage reached a kind of completion in Llama 3.x's dense Transformers
• In practice, two things are mandatory: the llama.cpp / GGUF / Ollama toolchain, and a careful reading of a bespoke license that sits outside the OSI definition

Next, follow the mechanics of quantization — the thing that directly governs both quality and memory when you run locally — down to the equations, in the article linked above. It is the shortest route from being someone who can run Llama to being someone who can tune it.