ragas

Definition

A framework for automatically evaluating retrieval-augmented generation pipelines. Instead of requiring hand-written reference answers, it defines metrics that score the pieces separately — whether the answer is grounded in the retrieved passages, whether the answer addresses the question, whether retrieval brought back the right context — so you can tell a retrieval failure from a generation failure. Because the scoring itself uses a language model, its judgements inherit that model's biases.

Explainers using this term

• Evaluating RAG in Practice — Turning “Seems Better” Into a NumberRAGAS: Automated Evaluation of Retrieval Augmented Generation