information

Definition

In papers this is usually the Shannon sense: a measurable amount of uncertainty reduction, counted in bits or nats, not the everyday sense of facts or content. Entropy, mutual information and the information bottleneck all build on it. In representation learning the amount of information a code retains or discards is frequently the objective itself, not merely a diagnostic.