network

Definition

Shorthand for neural network: a stack of differentiable operations connecting input to output, trainable end-to-end by gradient descent. That layered structure is what lets one loss update every parameter. Be aware of the homonym — in graph learning and systems papers, network can also mean a graph of entities or a communication network, and graph neural networks involve both senses at once.

Explainers using this term

• Distillation vs. Quantization vs. Pruning — Three Roads to a Smaller ModelDistilling the Knowledge in a Neural Network
• A Field Guide to Distillation Recipes — logit, feature, attention, selfDistilling the Knowledge in a Neural Network
• The Math of Distillation — Why Soft Answers Teach MoreDistilling the Knowledge in a Neural Network