searching

Definition

Letting an algorithm choose a design instead of a human — the sense in titles of the form "Searching for X." Canonical examples are neural architecture search and the automated search over candidate activation functions that turned up Swish. The cost is enormous compute for a design you then use everywhere, so results are judged by whether the discovered component transfers beyond the search setting. Distinct from retrieval-style search at inference time.