Framework
Loss Landscape Vocabulary Framework
v12 · April 2026 · Atlas Heritage Systems Inc. · Working document — not a finished product
Macro-Topology Shapes
Large-scale terrain features produced by the interaction of terrain properties across parameter space. Determines where models converge, stall, or drift. The archaeological signal Atlas seeks lives in specific topological regions.
Inward gradients from all directions. Single smooth minimum. Stable convergence. Produces laminar flow. Archaeological signal absent — remagnetization likely complete.
Goodfellow et al. (2014) neural network optimization problems
Two-sided inward gradients, flat floor. Stall-prone along valley axis. Flat minima generalize better than sharp ones.
Keskar et al. (2016); Izmailov et al. (2018) stochastic weight averaging
Downhill in some parameter directions, uphill in others. High-perplexity, turbulent. Primary archaeological territory — competing orientations were never resolved into clean convergence here.
Dauphin et al. (2014) saddle point problem in high-dimensional optimization
Near-zero gradient everywhere. No signal. Training dies silently. Highest effective drag. Vanishing gradient problem is plateau behavior.
Glorot & Bengio (2010) vanishing gradient; Ioffe & Szegedy (2015) batch normalization
High curvature boundary between basins. Unstable traversal. Which side you fall to matters — determines which attractor captures the model.
Li et al. (2018) initialization and optimization role
The catchment region around a minimum. Wide basins generalize better. Narrow basins are sensitive to perturbation. The model's training path determines which basin it occupies.
Keskar et al. (2016); Izmailov et al. (2018)