Framework
Loss Landscape Vocabulary Framework
v12 · April 2026 · Atlas Heritage Systems Inc. · Working document — not a finished product
Structural Integrity
A category distinct from terrain and navigator, governing whether the model's internal representational geometry holds its shape under sustained operational load. Invisible to both static landscape analysis and training-time navigator properties. Observable only at inference time.
Maintenance of consistent representational geometry under sustained inference load, context pressure, and competing objective tension. Rigid means outputs remain consistent. Compliant means the architecture deforms under load.
Elhage et al. (2021) transformer circuits; Meng et al. (2022) ROME
As context window fills, earlier token representation degrades. Collapse is architectural, not weight-based. The weights are frozen; the architecture deforms relative to its own prior outputs.
Liu et al. (2023) lost in the middle
One objective (safety, fluency, instruction-following) overwhelms others under sustained pressure, collapsing the output distribution.
Bai et al. (2022) training a helpful and harmless assistant with RLHF
Input arriving outside the training data manifold — so far outside that the model's representational geometry has no stable orientation for it. The model snaps to the nearest high-probability trained attractor with full confidence, pointing the wrong direction.
Hendrycks et al. (2021) natural adversarial examples; Nalisnick et al. (2019)
A training point where the internal algorithm reorganizes while surface metrics drift smoothly. Loss and curvature change slowly, but representation similarity jumps, attention patterns rewire, or circuit structure reorganizes. Regime switches are phase-transition events in the model's computation — not in the loss surface itself. The landscape looks continuous; the algorithm changes underneath it. This is the local signature of what global geometry calls a phase transition: observable from inside a single training run as a discontinuity in behavioral metrics while loss stays smooth.
Saxe et al. (2019) a mathematical theory of semantic development in deep neural networks; Power et al. (2022) grokking — added v12, Nemotron-3 adversarial review April 2026
Whether long-horizon behavior across checkpoints can be explained by local resistance alone, or whether path history contributes independently. Entangled means viscosity and memory cannot be teased apart from a frozen endpoint: curvature changes and training trajectory produce indistinguishable behavioral effects. Separable means multi-scale checkpoint data reveal behaviors that viscosity cannot account for, isolating a distinct memory contribution. This distinction is the central open experiment of the Atlas framework. The Pythia checkpoint series is designed to produce the data needed to determine which regime applies.
Open problem flagged in Atlas framework v11. Pythia (EleutherAI) multi-checkpoint series is the designated empirical test.