Framework

Loss Landscape Vocabulary Framework

v12 · April 2026 · Atlas Heritage Systems Inc. · Working document — not a finished product

Terrain Topology Global Geometry Navigator Flow & Resistance Structural Integrity Ablation Potential & Tension Architecture Summary

Ablation & the Drift Vector

Ablation removes structural elements — weights, heads, layers — changing the dimensionality of the landscape. Every qualifier shifts simultaneously. The drift vector and drag coefficient delta are proposed metrics with known structural problems identified by Skywork adversarial review.

Ablationwhat changes

Removal of parameters, heads, or layers. Does not simply reduce the model — changes the topology of the space the model navigates. Every qualifier shifts simultaneously.

Michel et al. (2019) sixteen heads are better than one; Brown et al. (2020) GPT-3

Ablation Drift Vectorproposed metric

Displacement of model position in the loss landscape before and after ablation. Identifies what ablated components were doing. Three structural problems identified: dimensionality mismatch, weight-space ≠ functional distance, post-hoc ≠ pre-training absence.

Δθ = θ_post − θ_pre

Flagged: dimensionality mismatch — before and after ablation the model lives in spaces of different dimension. Embedding choice is interpretively load-bearing and unspecified.

Li et al. (2018) visualizing loss landscape; Meyes et al. (2019) ablation studies

Drag Coefficient Deltaproposed metric

Change in the resistance profile caused by ablation. Computationally expensive. Not yet a named method in the literature. Requires full knowledge of all terrain and navigator properties pre- and post-ablation.

Δb(θ) = b_post(θ) − b_pre(θ)

Proposed — mathematical grounding neighbors but does not fully support this application

Meng et al. (2022) ROME; Olah et al. (2020) circuits

Priority experiment: PyHessian on GPT-2 small to test whether the attention correlation probe (coupling measurement) correlates with actual Hessian off-diagonal structure. Until this is done, the Skywork collapse hierarchy finding is unconfirmed empirically.