Moonshot
Treat the submission artifact less like a checkpoint and more like a compiled binary with macros.
Instead of storing most matrices directly, store:
- a tiny shared reconstruction engine
- a latent construction tape for each block or matrix tile
- sparse corrections for the hard leftover structure
Why this is outside the current prior
Most compression work still assumes the artifact is fundamentally “quantized tensors plus side information.” This moonshot changes the artifact ontology itself.
The object being stored becomes:
instructions for building the model
not simply the model weights in a smaller numeric format.
Mechanism sketch
A concrete version might use:
- one tiny shared decoder or compiler network
- tile-level latent codes
- optional role or block IDs
- correction atlases only where reconstruction error stays stubborn
The compiler amortizes across many matrices. The tape and corrections carry model-specific detail.
Why it might matter for Parameter Golf
If many tensors share repeated local structure, then directly storing them is wasteful. A compiled representation could win when:
- decoder cost is amortized widely enough
- latent tapes stay tiny
- corrections are sparse and structured
Cheapest falsifier
Prototype only a tiny subset:
- one repeated block family
- one shared decoder
- one correction stream
Kill it if decoder overhead overwhelms the reduction before residuals become truly small.
What would make it real
- shared decoder cost amortizes over many tensors
- final artifact beats direct quantized storage at equal quality
- reconstruction can be made deterministic and cheap enough for the challenge path