spectral norm

Pion keeps LLM weights’ spectrum fixed

Pion is a new LLM optimizer that updates weights with orthogonal transforms, preserving singular values instead of adding gradients directly.