anchored weight decay

How to Prevent Catastrophic Forgetting in LLM Fine-Tuning

Use Anchored Weight Decay to reduce prior-task drift during LLM fine-tuning.