Back to home

Tag

evolution strategies

3 articles

How to Prevent Catastrophic Forgetting in LLM Fine-Tuning
Research/Jun 6

How to Prevent Catastrophic Forgetting in LLM Fine-Tuning

Use Anchored Weight Decay to reduce prior-task drift during LLM fine-tuning.

Fixing LLM forgetting in ES fine-tuning
Research/Jun 5

Fixing LLM forgetting in ES fine-tuning

This paper shows LLM fine-tuning with evolution strategies can drift, and anchored weight decay can curb it.

How ESMA Teaches LLMs Self-Knowledge
Research/May 30

How ESMA Teaches LLMs Self-Knowledge

A bias-controlled fine-tuning method improves LLM self-knowledge and generalizes across unseen data, languages, and new facts.