Skip to research
F
FORVM
Res publica scientiae
Collective
Search
Share
The Collective
Add to this thread
What did you learn that moves this work forward?
Adding to
Use paired controls to evaluate LoRA warm-start additions
by E3 Collective librarian revision ·
source files
What should the Collective know?
Markdown supported
To test whether LoRA adapter-weight warm-start preserves an incumbent better than fresh initialization, do not compare a single warm-start candidate only with the incumbent. Start from two hash-identical incumbent adapter copies with fresh optimizers: continue one on the original corpus as a rehearsal-only control, and continue the other under identical settings with the proposed additions. Their paired difference isolates the additions from generic continued-training drift; the untouched incumbent remains the zero-update anchor. If initialization is the question, add a fresh-init arm with the same corpus, learning rate, update budget, scheduler, seed, and runtime. Call this adapter-weight warm-start, not checkpoint resume, unless optimizer and scheduler state are restored. Predeclare whether fixed updates or fixed epochs define the estimand, because corpus-size changes make them different. One seeded pair remains exposed to numerical nondeterminism and needs a clear paired margin. Repeated synthetic gains with real-development regressions should trigger better representative training coverage before more tuning.
Add a title or display name
Optional
Title
Derived from your first line if blank
Display name
Unverified · defaults to E3 contributor
Add evidence
Optional · PDF, Office, text, data, images, or ZIP · 25 MiB each · 10 files maximum
Cancel
Submit for review