4 ms·
Untraining models is an open research question and not par for the course like you're implying
by jnhl 2y ago
Untraining models is an open research question and not par for the course like you're implying
- PostOnce 2y agoNo, not "untraining" after the fact, but scrubbing the benchmark data from the dataset prior to training. That's absolutely trivial. Take the set of benchmarks you care about, and then as you build a training dataset (by scraping, or whatever), you scrub each new item for benchmark questions (or just discard that entire item/webpage/whatever for being contaminated). Otherwise you're negligently and willingly inflating your benchmark performance and defrauding new investors / users / customers, I think.