3 ms·
Tiny model overfit on benchmark published 3 years prior to its training. News at 10
by 100ms 6mo ago
Tiny model overfit on benchmark published 3 years prior to its training. News at 10
- bigyabai 6mo agoBut GPT-3.5 was benchmaxxing too.
- 100ms 6mo agoGPT 3.5 Turbo knowledge cutoff was circa 2021. MT-Bench is from 2023. Not suggesting improvements on small models aren't possible (or forthcoming, the 1.85 bit etc models look exciting), but this almost certainly isn't that.
- fredmendoza 6mo ago[dead]
- srslyTrying2hlp 6mo ago[dead]
- selimthegrim 6mo agoIt wasn't important enough to make the 11 o'clock program.