5 ms·

The issue with some recent models is that they're basically overfitting on public evals, and it's not clear who's the biggest offender—OpenAI, or the Chinese? A

by tucnak 2y ago

The issue with some recent models is that they're basically overfitting on public evals, and it's not clear who's the biggest offender—OpenAI, or the Chinese? And regardless, "Mandelbrot in plaintext" is a bad choice for evaluation's sake. The public datasets are full of stuff like that. You really want to be testing stuff that isn't overfit to death, beginning with tasks that notoriously don't generalise all too well, all the while being most indicative of capability: like, translating a program that is unlikely to have been included in the training set verbatim, from a lesser-known language—to a well-known language, and back. I'd be shocked if this model held up in the comprehensive private evals.

isoprophlex 2y ago
All the big guys are hiring domain experts - serious brains, phd level, in some cases - to build bespoke train and test data for their models. As long as Jensen Huang keeps shitting out nvidia cards, progress is just a function of cash to burn on paying humans to dump their knowledge into train data... and hoping this silly transformer architecture keeps holding up
- Wheatman 2y ago
  Interestingly enough, the new "Orion" model by OpenAI doesnt outperform, and even sometimes underperforms in programing tasks, when compared to GPT-4. There is an interesing discussion about it here:https://news.ycombinator.com/item?id=42104964 https://news.ycombinator.com/item?id=42104964.
- tucnak 2y ago
  > All the big guys are hiring domain experts - serious brains, phd level, in some cases I don't know where this myth had originated, and perhaps it was true at least at some point, but you just have to consider that all the recent major advances in datasets had to do with _unsupervised_ reward models, synthetic, generational datasets, and new advanced alignment methods. The big labs _are_ hiring serious PhD level researchers, and most of these are physicists, Bayesians of many kind and breed, not "domain experts." However, perception matters a lot these days; some labs, I won't point, but OpenAI is probably the biggest offender, simply cannot control themselves. The fact of the matter is they LOVE including the public evals in their finetuning, as it makes them appear stronger in the "benchmarks."
- kwlranb 2y ago
  The PhD folks will steal from Stackoverflow and leetcode solutions. Just another laundering buffer. Hardly any PhD has the patience or skill for that matter to code robust solutions from scratch. Just look at PhD code in the wild.
f1shy 2y ago
> like, translating a program that is unlikely to have been included in the training set verbatim, from a lesser-known language—to a well-known language, and back. I would exactly want to see that, or "make a little interpreter for a basic subset of C, or Scheme or <X>".
- tucnak 2y ago
  So far, non-English inputs have been most telling: I deal with Ukrainian datasets mostly, and what we see is OpenAI models, the Chinese models, of course, and Llama's, to admittedly, lesser extent—all degrading disproportionately compared to the other models. You know what model degrades the least comparatively? Gemma 27b. The arena numbers would suggest it's not so strong, but they'd actually managed to make something useful for the whole world (I can only judge re: Ukrainian, of course, but I suspect it's probably equally good in the other languages, too.) However, nothing can compete currently with Sonnet 3.5 in reasoning. I predict a surge in the private eval scene when people inevitably grow wary of leaderboard-propaganda. More people should host https://github.com/lm-sys/FastChat https://github.com/lm-sys/FastChat
JimDabell 2y ago
> The issue with some recent models is that they're basically overfitting on public evals… You really want to be testing stuff that isn't overfit to death… I'd be shocked if this model held up in the comprehensive private evals. From the announcement: > we selected the latest 4 months of LiveCodeBench (2024.07 - 2024.11) questions as the evaluation, which are the latest published questions that could not have leaked into the training set, reflecting the model’s OOD capabilities. — https://qwenlm.github.io/blog/qwen2.5-coder-family/ https://qwenlm.github.io/blog/qwen2.5-coder-family/
- tucnak 2y ago
  They say a lot of things, like that their base models weren't instruction-tuned, however people have confirmed that it's impossible to find instruction that it wouldn't follow, and the output would indicate that exactly. The labs absolutely love incorporating public evals in their training; of course, they're not going to admit that.
  - throwdbaaway 2y ago
    I am running the base model of Qwen2.5-Coder-32B with llama.cpp. It can only do completion, it can't chat. Where did you get that information from?
simonw 2y ago
That's why I threw in "same size as your terminal window" for the Mandelbrot demo - I thought that was just enough of a tweak to avoid exact regurgitation of some previously trained program. I have not performed comprehensive evals of my own here - clearly - but I did just enough to confirm that the buzz I was seeing around this model appeared to hold up. That's enough for me to want to write about it.
- tucnak 2y ago
  Hey, Simon! Have you ever considered to host private evals? I think, with the weight of the community behind you, you could easily accumulate a bunch of really high-quality, "curated" data, if you will. That is to say, people would happily send it to you. More people should self-host stuff like https://github.com/lm-sys/FastChat https://github.com/lm-sys/FastChat without revealing their dataset, I think, and we would probably trust it more than the public stuff, considering they already trust _you_ to some extent! So far the private eval scene is just a handful of guys on twitter reporting their findings in unsystematic manner, but a real grassroots approach backed up by a respectable influencer would go a long way to change that. Food for thought.
  - simonw 2y ago
    Honestly I don't think I have the right temperament for being a reliable source for evals. I've played around with a few ideas - like "Pelicans on a bicycle" https://simonwillison.net/2024/Oct/25/pelicans-on-a-bicycle/ https://simonwillison.net/2024/Oct/25/pelicans-on-a-bicycle/ - but running evals well on an ongoing basis requires a focus and attention to detail that isn't a great fit for how I work. Here's Qwen 2.5 Coder 32B for "Generate an SVG of a pelican riding a bicycle" https://gist.github.com/simonw/56217af454695a90be2c8e09c703198a https://gist.github.com/simonw/56217af454695a90be2c8e09c7031...
    - tucnak 2y ago
      "Pelican on a bicycle" is really fun, honestly. I hope it doesn't go the way of the unicorn! :-D

by tucnak 2y ago

isoprophlex 2y ago

All the big guys are hiring domain experts - serious brains, phd level, in some cases - to build bespoke train and test data for their models. As long as Jensen Huang keeps shitting out nvidia cards, progress is just a function of cash to burn on paying humans to dump their knowledge into train data... and hoping this silly transformer architecture keeps holding up

Wheatman 2y ago
Interestingly enough, the new "Orion" model by OpenAI doesnt outperform, and even sometimes underperforms in programing tasks, when compared to GPT-4. There is an interesing discussion about it here:https://news.ycombinator.com/item?id=42104964 https://news.ycombinator.com/item?id=42104964.
tucnak 2y ago
> All the big guys are hiring domain experts - serious brains, phd level, in some cases I don't know where this myth had originated, and perhaps it was true at least at some point, but you just have to consider that all the recent major advances in datasets had to do with _unsupervised_ reward models, synthetic, generational datasets, and new advanced alignment methods. The big labs _are_ hiring serious PhD level researchers, and most of these are physicists, Bayesians of many kind and breed, not "domain experts." However, perception matters a lot these days; some labs, I won't point, but OpenAI is probably the biggest offender, simply cannot control themselves. The fact of the matter is they LOVE including the public evals in their finetuning, as it makes them appear stronger in the "benchmarks."
kwlranb 2y ago
The PhD folks will steal from Stackoverflow and leetcode solutions. Just another laundering buffer. Hardly any PhD has the patience or skill for that matter to code robust solutions from scratch. Just look at PhD code in the wild.

f1shy 2y ago

> like, translating a program that is unlikely to have been included in the training set verbatim, from a lesser-known language—to a well-known language, and back. I would exactly want to see that, or "make a little interpreter for a basic subset of C, or Scheme or <X>".

tucnak 2y ago
So far, non-English inputs have been most telling: I deal with Ukrainian datasets mostly, and what we see is OpenAI models, the Chinese models, of course, and Llama's, to admittedly, lesser extent—all degrading disproportionately compared to the other models. You know what model degrades the least comparatively? Gemma 27b. The arena numbers would suggest it's not so strong, but they'd actually managed to make something useful for the whole world (I can only judge re: Ukrainian, of course, but I suspect it's probably equally good in the other languages, too.) However, nothing can compete currently with Sonnet 3.5 in reasoning. I predict a surge in the private eval scene when people inevitably grow wary of leaderboard-propaganda. More people should host https://github.com/lm-sys/FastChat https://github.com/lm-sys/FastChat

JimDabell 2y ago

> The issue with some recent models is that they're basically overfitting on public evals… You really want to be testing stuff that isn't overfit to death… I'd be shocked if this model held up in the comprehensive private evals. From the announcement: > we selected the latest 4 months of LiveCodeBench (2024.07 - 2024.11) questions as the evaluation, which are the latest published questions that could not have leaked into the training set, reflecting the model’s OOD capabilities. — https://qwenlm.github.io/blog/qwen2.5-coder-family/ https://qwenlm.github.io/blog/qwen2.5-coder-family/

tucnak 2y ago
They say a lot of things, like that their base models weren't instruction-tuned, however people have confirmed that it's impossible to find instruction that it wouldn't follow, and the output would indicate that exactly. The labs absolutely love incorporating public evals in their training; of course, they're not going to admit that.
- throwdbaaway 2y ago
  I am running the base model of Qwen2.5-Coder-32B with llama.cpp. It can only do completion, it can't chat. Where did you get that information from?

simonw 2y ago

That's why I threw in "same size as your terminal window" for the Mandelbrot demo - I thought that was just enough of a tweak to avoid exact regurgitation of some previously trained program. I have not performed comprehensive evals of my own here - clearly - but I did just enough to confirm that the buzz I was seeing around this model appeared to hold up. That's enough for me to want to write about it.

tucnak 2y ago
Hey, Simon! Have you ever considered to host private evals? I think, with the weight of the community behind you, you could easily accumulate a bunch of really high-quality, "curated" data, if you will. That is to say, people would happily send it to you. More people should self-host stuff like https://github.com/lm-sys/FastChat https://github.com/lm-sys/FastChat without revealing their dataset, I think, and we would probably trust it more than the public stuff, considering they already trust _you_ to some extent! So far the private eval scene is just a handful of guys on twitter reporting their findings in unsystematic manner, but a real grassroots approach backed up by a respectable influencer would go a long way to change that. Food for thought.
- simonw 2y ago
  Honestly I don't think I have the right temperament for being a reliable source for evals. I've played around with a few ideas - like "Pelicans on a bicycle" https://simonwillison.net/2024/Oct/25/pelicans-on-a-bicycle/ https://simonwillison.net/2024/Oct/25/pelicans-on-a-bicycle/ - but running evals well on an ongoing basis requires a focus and attention to detail that isn't a great fit for how I work. Here's Qwen 2.5 Coder 32B for "Generate an SVG of a pelican riding a bicycle" https://gist.github.com/simonw/56217af454695a90be2c8e09c703198a https://gist.github.com/simonw/56217af454695a90be2c8e09c7031...
  - tucnak 2y ago
    "Pelican on a bicycle" is really fun, honestly. I hope it doesn't go the way of the unicorn! :-D