3 ms·

I have a few "SVG of an X riding a Y" tests that I don't publish online which I run occasionally to see if a model is suspiciously better at drawing a pelican r

by simonw 1y ago

I have a few "SVG of an X riding a Y" tests that I don't publish online which I run occasionally to see if a model is suspiciously better at drawing a pelican riding a bicycle than some other creature on some other form of transport. I would be so entertained if I found out an AI lab had wasted their time cheating on my dumb benchmark!

ajcp 1y ago
-> I would be so entertained if I found out an AI lab had wasted their time cheating on my dumb benchmark! Que intro: "The gang wastes their time cheating on a dumb benchmark"
- mcny 1y ago
  A shower thought I just had: there must be some AI training company somewhere that has injested all It is always sunny in Philadelphia, not just the text but all the video from all episodes somehow...
Imustaskforhelp 1y ago
Please do let us know through your blog post if you ever find AI labs to cheat on your benchmark. But now I am worried that since you have shared that you do SVG of an X riding a Y thing, maybe these models will try to cheat on the whole SVG of X riding Y thing instead of hyper focusing the pelican. So now I suppose you might need to come up with an entirely new thing though :)
- throwup238 1y ago
  There are so many X and Y combinations that I find it hard to believe they could realistically train for a even a small fraction of them. Someone has to generate the graphics output for the training. A duck billed platypus riding a unicycle? A man o' war riding a pyrosome? A chicken riding a Quetzalcoatlus? A tardigrade riding a surf board?
  - gnatolf 1y ago
    You're assuming that given the collection of simonw's publicly available blog posts, the creativity of those combinations can't be narrowed down. Simply reverse engineer his brain this way and you'll get your Xs and Ys ;)
    - throwup238 1y ago
      I feel like that would over fit on various snakes like pythons.
  - fragmede 1y ago
    If we accept ChatGPT telling me that there are approximately 200k common nouns in English, and then we square that, we get 40 billion combinations. At one second per, that's ~1200 years, but then if we parallelize it on a supercomputer that can do 100,000 per second that would only take 3 days. Given that ChatGPT was trained on all of the Internet and every book written, I'm not sure that still seems infeasible.
    - throwup238 1y ago
      It still can't satisfactorily draw a pelican on a bicycle because that's either not in the training data or the signal is too weak, so why would it be able to satisfactorily draw every random noun-riding-noun combination just because you threw a for loop at it? The point is that in order to cheat on @simonw's benchmark across any arbitrary combination, they'd have to come up with an absurd number of human crafted input-output training pairs with human produced drawings. You can't just ask ChatGPT to generate every combination because all it'll produce is garbage that gets a lot worse the further from a pelican riding a bicycle. It might work at first for the pelican and a few other animals/transport combination but what does it even mean for a man o' war riding a pyrosome? I asked every model I have access to generate an SVG for a "man o' war riding a pyrosome" and not a single one managed to draw anything resembling a pyrosome. Most couldn't even produce something resembling a man o' war except as a generic ellipsoid-shaped jellyfish with a few tenticles. Expand that to every weird noun-noun combination and it's just not practical to train even a tiny fraction of them.
diggan 1y ago
> I would be so entertained if I found out an AI lab had wasted their time cheating on my dumb benchmark! I don't think it's necessarily "cheating", it just happens as they're discovering and ingesting large ranges of content. A problem of public content, it's bound to be included sooner or later, directly or indirectly. Nice to hear you're doing some sort of contingency though, and looking forward to the inevitable blog post announcing the change to a different bird and vehicle :)
- svachalek 1y ago
  The thing is most of the discussion about it is embarrassingly bad SVGs so training on them would actually hurt their performance.
  - JSR_FDED 1y ago
    Regrettably AI is still better at SVG than I am
deleted 1y ago
[deleted]
reissbaker 1y ago
I doubt they'd cheat that obviously... But "SVG of X" has become common enough that I suspect most frontier labs train on it, especially since the models are multimodal now anyway. Not that I mind; I want models to be good at generating SVG! Makes icons much simpler.
fragmede 1y ago
But how would you know it's from what you would consider cheating as opposed to pelicans on bicycles existing in the latest training data? Obviously your blog gets fed into the training set for GPT-6, as well as everyone else talking about your test, so how would the comparison to a secret X riding a Y tell you if an AI lab is cheating as opposed to merely there being more examples in the training data?
- simonw 1y ago
  Mainly because if they train on the pelican on bicycle SVGs from my blog they are going to get some very weird looking pelicans riding some terrible looking bicycles.
  - fragmede 1y ago
    It's not that I claiming they're training on SVG pelicans on bicycles from your blog, it's that thanks to your popularity, there are simply now more pictures of pelicans on bicycles floating around on the Internet and thus ChatGPT's training data. Eg https://www.reddit.com/r/ColoredPencils/comments/1l9l4fq/pelican_riding_bicycle/ https://www.reddit.com/r/ColoredPencils/comments/1l9l4fq/pel... How would you determine that improvements to SVG pelicans on bicycles (and not your secret X on Ys) are from an OpenAI employee cheating your benchmark vs being an improvement on pelicans on bicycles thanks to that picture from Reddit and everywhere elsewhere in the training data?
    - simonw 1y ago
      See comment here: https://news.ycombinator.com/item?id=45454269 https://news.ycombinator.com/item?id=45454269
jgalt212 1y ago
Your benchmark may or may not be dumb, but it is definitely widely followed. So much so this is what Bing AI has to say on the matter. > Absolutely — the “pelican riding a bicycle” SVG test is a quirky but clever benchmark created by Simon Willison to evaluate how well different large language models (LLMs) can generate SVG (Scalable Vector Graphics) images from a prompt that’s both unusual and unlikely to be in their training data.

by simonw 1y ago

ajcp 1y ago

-> I would be so entertained if I found out an AI lab had wasted their time cheating on my dumb benchmark! Que intro: "The gang wastes their time cheating on a dumb benchmark"

mcny 1y ago
A shower thought I just had: there must be some AI training company somewhere that has injested all It is always sunny in Philadelphia, not just the text but all the video from all episodes somehow...

Imustaskforhelp 1y ago

Please do let us know through your blog post if you ever find AI labs to cheat on your benchmark. But now I am worried that since you have shared that you do SVG of an X riding a Y thing, maybe these models will try to cheat on the whole SVG of X riding Y thing instead of hyper focusing the pelican. So now I suppose you might need to come up with an entirely new thing though :)

throwup238 1y ago
There are so many X and Y combinations that I find it hard to believe they could realistically train for a even a small fraction of them. Someone has to generate the graphics output for the training. A duck billed platypus riding a unicycle? A man o' war riding a pyrosome? A chicken riding a Quetzalcoatlus? A tardigrade riding a surf board?
- gnatolf 1y ago
  You're assuming that given the collection of simonw's publicly available blog posts, the creativity of those combinations can't be narrowed down. Simply reverse engineer his brain this way and you'll get your Xs and Ys ;)
  - throwup238 1y ago
    I feel like that would over fit on various snakes like pythons.
- fragmede 1y ago
  If we accept ChatGPT telling me that there are approximately 200k common nouns in English, and then we square that, we get 40 billion combinations. At one second per, that's ~1200 years, but then if we parallelize it on a supercomputer that can do 100,000 per second that would only take 3 days. Given that ChatGPT was trained on all of the Internet and every book written, I'm not sure that still seems infeasible.
  - throwup238 1y ago
    It still can't satisfactorily draw a pelican on a bicycle because that's either not in the training data or the signal is too weak, so why would it be able to satisfactorily draw every random noun-riding-noun combination just because you threw a for loop at it? The point is that in order to cheat on @simonw's benchmark across any arbitrary combination, they'd have to come up with an absurd number of human crafted input-output training pairs with human produced drawings. You can't just ask ChatGPT to generate every combination because all it'll produce is garbage that gets a lot worse the further from a pelican riding a bicycle. It might work at first for the pelican and a few other animals/transport combination but what does it even mean for a man o' war riding a pyrosome? I asked every model I have access to generate an SVG for a "man o' war riding a pyrosome" and not a single one managed to draw anything resembling a pyrosome. Most couldn't even produce something resembling a man o' war except as a generic ellipsoid-shaped jellyfish with a few tenticles. Expand that to every weird noun-noun combination and it's just not practical to train even a tiny fraction of them.

diggan 1y ago

> I would be so entertained if I found out an AI lab had wasted their time cheating on my dumb benchmark! I don't think it's necessarily "cheating", it just happens as they're discovering and ingesting large ranges of content. A problem of public content, it's bound to be included sooner or later, directly or indirectly. Nice to hear you're doing some sort of contingency though, and looking forward to the inevitable blog post announcing the change to a different bird and vehicle :)

svachalek 1y ago
The thing is most of the discussion about it is embarrassingly bad SVGs so training on them would actually hurt their performance.
- JSR_FDED 1y ago
  Regrettably AI is still better at SVG than I am

deleted 1y ago

[deleted]

reissbaker 1y ago

I doubt they'd cheat that obviously... But "SVG of X" has become common enough that I suspect most frontier labs train on it, especially since the models are multimodal now anyway. Not that I mind; I want models to be good at generating SVG! Makes icons much simpler.

fragmede 1y ago

But how would you know it's from what you would consider cheating as opposed to pelicans on bicycles existing in the latest training data? Obviously your blog gets fed into the training set for GPT-6, as well as everyone else talking about your test, so how would the comparison to a secret X riding a Y tell you if an AI lab is cheating as opposed to merely there being more examples in the training data?

simonw 1y ago
Mainly because if they train on the pelican on bicycle SVGs from my blog they are going to get some very weird looking pelicans riding some terrible looking bicycles.
- fragmede 1y ago
  It's not that I claiming they're training on SVG pelicans on bicycles from your blog, it's that thanks to your popularity, there are simply now more pictures of pelicans on bicycles floating around on the Internet and thus ChatGPT's training data. Eg https://www.reddit.com/r/ColoredPencils/comments/1l9l4fq/pelican_riding_bicycle/ https://www.reddit.com/r/ColoredPencils/comments/1l9l4fq/pel... How would you determine that improvements to SVG pelicans on bicycles (and not your secret X on Ys) are from an OpenAI employee cheating your benchmark vs being an improvement on pelicans on bicycles thanks to that picture from Reddit and everywhere elsewhere in the training data?
  - simonw 1y ago
    See comment here: https://news.ycombinator.com/item?id=45454269 https://news.ycombinator.com/item?id=45454269

jgalt212 1y ago

Your benchmark may or may not be dumb, but it is definitely widely followed. So much so this is what Bing AI has to say on the matter. > Absolutely — the “pelican riding a bicycle” SVG test is a quirky but clever benchmark created by Simon Willison to evaluate how well different large language models (LLMs) can generate SVG (Scalable Vector Graphics) images from a prompt that’s both unusual and unlikely to be in their training data.