3 ms·

For the moment the best place to see them all is to browse the tag on my blog - 140 posts now! https://simonwillison.net/tags/pelican-riding-a-bicycle/ https://

by simonw 1mo ago

For the moment the best place to see them all is to browse the tag on my blog - 140 posts now! https://simonwillison.net/tags/pelican-riding-a-bicycle/ https://simonwillison.net/tags/pelican-riding-a-bicycle/ I'm running out of excuses not to build a proper comparison site though. Maybe I'll have Astra do rhat.

bahmboo 1mo ago
Would be great to spend some money (other people's money) on paying some illustrators and artists to do the same for comparison. It could be straight images converted to SVG or direct vector graphics.
- Gander5739 1mo ago
  Doesn't that somewhat miss the point? The models can't see and check their work, for instance.
  - bahmboo 1mo ago
    I'm not clear what you are saying. I am curious about what humans can do in the same context. It's becoming a standard I think for some tasks. E.g. how does a Gen AI stack up against a human in quality and "cost". Perhaps I'm missing what you are asking.
  - spockz 1mo ago
    They can. Ask them to generate SVG , call a renderer and let them inspect the bitmap/png. They can learn about it.
    - Gander5739 1mo ago
      They can, but for this specifc test, they don't. It's a oneshot with no feedback.
- WarmWash 1mo ago
  Technically you would pay programmers to write SVG code with no visual feedback if you want a straight 1:1 comparison.
  - bahmboo 1mo ago
    Kinda. It's the difference between measuring the end product vs how it's made.
xfax 1mo ago
FWIW, I had Astra do it as you suggested: https://pelican-model-gallery.alienfluid.chatgpt.site https://pelican-model-gallery.alienfluid.chatgpt.site I used Light mode and it used up all my limits for the day and had to continue the following day.
- CamperBob2 1mo ago
  This is great! I can see some of the more abstract ones ending up on the wall at SF-MoMA. Missing GLM-5.3, though.
  - carljungslabtek 1mo ago
    The GLM 5.1 one where the pelican is missing because it’s broken would be a great candidate. “A pelican enjoying a sunny ride!” is very “this is not a pipe”
jostylr 1mo ago
You could also have it create a viewer competition selection; show two images without attribution and let the use select which one. Could be cool to see the wisdom of the crowds; could also have price ranges to have subcategories. I bet Astra could do that in a half hour. Loving the pelican silliness. My ChatGPT is over the moon about it. Gonna miss it when it really is done. Though maybe a pelican riding a bike game could become the benchmark in a year.

by simonw 1mo ago

bahmboo 1mo ago

Would be great to spend some money (other people's money) on paying some illustrators and artists to do the same for comparison. It could be straight images converted to SVG or direct vector graphics.

Gander5739 1mo ago
Doesn't that somewhat miss the point? The models can't see and check their work, for instance.
- bahmboo 1mo ago
  I'm not clear what you are saying. I am curious about what humans can do in the same context. It's becoming a standard I think for some tasks. E.g. how does a Gen AI stack up against a human in quality and "cost". Perhaps I'm missing what you are asking.
- spockz 1mo ago
  They can. Ask them to generate SVG , call a renderer and let them inspect the bitmap/png. They can learn about it.
  - Gander5739 1mo ago
    They can, but for this specifc test, they don't. It's a oneshot with no feedback.
WarmWash 1mo ago
Technically you would pay programmers to write SVG code with no visual feedback if you want a straight 1:1 comparison.
- bahmboo 1mo ago
  Kinda. It's the difference between measuring the end product vs how it's made.

xfax 1mo ago

FWIW, I had Astra do it as you suggested: https://pelican-model-gallery.alienfluid.chatgpt.site https://pelican-model-gallery.alienfluid.chatgpt.site I used Light mode and it used up all my limits for the day and had to continue the following day.

CamperBob2 1mo ago
This is great! I can see some of the more abstract ones ending up on the wall at SF-MoMA. Missing GLM-5.3, though.
- carljungslabtek 1mo ago
  The GLM 5.1 one where the pelican is missing because it’s broken would be a great candidate. “A pelican enjoying a sunny ride!” is very “this is not a pipe”

jostylr 1mo ago

You could also have it create a viewer competition selection; show two images without attribution and let the use select which one. Could be cool to see the wisdom of the crowds; could also have price ranges to have subcategories. I bet Astra could do that in a half hour. Loving the pelican silliness. My ChatGPT is over the moon about it. Gonna miss it when it really is done. Though maybe a pelican riding a bike game could become the benchmark in a year.