2 ms·

The intensity of competition between models is so intense right now they are definitely benchmaxxing pelican on bike SVGs and Will Smith spaghetti dinner videos

The intensity of competition between models is so intense right now they are definitely benchmaxxing pelican on bike SVGs and Will Smith spaghetti dinner videos.

bayindirh 8mo ago
So, again, when the indicator becomes a target, it stops being a good indicator.
- rcbdev 8mo ago
  Goodhart's law in action.
- kakugawa 8mo ago
  That's how you know you've made it: when your pet benchmark becomes a target.
- JumpCrisscross 8mo ago
  > when the indicator becomes a target, it stops being a good indicator But it's still a fair target. Unless it's hard coded into Gemini 3 DT, for which we have no evidence and decent evidence against, I'd say it's still informative.
yieldcrv 8mo ago
note that this benchmark aside, they've gotten really good at SVGs, I used to rely on the nounproject for icons, and sometimes various libraries, but now coding agents just synthesize an SVG tag in the code and draw all icons.
stared 8mo ago
There was Lenna for digital image compression (https://en.wikipedia.org/wiki/Lenna https://en.wikipedia.org/wiki/Lenna). A pelican on a bike is SFW, inclusive, yet cool. It is not a full benchmark - rather a litmus test.
- oplav 8mo ago
  I never realized Lenna was a Playboy centerfold until years after I first encountered it, which was part of an MP in the data structures class all CS undergrads take at UIUC. There’s also the foreman for video: https://youtube.com/watch?v=0cdM-7_xUXM https://youtube.com/watch?v=0cdM-7_xUXM
thatguysaguy 8mo ago
You can just try other svgs, I got some pretty good ones. (*Disclaimer: I work for Google, but also I have zero idea about what they trained deepthink on)
bonesss 8mo ago
Parallel hypothesis: the intensity of competition between models is so intense that any high-engagement high-relevance web discussion about any LLM/AI generation is gonna hit the self-guided self-reinforced model training and result in de facto benchmaxxing. Which is only to say: if we HN-front-page it, they will come (generate).

The intensity of competition between models is so intense right now they are definitely benchmaxxing pelican on bike SVGs and Will Smith spaghetti dinner videos.

bayindirh 8mo ago

So, again, when the indicator becomes a target, it stops being a good indicator.

rcbdev 8mo ago
Goodhart's law in action.
kakugawa 8mo ago
That's how you know you've made it: when your pet benchmark becomes a target.
JumpCrisscross 8mo ago
> when the indicator becomes a target, it stops being a good indicator But it's still a fair target. Unless it's hard coded into Gemini 3 DT, for which we have no evidence and decent evidence against, I'd say it's still informative.

yieldcrv 8mo ago

note that this benchmark aside, they've gotten really good at SVGs, I used to rely on the nounproject for icons, and sometimes various libraries, but now coding agents just synthesize an SVG tag in the code and draw all icons.

stared 8mo ago

There was Lenna for digital image compression (https://en.wikipedia.org/wiki/Lenna https://en.wikipedia.org/wiki/Lenna). A pelican on a bike is SFW, inclusive, yet cool. It is not a full benchmark - rather a litmus test.

oplav 8mo ago
I never realized Lenna was a Playboy centerfold until years after I first encountered it, which was part of an MP in the data structures class all CS undergrads take at UIUC. There’s also the foreman for video: https://youtube.com/watch?v=0cdM-7_xUXM https://youtube.com/watch?v=0cdM-7_xUXM

thatguysaguy 8mo ago

You can just try other svgs, I got some pretty good ones. (*Disclaimer: I work for Google, but also I have zero idea about what they trained deepthink on)

bonesss 8mo ago

Parallel hypothesis: the intensity of competition between models is so intense that any high-engagement high-relevance web discussion about any LLM/AI generation is gonna hit the self-guided self-reinforced model training and result in de facto benchmaxxing. Which is only to say: if we HN-front-page it, they will come (generate).