4 ms·
This is funny, I actually did a similar experiment just yesterday. Looking for evidence of the same, but with another twist: checking if the models would choos
by bnfcl 2mo ago
This is funny, I actually did a similar experiment just yesterday.
Looking for evidence of the same, but with another twist: checking if the models would choose to create a pelican on a bicycle, if no specific bird or method of transportation was specified.
My version of it: https://www.modelbias.ai/pelican-on-a-bicycle-test https://www.modelbias.ai/pelican-on-a-bicycle-test
- wasabi991011 2mo agoI find your analysis much more convincing than TFA, since it doesn't require a subjective evaluation and is more robust to animal/transport complexity.
- bnfcl 2mo agoThanks! Because I think that models are becoming better at creating SVGs in general. If you look at Claude Fable 5 and Kimi K3 for example. In my tests it did create bicycles the most, but this is just a general bias I believe, as tested here: https://www.modelbias.ai/prompt/transport https://www.modelbias.ai/prompt/transport
- zahlman 2mo agoSimple as they are, there are some really aesthetically pleasing penguins on skateboards in there, including from less capable models. (In fact, I would say the Opus series got progressively worse at it over time.)
- bnfcl 2mo agoAgreed!
- michaelt 2mo agoThe outputs of Qwen3.7 Plus have to be seen to be believed: https://www.modelbias.ai/pelican-on-a-bicycle-test/result/121727 https://www.modelbias.ai/pelican-on-a-bicycle-test/result/12... https://www.modelbias.ai/pelican-on-a-bicycle-test/result/121728 https://www.modelbias.ai/pelican-on-a-bicycle-test/result/12...