3 ms·
At the bottom it says "Score 98.58", what measure is used for this score? It's kind of horrible, the perspective is all off (legs of the table makes that very o
by embedding-shape 1mo ago
At the bottom it says "Score 98.58", what measure is used for this score? It's kind of horrible, the perspective is all off (legs of the table makes that very obvious), the mouse/hamster has two mouths, a stub for a right paw, looks like left hand holds a melon on a stick or something, and there are pluses in the background for some reason. Not sure it'd call it "close to perfect" which the score seems to want to indicate.
- XCSme 1mo agoYeah, that's confusing, the score is for the entire benchmark, not for SVG generation only. Good point about the mouths, I just noticed, lol Imo, it's still better than most models, I personally like the stylized perspective. You can view here all generations for all models: https://aibenchy.com/showcase/ https://aibenchy.com/showcase/
- XCSme 1mo agoDo you prefer the fable one? It's more "correct" but looks a lot worse in my opinion: https://aibenchy.com/compare/openai-gpt-6-astra-high/google-gemini-3-8-flash-high/anthropic-claude-fable-5-1-high/#showcase=f36ffedba4331387 https://aibenchy.com/compare/openai-gpt-6-astra-high/google-...
- kulahan 1mo agoNo, both of them look awful. I genuinely am unsure if this is a meme you're making that's going over my head...?
- XCSme 1mo agoWell, this was the state of SVG generation until a few months ago: https://aibenchy.com/showcase/?page=2#showcase=67fc6d6c8e4c3aba https://aibenchy.com/showcase/?page=2#showcase=67fc6d6c8e4c3... https://aibenchy.com/showcase/?page=3#showcase=c215b5c915da6cd2 https://aibenchy.com/showcase/?page=3#showcase=c215b5c915da6...
- kulahan 1mo agoAh, okay, that makes a lot more sense! Thanks!
- XCSme 1mo agoI've replaced "Score" there with model ranking, to reduce confusion, thanks for the feedback!
- embedding-shape 1mo agoNow I'm wondering why it's ranked #4 instead, not sure this reduced confusion :P What panel of judges are you using for scoring/ranking this? Seems subjective enough to not be able to be ranked/scored at all
- XCSme 1mo agoYeah, ranking is for entire model, not only SVG generation. Maybe I remove it entirely and add generation date instead. I was thinking to manually grade/rank the SVGs, but I decided against it, as it is indeed subjective. I was thinking it could have at least a simple objective check (hamster doesn't have extra or missing parts, table has 2 sides, and net is in the middle, etc.).
- kzrdude 1mo agoAnd not to mention, the hamster is standing at the wrong end of the table.
- embedding-shape 1mo agoIncredible, I've played table-tennis since I was like 14, and I didn't notice that egregious error yet I noticed all the other small ones! Thanks for pointing that out, should have been very obvious.
- sehugg 1mo agoAnd also the hamster has two mouths
- embedding-shape 1mo agoI think it might be standing at the wrong edge of the table too, worth adding.