2 ms·
Is there any point anymore regarding this svg test? I would not be surprised if in the training they're fine tuned for this task too
by hollowturtle 1mo ago
Is there any point anymore regarding this svg test? I would not be surprised if in the training they're fine tuned for this task too
- tintor 1mo agoIt would be very embarrassing for any lab to benchmaxx the pelican on bicycle svg prompt, since it would be very easy to detect it by varying the prompt.
- nojs 1mo agoThe amount of discussion around it means that the test and all the reviews of results, images, approaches etc are implicitly included in training data. It’s not deliberate “benchmaxxing” but things that are discussed a lot online are naturally things that LLMs learn better.
- fc417fc802 1mo agoYou can't benchmaxx spatial awareness without solving the fully general problem (at least I figure).
- BeetleB 1mo agoYou win this thread's prize: https://news.ycombinator.com/item?id=49538333 https://news.ycombinator.com/item?id=49538333
- cheesecakegood 1mo agoIt also works as extremely effective engagement farming, for lack of a better phrase