3 ms·
Because it excercises thinking about a pelican riding a bike (not common) and then describing that using SVG. It's quite nice imho and seems to scale with the p
by _joel 1y ago
Because it excercises thinking about a pelican riding a bike (not common) and then describing that using SVG. It's quite nice imho and seems to scale with the power of the LLM model. Sure Simon has some actual reasons though.
- imiric 1y agoThe only thing it exercises is the ability of the model to recall its pelican-on-bicycle and other SVG training data.
- Kuinox 1y ago> Because it excercises thinking about a pelican riding a bike (not common) It is extremely common, since it's used on every single LLM to bench it. And there is nothing logic, LLMs are never trained for graphics tasks, they dont see the output of a code.
- _joel 1y agoI mean the real world examples of a pelican riding a bike is not common. It's common in benchmarking LLM's but that's not what I meant.