3 ms·
I think this benchmark could be slightly misleading to assess coding model. But still very good result. Yes, SVG is code, but not in a sense of executable with
by willahmad 10mo ago
I think this benchmark could be slightly misleading to assess coding model. But still very good result.
Yes, SVG is code, but not in a sense of executable with verifiable inputs and outputs.
- jstummbillig 10mo agoI love that we are earnestly contemplating the merits of the pelican benchmark. What a timeline.
- andrepd 10mo agoIt's not even halfway up the list of inane things of the AI hype cycle.
- hdjrudni 10mo agoBut it does have a verifiable output, no more or less than HTML+CSS. Not sure what you mean by "input" -- it's not a function that takes in parameters if that's what you're getting at, but not every app does.