4 ms·

I hope it does. These coding benchmarks have often seemed frustratingly out of touch with my experience.

• 3abiton 1y ago
  Because I would argue there is no benchmark to rule them all. It highly depends on individual use cases.
• typpilol 1y ago
  The agentic ones seem better. Typescript is like at 25% last I saw on the models. Python was higher. That seems more accurate than the huge scores the other ones get