36 ms·
For a while now, everytime I see „for AI agents” Im looking for benchmarks with comparable solutions. The only „for AI agent” solution that will gain adoption
by piterrro 3mo ago
For a while now, everytime I see „for AI agents” Im looking for benchmarks with comparable solutions.
The only „for AI agent” solution that will gain adoption is/should be measured by two dimensions: token usage and correctness.
If the solution doesnt use less tokens than generating chart.js code for example - why should I use it?
Same for correctness, if the generated chart spec is correct only 90% of the time - why should I use it?
It’s still early but I think this is the direction we should be thinking about „for AI agents” libraries and projects.
- chenglong-hn 3mo agoYes! correctness, expressiveness, reliability and cost (token + time)
- ebeirne 3mo agoI agree this is the right time to be exploring this more.