4 ms·

I feel like that at this point, we need an AI agent to compare AI agent frameworks. And benchmarks. Well thought out, structured, non-cherry-picked benchmarks

I feel like that at this point, we need an AI agent to compare AI agent frameworks. And benchmarks. Well thought out, structured, non-cherry-picked benchmarks to highlight which framework does well in what area.

yaronsc 2y ago
Benchmarks are WIP. We're thinking about durability, task latency, agent throughput. What else would you like to see?
- namnnumbr 2y ago
  Pass^k and not Pass@k (see https://www.philschmid.de/agents-pass-at-k-pass-power-k https://www.philschmid.de/agents-pass-at-k-pass-power-k). Would be a great twofer to see the code used to run the benchmarks as examples.
  - yaronsc 2y ago
    Will take a look, thanks!