2 ms·
We need more rigorous benchmarks for SRE tasks, which is much easier said that done. The only other benchmark I've come across is https://sreben.ch/ https://sr
by smithclay 8mo ago
We need more rigorous benchmarks for SRE tasks, which is much easier said that done.
The only other benchmark I've come across is https://sreben.ch/ https://sreben.ch/ ... certainly there must be others by now?
- nyellin 8mo agoWe publish the benchmarks for HolmesGPT (CNCF sandbox project) at https://holmesgpt.dev/development/evaluations/ https://holmesgpt.dev/development/evaluations/