3 ms·

this maybe? https://liveswebench.ai/ https://liveswebench.ai/

• tptacek 1y ago
  I don't think you really want to boil this down to a number; there's a whole lot of feature and workflow differences to capture: * BYO model or not * CLI, UI, VSC-plugin or web * async/sync * MCP support * context size * indexed or live grep-style search There's probably like 10 more.
• wiradikusuma 1y ago
  Never heard SWE-agent until now, and seems to beat Aider (the tool I use) consistently. Does anyone know if it's GitHub-only or can it be used as a CLI (i.e., Aider replacement)?
  • mdaniel 1y ago
    Heh, the irony doesn't escape me https://github.com/SWE-agent/SWE-agent/tree/v1.1.0/.cursor/rules https://github.com/SWE-agent/SWE-agent/tree/v1.1.0/.cursor/r... Anyway https://github.com/SWE-agent/SWE-agent/blob/v1.1.0/docs/usage/cli.md https://github.com/SWE-agent/SWE-agent/blob/v1.1.0/docs/usag... and https://github.com/SWE-agent/SWE-agent/blob/v1.1.0/docs/config/tools.md https://github.com/SWE-agent/SWE-agent/blob/v1.1.0/docs/conf...
• EgoIncarnate 1y ago
  I don't think it's being kept up to date. I believe for the IDEs, it requires manual testing to get the numbers. Since things change so quickly, it's mostly just a historical artifact. Hopefully some future version is automated.