3 ms·
Free tier ChatGPT (so probably gpt-4o) is quite a bit behind the SOTA, especially compared to agentic workflows (LLM that autonomously perform actions, run test
by danielbln 1y ago
Free tier ChatGPT (so probably gpt-4o) is quite a bit behind the SOTA, especially compared to agentic workflows (LLM that autonomously perform actions, run tests, read/write/edit files, validate output etc.).
Gemini 2.5 pro is a much stronger model, so is Claude 3.7 and presumably GPT4.1 (vis API).
- greenavocado 1y agoGemini 2.5 pro is far ahead of even Claude Chart: https://raw.githubusercontent.com/KCORES/kcores-llm-arena/refs/heads/main/scripts/llm_benchmark_results_normalized.png https://raw.githubusercontent.com/KCORES/kcores-llm-arena/re... Description of the challenges: https://github.com/KCORES/kcores-llm-arena https://github.com/KCORES/kcores-llm-arena