4 ms·
I was thinking at one point if all these companies just hit a wall in performance and improvements of the underlying technology and all the version updates and
by mns 1y ago
I was thinking at one point if all these companies just hit a wall in performance and improvements of the underlying technology and all the version updates and new "models" presented are them just editing and creating more and more complex system prompts. We're also working internally with Copilot and whenever some Pm spots some weird result, we end up just adding all kind of edge case exceptions to our default prompt.
- Flemlo 1y agoBut that's part of the human based reinforcement learning. It now just happens on a way bigger level because now it's actually worth while to do. That's one of the core beauties of the AI Ara
- antupis 1y agoI think we already hit somekind of performance wall begin of this year. It feels that models are now balancing between rule following and agentic case and general stuff. eg Claude 4 sonet just feels better in Cursor and follows rules very well, and same time it gets equal or worse scores in benchmark against 3.7 Sonet.
- merksittich 1y agoSpeaking of performance wall: The Claude 4 results were added to the Aider LLM Leaderboard [0] yesterday. Opus 4 is clearly below Gemini 2.5 Pro at almost twice the price. Sonnet 4 fares worse than Sonnet 3.7, with the thinking version of Sonnet 4 being somewhat cheaper than its 3.7 counterpart. [0] https://aider.chat/docs/leaderboards/ https://aider.chat/docs/leaderboards/
- smusamashah 1y agoThere might be 4.1 soon to make up for it's shortcomings.