5 ms·
It's probably optimized in some way, but if the optimizations degrade performance, let's hope it is reflected in various benchmarks. One alternative hypothesis
by resters 1y ago
It's probably optimized in some way, but if the optimizations degrade performance, let's hope it is reflected in various benchmarks. One alternative hypothesis is that it's the same model, but in the early days they make it think "harder" and run a meta-process to collect training data for reinforcement learning for use on future models.
- SparkyMcUnicorn 1y agoIt's a bit dated now, but it would be cool if people submitted PRs for this one: https://aider.chat/docs/leaderboards/by-release-date.html https://aider.chat/docs/leaderboards/by-release-date.html
- __mharrison__ 1y agoDated? This was updated yesterday https://aider.chat/docs/leaderboards/ https://aider.chat/docs/leaderboards/
- SparkyMcUnicorn 1y agoMy link is to the benchmark results _over time_. The main leaderboard page that you linked to is updated quite frequently, but it doesn't contain multiple benchmarks for the same exact model.