3 ms·

Here are a few examples you might find credible: - pola-rs/polars: https://github.com/pola-rs/polars/pull/26823 https://github.com/pola-rs/polars/pull/26823 -

by simonw 2mo ago

Here are a few examples you might find credible: - pola-rs/polars: https://github.com/pola-rs/polars/pull/26823 https://github.com/pola-rs/polars/pull/26823 - ~2.68x median speedup of primitive-to-boolean casting credited to Claude Opus 4.6 - pydantic/monty: https://github.com/pydantic/monty/pull/643 https://github.com/pydantic/monty/pull/643 - ~53x speedup (488ms down to 9.2ms) of bytes substring search generated with Claude Code - numpy/numpy: https://github.com/numpy/numpy/pull/31573 https://github.com/numpy/numpy/pull/31573 - ~21x speedup of Python datetime → datetime64 conversion, with Claude Code used for profiling and implementing the performance improvements - apache/datafusion: https://github.com/apache/datafusion/pull/21182 https://github.com/apache/datafusion/pull/21182 - up to 49x speedup of LIMIT queries by eliminating unnecessary sorts and pushing limits into file scans, generated with Claude Code - apache/datafusion: https://github.com/apache/datafusion/pull/21651 https://github.com/apache/datafusion/pull/21651 - ~4.39x speedup of ClickBench Q6 by resolving MIN/MAX directly from Parquet metadata instead of scanning columns, generated with Claude Code - pola-rs/polars: https://github.com/pola-rs/polars/pull/27958 https://github.com/pola-rs/polars/pull/27958 - ~3.2x speedup of Int8 Series sum, developed with assistance from Claude Fable and Claude Opus 4.8 - numpy/numpy: https://github.com/numpy/numpy/pull/31274 https://github.com/numpy/numpy/pull/31274 - up to ~1.44x speedup of common small NumPy reductions, with the fast path written by Claude Code and manually refined

vatsachak 2mo ago
That's a fair use case for them though. LLMs are superhuman at short term performance engineering/debugging/testing. But even that comes with the caveat that these commits come from talented coders using LLMs as a grad student. I think that OP is against the claim that "agent in a loop beats a talented human at long term coding tasks". Because if it was true then open source projects such as GIMP could basically be as feature heavy as Photoshop overnight.
- simonw 2mo ago
  I, too, will reject the idea that "agent in a loop beats a talented human at long term coding tasks". LLMs amplify existing expertise. Give them to experts and you can get fantastic results. Give them to amateurs and you might get the occasional impressive demo, but you're not going to get anything that a responsible software team would commit to maintaining in the long term.

by simonw 2mo ago

vatsachak 2mo ago

That's a fair use case for them though. LLMs are superhuman at short term performance engineering/debugging/testing. But even that comes with the caveat that these commits come from talented coders using LLMs as a grad student. I think that OP is against the claim that "agent in a loop beats a talented human at long term coding tasks". Because if it was true then open source projects such as GIMP could basically be as feature heavy as Photoshop overnight.

simonw 2mo ago
I, too, will reject the idea that "agent in a loop beats a talented human at long term coding tasks". LLMs amplify existing expertise. Give them to experts and you can get fantastic results. Give them to amateurs and you might get the occasional impressive demo, but you're not going to get anything that a responsible software team would commit to maintaining in the long term.