3 ms·
- Claude, please optimise the project for performance. o Claude goes away for 15 minutes, doesn't profile anything, many code changes. o Announces project now
by nickdothutton 10mo ago
- Claude, please optimise the project for performance.
o Claude goes away for 15 minutes, doesn't profile anything, many code changes.
o Announces project now performs much better, saving 70% CPU.
- Claude, test the performance.
o Performance is 1% _slower_ than previous.
- Claude, can I have a refund for the $15 you just wasted?
o [Claude waffles], "no".
- klysm 10mo agoI’ve always found the hard numbers on performance improvement hilarious. It’s just mimicking what people say on the internet when they get performance gains
- dominotw 10mo ago> It’s just mimicking what people say on the internet when they get performance gains probably read bunch of junior/mid level resumes saying they optimized 90% of company by 80%
- antonvs 10mo agoI can't help but notice that your first two bullets match rather closely the behavior of countless pre-AI university students assigned a project.
- _flux 10mo agoIf you really want to do this, you should probably ask for a plan first and review it.
- jama211 10mo agoWhile you’re making unstructured requests and expecting results, why don’t you ask your barista to make you a “better coffee” with no instructions. Then, when they make a coffee with their own brand of creativity, complain that it tastes worse and you want your money back.
- chillfox 10mo agoI assume a good barista would ask some follow up questions before making the coffee.
- jama211 10mo agoA fair criticism!
- wongarsu 10mo agoBoth "better coffee" and "faster code" are measurable targets. Somewhat vaguely defined, but nobody is stopping the Barista or Claude from asking clarifying questions. If I gave a human this task I would expect them to transform the vague goal into measurable metrics, confirm that the metrics match customer (==my) expectations then measure their improvements on these metrics. This kind of stuff is a major topic for MBAs, but it's really not beyond what you could expect from a programmer or a barista. If I ask you for a better coffee, what you deliver should be better on some metric you can name, otherwise it's simply not better. Bonus points if it's better in a way I care about
- jama211 10mo agoSure, and LLM’s are pretty good at using measurable targets such as using tests to verify their work - if you direct them to do so.
- nickdothutton 10mo agoI was experimenting with Claude Code and requested something more CPU efficient in a very small project, there were a few avenues to explore, I was interested to see what path it would take. It turned out that it seized upon something which wasn't consuming much CPU anyway and was difficult to optimise further. I learned that I'd have to be more explicit in future and direct an analysis phase and probably kick-in a few strategies for performance optimisation which it could then explore. The refund request was an amusement. It was $15 well spent on my own learning.
- 10mo ago
- lukev 10mo agoIf you provide it a benchmark script (or ask it to write one) so it has concrete numbers to go off of, it will do a better job. I'm not saying these things don't hallucinate constantly, they do. But you can steer them toward better output by giving them better input.
- touristtam 10mo agoThe last bit, in my limited experience: > Claude: sorry you have to want until XX:00 as you have run out of credit.
- bongodongobob 10mo agoYou need to let it actually benchmark. They are only as good as the tools you give them.