8 ms·
Using LLMs for coding complex projects at scale over a long time is really challenging! This is partly because defining requirements alone is much more challeng
by causal 1y ago
Using LLMs for coding complex projects at scale over a long time is really challenging! This is partly because defining requirements alone is much more challenging than most people want to believe. LLMs accelerate any move in the wrong direction.
- dexwiz 1y agoMy analogy is LLMs are a gas pedal. Makes you go fast, but you still have to know when to turn.
- sreekanth850 1y agoTrue
- sreekanth850 1y agoOne should know theend to end design and architecture. Should stop llm when adding complex fancy things.
- SteveJS 1y agoHaving the llm write the spec/workunit from a conversation works well. Exploring a problem space with a (good) coding agent is fantastic. However for complex projects IMO one must read what was written by the llm … every actual word. When it ‘got away’ from me, in each case I left something in the llm written markdown that I should have removed. 99% “I can ask for that later” and 1% “that’s a good idea i hadn’t considered” might be the right ratio when reading an llm generated plan/spec/workunit. Breaking work into single context passes … 50-60k tokens in sonnet 4.5 has had typically fantastic results for me. My side project is using lean 4 and a carelessly left in ‘validate’ rather than ‘verify’ lead down a hilariously complicated path equivalent to matching an output against a known string. I recovered, but it wasn’t obvious to me that was happening. I however would not be able to write lean proofs myself, so diagnosing the problem and fixing it is a small price to be able to mechanically verify part of my software is correct.