3 ms·
Who ensures it followed the specs? The more context an LLM gets, the more likely it will start to ignore instructions. If the LLM runs a context compression,
by bhaak 4mo ago
Who ensures it followed the specs?
The more context an LLM gets, the more likely it will start to ignore instructions.
If the LLM runs a context compression, all bets are off. There's a reason Anthropic upped the context to 1M tokens to reduce the chance of this from happening.
- ceejayoz 4mo ago> Who ensures it followed the specs? I mean, it's the same with building a bridge in the real world, right? Someone has to check the work.
- not_a_bot_4sho 4mo ago> Who ensures it followed the specs? The human. But only if you care about verification.
- bhaak 4mo agoThe human is missing form OP's description. "and it fills in the implementation". No human in sight. You can't call it "engineering" if you don't care about verification.
- ceejayoz 4mo agoIf you build a bridge, the engineers aren't the one doing the welding and crane operation and bolts and digging holes and whatnot. They're the ones checking that work matches the plan.
- montroser 4mo agoCome on, now. The human writes the plan up front, which includes guidance on testing strategy, classes of tests, particular test cases to cover, etc. And just like normal, of course you don't just ship the code without doing manual verification, code review, auditing the test cases, and all the rest.