3 ms·
This article makes points that are valid in general but not apposite to code generation with agents. It is indeed difficult to verify a piece of code that is e
by tptacek 1y ago
This article makes points that are valid in general but not apposite to code generation with agents.
It is indeed difficult to verify a piece of code that is either going to ship as-is, or with the specific modifications your verification identifies. In cryptography engineering the rule of thumb (never followed in practice) is 10x verification cost to x implementation cost. Verification is hard and expensive.
But qualifying agent-generated code isn't the verification problem, in the same way that validating an eBPF program in the kernel isn't solving the halting problem.
That's because the agentic scenario gives us an additional outcome: we can allow the code as-is, we can make modifications to the code, or we can throw out the code and re-prompt --- discarding (many) probably-valid programs in a search for the subset of programs that are easy to validate.
In practice, most of what people generate is boring and easy to validate: you know within a couple minutes whether it's the right shape, whether anything sticks out the wrong way, the way a veteran chess player can quickly pattern-match a whole chessboard. When it isn't boring, you read carefully (and expensively), or you just say "no, try again, give me something more boring", or you break your LLM generation into smaller steps that are easier to pattern match and recurse.
What professionals generally don't (I think) do with LLMs is generate large gnarly PRs, all at once, and then do a close-reading of those gnarly PRs. They read PRs that are easy (not gnarly). They reject the gnarly ones, and compensate for gnarliness by approaching the problem at a smaller level of granularity. Or, you know, just write those bits by hand!
- kiitos 1y ago> In practice, most of what people generate is boring and easy to validate: you know within a couple minutes whether it's the right shape, whether anything sticks out the wrong way, the way a veteran chess player can quickly pattern-match a whole chessboard. When it isn't boring, you read carefully (and expensively), or you just say "no, try again, give me something more boring", or you break your LLM generation into smaller steps that are easier to pattern match and recurse. nope! https://sketch.dev/blog/our-first-outage-from-llm-written-code https://sketch.dev/blog/our-first-outage-from-llm-written-co...
- tptacek 1y agoWhat do you think your "nope" here is about? (You and I are probably the only 2 people reading this long-dead thread now, and I'm very familiar with the post you just shared --- I've been using Sketch for several months and following along with their Github repo. Sketch is great.)
- kiitos 1y ago"what people generate is boring and easy to validate" mostly that -^ but i guess also transitively, that (a) you can even identify whether code is boring and simple in the first place; and (b) you're able to tell the LLM to make it "more boring" in any kind of effective way i think these kinds of conversations are revealing a really interesting dichotomy among software professionals, around the (recent and LLM-mandatory) concept of inference, and how it relates to their pre-existing mental model of software some software professionals have a mental model of programming that is just fundamentally incompatible with non-determinism, and for them a tool that's built on inference is delivering way more cost than benefit, and maybe essentially incompatible with how they see what they're doing whereas the LLM maxis never really thought of their software as deterministic or anything like that in the first place, so for them all of the non-determinism from inference is, like, same as it ever was, and delivers way more benefit than cost
- tptacek 1y agoSketch is an extraordinarily solid piece of software and they continue to use Sketch to build Sketch; meanwhile, show me someone who hasn't shipped an outage and I will show you someone who has probably never seriously worked in infra, so I don't really see what the high-order bit of the observation is. I have literally seen this exact pattern --- a refactoring PR that flipped a continue to a break --- cause an incident in human code (which is what all code was up until like 9 months ago).
- kiitos 1y agoi don't know how to explain my points any more clearly than i have, so if you don't grok the intent then i guess we're at an impasse, and i guess also to my points you would be someone in the "llm maxi" camp :)