3 ms·

> AI does not know whether the code it just added violates some legal requirement to which your product is subject. The rest of the paragraph is a list of simi

> AI does not know whether the code it just added violates some legal requirement to which your product is subject. The rest of the paragraph is a list of similar complaints. All of these can be codified - the new work is the (fascinating!) challenge of working out how and codifying them, then giving the codified checker logic to the agent to run at will - thus taking yourself out of all of those loops. They don’t even need to be deterministic checks - a shell script that wraps a `claude -p` - and maybe fetches some online resources to stuff into the context - can do your agent’s legal check for it. As a result, you’re only being effective if you can engineer (not hack, not vibe) an agent that can produce work your company can use with as little cleanup as possible - and with less cleanup over time. You get job by codifying what you know - what most don’t account for is that there’ll never be a limit to what there is to know, and codifying expertise will be challenging for a long time to come.

lowsong 4mo ago
> They don’t even need to be deterministic checks - a shell script that wraps a `claude -p` - and maybe fetches some online resources to stuff into the context - can do your agent’s legal check for it. No. It can't. If you think that injecting legal text into the context window and appending "make sure the output complies with this law" will solve your problems you have not understood how an LLM works. The kind of checks you are talking about cannot be codified.
- cadamsdotcom 4mo ago
  So we should give up before we even tried?
  - lowsong 4mo ago
    Tried what? Using LLMs to produce software? I'd say the industry has collectively "tried" to do this for a while now and the results have been an unmitigated disaster. Yes, we should "give up" on using LLMs, but certainly not for a lack of "trying".
    - cadamsdotcom 4mo ago
      How’d you get so cynical, friend. Come to the dark side - we have free tokens!

lowsong 4mo ago

> They don’t even need to be deterministic checks - a shell script that wraps a `claude -p` - and maybe fetches some online resources to stuff into the context - can do your agent’s legal check for it. No. It can't. If you think that injecting legal text into the context window and appending "make sure the output complies with this law" will solve your problems you have not understood how an LLM works. The kind of checks you are talking about cannot be codified.

cadamsdotcom 4mo ago
So we should give up before we even tried?
- lowsong 4mo ago
  Tried what? Using LLMs to produce software? I'd say the industry has collectively "tried" to do this for a while now and the results have been an unmitigated disaster. Yes, we should "give up" on using LLMs, but certainly not for a lack of "trying".
  - cadamsdotcom 4mo ago
    How’d you get so cynical, friend. Come to the dark side - we have free tokens!