5 ms·

The "people" in your hypothetical story have been wrong the whole time. The correct attitude is: When AI can complete lines, you still have to read and underst

The "people" in your hypothetical story have been wrong the whole time. The correct attitude is: When AI can complete lines, you still have to read and understand the code. When AI can complete whole functions, you still have to read and understand the code. When AI can complete features and tickets, you still have to read and understand the code.

jstummbillig 5mo ago
Not at all. Code is not important, intent is. The leader of a product/company does not have to read code. It doesn't matter if it is generated by humans or non-humans. It simply needs to be correct enough to be usable and then steerable towards better outcomes. Understanding of code never existed from the business perspective.
- throwaway173738 5mo ago
  It does in safety critical industries. You can get grilled by regulators about your source code. And lawyers will use it as evidence in court.
- raincole 5mo ago
  > The leader of a product/company does not have to read code. Yeah, because they believe (sometimes wrongly) their subordinates read it. > Understanding of code never existed from the business perspective. It does, it's called organizational wisdom and domain knowledge, because you need those witty names to sell books to aspiring managers.
- contagiousflow 5mo ago
  Can you think of a good way to encode intent into a system?
- roncesvalles 5mo ago
  The code codifies the intent and is the long-term source of truth for how your business actually operates. >The leader of a product/company does not have to read code. That's because he's paid a bunch of people 300k to read it and make sure it aligns with the company's objectives and interests. Part of the reason why devs are paid so much is because they're literal business administrators for some narrow slice of the company's operations. The devs are the leaders that you're referring to. Even in multi-hundred-billion-dollar companies there are so many mission critical things that are owned by just 2 SWEs.
herdcall 5mo ago
I'm no longer sure you have to, actually. I mean, we do trust the assembly that compilers produce without having to read it, don't we? We're rapidly getting to that stage with LLMs, IMO.
- bigfishrunning 5mo ago
  The assembly is a deterministic transform of the input logic, and if it doesn't match then it's a bug in the compiler. If an LLM-based code generator doesn't match what you asked for, that's OK, just pull the slot-machine handle again. that's the difference.
  - estearum 5mo ago
    For now, but obviously they're becoming (effectively) more and more similar to the former every day.
    - tuetuopay 5mo ago
      They’re not, and will never be in their current form and architecture. Compilers are mechanical and engineered to produce a correct output. A compiler emitting incorrect machine code is exceedingly rare, and considered a bug. They have heuristics and probabilities in them, but those are to pick between a set of known-good outputs. An AI is a bag of weights outputting a probability of the most plausible token that follows [1]. It is inherently probabilistic in nature and its output is organic (by design, they’re designed to mimic human speech), as opposed to mechanical like a compiler. A compiler follows hard rules. An AI does its best. And to be fair, AIs are no better than human in this regard: humans are pretty bad at generating correct code without mechanical tools to keep them in line (compilers, linters, formatters). It’s not a wonder we use the same tools to keep LLM output in line as we do humans. (And, to be fair, LLMs are better than humans at oneshotting valid code). [1]: to those that tell me this vision of an LLM is outdated: nope. The heavy lifting is done in the probability generation. Debates about understanding are not relevant here, and the net output of an LLM is a probability vector over raw tokens. This basic description can be contrasted to a compiler whose output is a glorified Jinja template.
brightball 5mo ago
I heard a talk from a VP at NVIDIA a couple of months ago and he echoed this. Essentially their policy is "you are still fully responsible for the code you ship, whether AI helps with it or not"
- senordevnyc 5mo ago
  I think being responsible for the code is a better framing. I run a saas and I don’t always review all the code, but this thing supports my family, so I am acutely aware that I’m responsible for what it does. My customers aren’t going to let me blame the agent for fucking up their workflows. But that still doesn’t mean I review all the code. I tend to review defensively, based on the potential for harm if this piece of code is broken. And I rely a lot on tests, static analysis, canaries, analytics, health checks, etc. to reduce risk for when I’m wrong. So far it’s working.
- rwmj 5mo ago
  While also no doubt telling C-level that the AI can write code completely automatically. The developers in this scenario are there to absorb the blame when things go wrong. "Human crumple-zones" to protect the company.
- 9dev 5mo ago
  Does anyone out there working with AI coding agents not have this policy?
  - AppleBananaPie 5mo ago
    Not to be pedantic and apologies if this is what you meant but my experience has been enforcement of that policy being the problem. Will be interesting to see if / when enforcement happens given management is currently being pushed to encourage AI use
  - CivBase 5mo ago
    Not at PocketOS. It's hard to believe they're the only ones.
  - 5mo ago
globnomulous 5mo ago
Precisely. And this is why all the MCP servers that people at my company are writing aren't worth using: their apparent goal is automate as much as possible. They're encouraging people not to pay attention. This results in bad code, bad tests, and bugs.
ventana 5mo ago
> you still have to read and understand the code Which is a very similar approach to any serious code. If you just hired a very clever, enormously knowledgeable intern, and they wrote a bunch of code for you overnight, you would probably review it. Yes, in some cases, either hobby projects or throwaway code, you could just take it and use it as is, and I surely do, for the code no one cares about. But at work, I would rather review it.

jstummbillig 5mo ago

Not at all. Code is not important, intent is. The leader of a product/company does not have to read code. It doesn't matter if it is generated by humans or non-humans. It simply needs to be correct enough to be usable and then steerable towards better outcomes. Understanding of code never existed from the business perspective.

throwaway173738 5mo ago
It does in safety critical industries. You can get grilled by regulators about your source code. And lawyers will use it as evidence in court.
raincole 5mo ago
> The leader of a product/company does not have to read code. Yeah, because they believe (sometimes wrongly) their subordinates read it. > Understanding of code never existed from the business perspective. It does, it's called organizational wisdom and domain knowledge, because you need those witty names to sell books to aspiring managers.
contagiousflow 5mo ago
Can you think of a good way to encode intent into a system?
roncesvalles 5mo ago
The code codifies the intent and is the long-term source of truth for how your business actually operates. >The leader of a product/company does not have to read code. That's because he's paid a bunch of people 300k to read it and make sure it aligns with the company's objectives and interests. Part of the reason why devs are paid so much is because they're literal business administrators for some narrow slice of the company's operations. The devs are the leaders that you're referring to. Even in multi-hundred-billion-dollar companies there are so many mission critical things that are owned by just 2 SWEs.

herdcall 5mo ago

I'm no longer sure you have to, actually. I mean, we do trust the assembly that compilers produce without having to read it, don't we? We're rapidly getting to that stage with LLMs, IMO.

bigfishrunning 5mo ago
The assembly is a deterministic transform of the input logic, and if it doesn't match then it's a bug in the compiler. If an LLM-based code generator doesn't match what you asked for, that's OK, just pull the slot-machine handle again. that's the difference.
- estearum 5mo ago
  For now, but obviously they're becoming (effectively) more and more similar to the former every day.
  - tuetuopay 5mo ago
    They’re not, and will never be in their current form and architecture. Compilers are mechanical and engineered to produce a correct output. A compiler emitting incorrect machine code is exceedingly rare, and considered a bug. They have heuristics and probabilities in them, but those are to pick between a set of known-good outputs. An AI is a bag of weights outputting a probability of the most plausible token that follows [1]. It is inherently probabilistic in nature and its output is organic (by design, they’re designed to mimic human speech), as opposed to mechanical like a compiler. A compiler follows hard rules. An AI does its best. And to be fair, AIs are no better than human in this regard: humans are pretty bad at generating correct code without mechanical tools to keep them in line (compilers, linters, formatters). It’s not a wonder we use the same tools to keep LLM output in line as we do humans. (And, to be fair, LLMs are better than humans at oneshotting valid code). [1]: to those that tell me this vision of an LLM is outdated: nope. The heavy lifting is done in the probability generation. Debates about understanding are not relevant here, and the net output of an LLM is a probability vector over raw tokens. This basic description can be contrasted to a compiler whose output is a glorified Jinja template.

brightball 5mo ago

I heard a talk from a VP at NVIDIA a couple of months ago and he echoed this. Essentially their policy is "you are still fully responsible for the code you ship, whether AI helps with it or not"

senordevnyc 5mo ago
I think being responsible for the code is a better framing. I run a saas and I don’t always review all the code, but this thing supports my family, so I am acutely aware that I’m responsible for what it does. My customers aren’t going to let me blame the agent for fucking up their workflows. But that still doesn’t mean I review all the code. I tend to review defensively, based on the potential for harm if this piece of code is broken. And I rely a lot on tests, static analysis, canaries, analytics, health checks, etc. to reduce risk for when I’m wrong. So far it’s working.
rwmj 5mo ago
While also no doubt telling C-level that the AI can write code completely automatically. The developers in this scenario are there to absorb the blame when things go wrong. "Human crumple-zones" to protect the company.
9dev 5mo ago
Does anyone out there working with AI coding agents not have this policy?
- AppleBananaPie 5mo ago
  Not to be pedantic and apologies if this is what you meant but my experience has been enforcement of that policy being the problem. Will be interesting to see if / when enforcement happens given management is currently being pushed to encourage AI use
- CivBase 5mo ago
  Not at PocketOS. It's hard to believe they're the only ones.
- 5mo ago

globnomulous 5mo ago

Precisely. And this is why all the MCP servers that people at my company are writing aren't worth using: their apparent goal is automate as much as possible. They're encouraging people not to pay attention. This results in bad code, bad tests, and bugs.

ventana 5mo ago

> you still have to read and understand the code Which is a very similar approach to any serious code. If you just hired a very clever, enormously knowledgeable intern, and they wrote a bunch of code for you overnight, you would probably review it. Yes, in some cases, either hobby projects or throwaway code, you could just take it and use it as is, and I surely do, for the code no one cares about. But at work, I would rather review it.