4 ms·

“The actual fundamentals, the things-in-themselves, the theory behind the action” don’t go away, they change. Programmers used to work with punch cards, then a

“The actual fundamentals, the things-in-themselves, the theory behind the action” don’t go away, they change. Programmers used to work with punch cards, then assembly, then low-level languages with odd quirks. Today few developers even think about first-party code size, micro-optimizations, register allocation, etc. LLMs are just another abstraction. A developer with the ideal AI code writer (which we’re not at yet) must still think about idea, design, scope, etc. like a product owner or manager. And these concepts have theory, sometimes even math (e.g. time complexity). EDIT to comment on the article: all abstractions are leaky, but sometimes it rarely matters. Today we do still need to understand code quality and architecture when working with LLMs, or the software will get bad enough that it will affect the company. But maybe not next year. An analogy: stack vs heap, memory allocations, etc. still matter in high-performance software, which isn’t uncommon, but programmers almost never think about register allocation.

lionkor 6mo ago
LLMs are not another abstraction. ALL OTHER LAYERS you named are fully deterministic, understood, debuggable, etc. You cannot be serious.
- antonvs 6mo ago
  LLMs are one of the most general abstractions possible. LLMs are also quite deterministic if you want them to be - generally, their final token selection is deliberately randomized (the model “temperature”). But the word you’re looking for here is probably not actually determinism, it’s probably something closer to predictability. In any case, it’s perfectly possible to ensure that the output of LLMs is fully deterministic, debuggable, understandable, and testable. > You cannot be serious. I don’t think you’re thinking about this clearly.
  - lionkor 6mo ago
    With a sufficiently complex prompt and a sufficiently complex codebase, LLMs consistently fail and make mistakes, "forget" parts of the prompt, etc. There's no comparison to be made between this and, for example, a compiler. It's an incompetent comparison. > I don’t think you’re thinking about this clearly. My literal job is dealing with layers of abstraction. I'm thinking pretty clearly when I tell you that, not only are LLMs a super leaky, terrible abstraction, they are also not comparable to any other layers of abstraction. All other layers of abstraction we use are well understood, predictable (as you put it), and DEBUGGABLE. When claude deletes a fix it did two weeks ago, while trying to fix some unrelated error, do you never stop and think "this is not quite the same as what GCC does"?
    - antonvs 6mo ago
      > With a sufficiently complex prompt and a sufficiently complex codebase ... With a sufficiently complex specification of a failure mode, you can find problems with anything. Humans, given sufficiently complex requirements and sufficiently complex codebases, also regularly fail. You're tacitly admitting that LLMs are approaching (if not exceeding) human levels of performance now. We somehow get non-deterministic humans to achieve useful work. In fact, staff provide managers with an abstraction over the work they're responsible for - managers don't know every detail of the systems they're responsible for. There are effective ways to use LLMs. I recommend using those, not using overly complex prompts, and not letting LLMs freely make changes to large code bases. Just as compilers only compile one source file at a time, LLMs work best if you scope their attention. Same goes for humans, in fact. > There's no comparison to be made between this and, for example, a compiler. A simple comparison is that both can generate useful code. You need to be more precise about the issues you're trying to identify. Anyway, the comparison to compilers isn't really the point. It's undeniable that LLMs are an abstraction themselves, and that they can generate new abstractions. Saying that they're "not another abstraction" is just definitionally wrong. Sure, they're not the same kind of abstraction as a traditional compiler. They require new ways of working, but actually not that new, as the manager example I gave suggests. > When claude deletes a fix it did two weeks ago, while trying to fix some unrelated error, do you never stop and think "this is not quite the same as what GCC does"? I never made the mistake of thinking LLMs were the same as GCC in the first place. And once again, I've seen human developers do exactly what you just described. That's why we review code. All the arguments you're making are essentially also arguments that humans shouldn't be involved in software development either.
  - Terr_ 6mo ago
    > LLMs are also quite deterministic if you want them to be In the shallow sense that any PRNG is deterministic if you set the seed and if you control triggering order. However that's not usually the situation/scope people are talking about.
    - antonvs 6mo ago
      I was just pointing out, in part, that the non-determinism is a choice, but I probably would have needed to go down a whole rabbit hole about exploration of search spaces etc. My broader point is that it's not really the non-determinism that's an issue. What the other commenter seems to be looking for is something along the lines of repeatable correctness, where correctness is generally a requirement that the model doesn't have full access to. The non-determinism is an implementation detail here.
  - vrighter 6mo ago
    because it is not actually trying to prove anything about its outputs, setting the temperature to zero will just ensure it always makes the same mistake when "compiling" english into code. A compiler simply always preserves semantics of its input. Even when randomness is used (ex. possibly during register allocation).
- ai_critic 6mo ago
  Counter-point: most developers have no idea or eagerness to actually do that debugging, so it doesn't really matter.
  - lionkor 6mo ago
    It DOES matter, because the claim that LLMs are a layer of abstraction implies that it's somehow more than a random word generator. It does a great job at generating words in the right order, and often, given enough time, datacenter resources, money, and training, they can produce code that runs and does things as expected. However, there is absolutely nothing stopping an LLM from "deciding" tomorrow that a fix it built a week ago is no longer real, because not only has that fix left its context, but also the bug was not obvious.
    - ai_critic 6mo ago
      > However, there is absolutely nothing stopping an LLM from "deciding" tomorrow that a fix it built a week ago is no longer real, because not only has that fix left its context, but also the bug was not obvious. Yeah, and we've never had deterministic tools like GCC suddenly fuck up commonly-relied-on undefined behavior between releases. Sure. I get what you're saying, but again, to the vast majority of devs, none of that shit matters. Whether that's a good thing or a bad thing is a different discussion.
      - lionkor 6mo ago
        No, deterministic tools so far have not fucked up completely by themselves. There's always a bug, maybe a fix, and maybe a regression test.
- true_religion 6mo ago
  A non-deterministic layer seems like exactly what would need a competent, professional to ensure a good outcome, so it doesn't follow that LLM usage would depress wage more than high-level languages depressed wages by opening up programming to tens of millions of people who could never grok assembly.
  - OkayPhysicist 6mo ago
    > high-level languages depressed wages by opening up programming to tens of millions of people who could never grok assembly. While I agree with your thesis, I don't think your example actually happened. From my experience teaching CS, the fundamental skill when it comes to programming a computer is algorithmic reasoning, that is, the ability to split a task into subtasks until you reach base tools that you exist in your toolbox. Whether those base tools are MOV or document.getElementById() is largely immaterial, IMO. Obviously it's quicker, and easier, for us to not have to drill all the way down to assembly all the time, but if you have a firm grasp of algorithmic reasoning, you are capable of it. Algorithmic reasoning is a valuable skill across the economy, one of the most valuable, in fact, right after "risk evaluation" and "having a lot of money", but it's difficult to teach. For some people it seems to fit right into their mental model, and it's simply a "duh". For most, though, it either takes a ton of grinding their head against it, or more often they simply bounce off and decide to do something else. I think that is responsible for the relatively limited supply of developers, which combined with software being wildly profitable because of the whole "copies are free to make" scaling created a shortage, which drove up wages. They drove up wages so much that approximately everyone who has any aptitude for algorithmic reasoning is now funneled into software development. I think that, more than anything, contributed to the explosive growth of the number of developers. Adults may be hesitant to jump industries, but subsequent generations flooded in.
vrighter 6mo ago
all abstractions are leaky, but you can always rewrite them in terms of a lower level and preserve semantics 100%. It'll be more verbose, it'll look ugly, it's not convenient to work with, but that's what compilers do. They lower the abstraction level, and emit assembly or machine code. Which could also be theoretically done by a human with the same level of reliability, given infinite time. The abstraction is deterministic. You can know exactly what it is abstracting. Not so with LLMs. I can give you an english prompt, and several different LLMs the same prompt and you would all understand it differently. There isn't a way to move between abstraction levels.
- armchairhacker 6mo ago
  LLMs can be deterministic, only the current implementations aren't. I agree that current LLMs are a bad abstraction, and any non-superintelligent AI is worse than a mechanical abstraction in important ways (it's much more complicated, which makes it harder to debug, impossible to prove correct, creates unnecessary coupling, etc.) But it's still technically an abstraction. A project's source (or part of it, e.g. a single function) could be a sequence of prompts, and even a non-deterministic LLM, if it's good enough, will always output correct code.
  - vrighter 6mo ago
    If I give someone a piece of code, they can prove things about the code, even if it is at a higher level of abstraction. A prompt isn't an abstraction. Say you evaluate an llm completely deterministically. Fixed seed for the sampling, or maybe zero temperature. You give the prompt to an engineer and ask them what it does. He can't really say for sure. 'Try "compiling it". It doesn't work? Well try adjusting your prompt a bit. How? I have no idea; try renaming some variable to shirley or something see what pops out.'

lionkor 6mo ago

LLMs are not another abstraction. ALL OTHER LAYERS you named are fully deterministic, understood, debuggable, etc. You cannot be serious.

antonvs 6mo ago
LLMs are one of the most general abstractions possible. LLMs are also quite deterministic if you want them to be - generally, their final token selection is deliberately randomized (the model “temperature”). But the word you’re looking for here is probably not actually determinism, it’s probably something closer to predictability. In any case, it’s perfectly possible to ensure that the output of LLMs is fully deterministic, debuggable, understandable, and testable. > You cannot be serious. I don’t think you’re thinking about this clearly.
- lionkor 6mo ago
  With a sufficiently complex prompt and a sufficiently complex codebase, LLMs consistently fail and make mistakes, "forget" parts of the prompt, etc. There's no comparison to be made between this and, for example, a compiler. It's an incompetent comparison. > I don’t think you’re thinking about this clearly. My literal job is dealing with layers of abstraction. I'm thinking pretty clearly when I tell you that, not only are LLMs a super leaky, terrible abstraction, they are also not comparable to any other layers of abstraction. All other layers of abstraction we use are well understood, predictable (as you put it), and DEBUGGABLE. When claude deletes a fix it did two weeks ago, while trying to fix some unrelated error, do you never stop and think "this is not quite the same as what GCC does"?
  - antonvs 6mo ago
    > With a sufficiently complex prompt and a sufficiently complex codebase ... With a sufficiently complex specification of a failure mode, you can find problems with anything. Humans, given sufficiently complex requirements and sufficiently complex codebases, also regularly fail. You're tacitly admitting that LLMs are approaching (if not exceeding) human levels of performance now. We somehow get non-deterministic humans to achieve useful work. In fact, staff provide managers with an abstraction over the work they're responsible for - managers don't know every detail of the systems they're responsible for. There are effective ways to use LLMs. I recommend using those, not using overly complex prompts, and not letting LLMs freely make changes to large code bases. Just as compilers only compile one source file at a time, LLMs work best if you scope their attention. Same goes for humans, in fact. > There's no comparison to be made between this and, for example, a compiler. A simple comparison is that both can generate useful code. You need to be more precise about the issues you're trying to identify. Anyway, the comparison to compilers isn't really the point. It's undeniable that LLMs are an abstraction themselves, and that they can generate new abstractions. Saying that they're "not another abstraction" is just definitionally wrong. Sure, they're not the same kind of abstraction as a traditional compiler. They require new ways of working, but actually not that new, as the manager example I gave suggests. > When claude deletes a fix it did two weeks ago, while trying to fix some unrelated error, do you never stop and think "this is not quite the same as what GCC does"? I never made the mistake of thinking LLMs were the same as GCC in the first place. And once again, I've seen human developers do exactly what you just described. That's why we review code. All the arguments you're making are essentially also arguments that humans shouldn't be involved in software development either.
- Terr_ 6mo ago
  > LLMs are also quite deterministic if you want them to be In the shallow sense that any PRNG is deterministic if you set the seed and if you control triggering order. However that's not usually the situation/scope people are talking about.
  - antonvs 6mo ago
    I was just pointing out, in part, that the non-determinism is a choice, but I probably would have needed to go down a whole rabbit hole about exploration of search spaces etc. My broader point is that it's not really the non-determinism that's an issue. What the other commenter seems to be looking for is something along the lines of repeatable correctness, where correctness is generally a requirement that the model doesn't have full access to. The non-determinism is an implementation detail here.
- vrighter 6mo ago
  because it is not actually trying to prove anything about its outputs, setting the temperature to zero will just ensure it always makes the same mistake when "compiling" english into code. A compiler simply always preserves semantics of its input. Even when randomness is used (ex. possibly during register allocation).
ai_critic 6mo ago
Counter-point: most developers have no idea or eagerness to actually do that debugging, so it doesn't really matter.
- lionkor 6mo ago
  It DOES matter, because the claim that LLMs are a layer of abstraction implies that it's somehow more than a random word generator. It does a great job at generating words in the right order, and often, given enough time, datacenter resources, money, and training, they can produce code that runs and does things as expected. However, there is absolutely nothing stopping an LLM from "deciding" tomorrow that a fix it built a week ago is no longer real, because not only has that fix left its context, but also the bug was not obvious.
  - ai_critic 6mo ago
    > However, there is absolutely nothing stopping an LLM from "deciding" tomorrow that a fix it built a week ago is no longer real, because not only has that fix left its context, but also the bug was not obvious. Yeah, and we've never had deterministic tools like GCC suddenly fuck up commonly-relied-on undefined behavior between releases. Sure. I get what you're saying, but again, to the vast majority of devs, none of that shit matters. Whether that's a good thing or a bad thing is a different discussion.
    - lionkor 6mo ago
      No, deterministic tools so far have not fucked up completely by themselves. There's always a bug, maybe a fix, and maybe a regression test.
true_religion 6mo ago
A non-deterministic layer seems like exactly what would need a competent, professional to ensure a good outcome, so it doesn't follow that LLM usage would depress wage more than high-level languages depressed wages by opening up programming to tens of millions of people who could never grok assembly.
- OkayPhysicist 6mo ago
  > high-level languages depressed wages by opening up programming to tens of millions of people who could never grok assembly. While I agree with your thesis, I don't think your example actually happened. From my experience teaching CS, the fundamental skill when it comes to programming a computer is algorithmic reasoning, that is, the ability to split a task into subtasks until you reach base tools that you exist in your toolbox. Whether those base tools are MOV or document.getElementById() is largely immaterial, IMO. Obviously it's quicker, and easier, for us to not have to drill all the way down to assembly all the time, but if you have a firm grasp of algorithmic reasoning, you are capable of it. Algorithmic reasoning is a valuable skill across the economy, one of the most valuable, in fact, right after "risk evaluation" and "having a lot of money", but it's difficult to teach. For some people it seems to fit right into their mental model, and it's simply a "duh". For most, though, it either takes a ton of grinding their head against it, or more often they simply bounce off and decide to do something else. I think that is responsible for the relatively limited supply of developers, which combined with software being wildly profitable because of the whole "copies are free to make" scaling created a shortage, which drove up wages. They drove up wages so much that approximately everyone who has any aptitude for algorithmic reasoning is now funneled into software development. I think that, more than anything, contributed to the explosive growth of the number of developers. Adults may be hesitant to jump industries, but subsequent generations flooded in.

vrighter 6mo ago

all abstractions are leaky, but you can always rewrite them in terms of a lower level and preserve semantics 100%. It'll be more verbose, it'll look ugly, it's not convenient to work with, but that's what compilers do. They lower the abstraction level, and emit assembly or machine code. Which could also be theoretically done by a human with the same level of reliability, given infinite time. The abstraction is deterministic. You can know exactly what it is abstracting. Not so with LLMs. I can give you an english prompt, and several different LLMs the same prompt and you would all understand it differently. There isn't a way to move between abstraction levels.

armchairhacker 6mo ago
LLMs can be deterministic, only the current implementations aren't. I agree that current LLMs are a bad abstraction, and any non-superintelligent AI is worse than a mechanical abstraction in important ways (it's much more complicated, which makes it harder to debug, impossible to prove correct, creates unnecessary coupling, etc.) But it's still technically an abstraction. A project's source (or part of it, e.g. a single function) could be a sequence of prompts, and even a non-deterministic LLM, if it's good enough, will always output correct code.
- vrighter 6mo ago
  If I give someone a piece of code, they can prove things about the code, even if it is at a higher level of abstraction. A prompt isn't an abstraction. Say you evaluate an llm completely deterministically. Fixed seed for the sampling, or maybe zero temperature. You give the prompt to an engineer and ask them what it does. He can't really say for sure. 'Try "compiling it". It doesn't work? Well try adjusting your prompt a bit. How? I have no idea; try renaming some variable to shirley or something see what pops out.'