4 ms·

Very few people have the expertise to write efficient assembly code, yet everyone relies on compilers and assemblers to translate high-level code to byte-level

Very few people have the expertise to write efficient assembly code, yet everyone relies on compilers and assemblers to translate high-level code to byte-level machine code. I think same concept is true here. Once coding agents become trivial, few people will know the detail of the programming language and make sure intent is correctly transformed to code, and the majority will focus on different objectives and take LLM programming for granted.

Marazan 9mo ago
Yes, but compilers (in the main), do not have a random number generator to decide what output to produce.
feanaro 9mo ago
No, that's a completely different concept, because we have faultless machines which perfectly and deterministically translate high-level code into byte-level machine code. This is another case of (nearly) perfect abstraction. On the other hand, the whole deal of the LLM is that it does so stochastically and unpredictably.
- theshrike79 9mo ago
  We also have machines that can perfectly and deterministically check written code for correctness. And the stohastic LLM can use those tools to check whether its work was sufficient, if not, it will try again - without human intervention. It will repeat this loop until the deterministic checks pass.
  - gspr 9mo ago
    > We also have machines that can perfectly and deterministically check written code for correctness. Please do provide a single example of this preposterous claim.
    - theshrike79 9mo ago
      It's not like testing code is a new thing. Junit is almost 30 years old today. For functionality: https://en.wikipedia.org/wiki/Unit_testing https://en.wikipedia.org/wiki/Unit_testing With robust enough test suites you can vibe code a HTML5 parser - https://ikyle.me/blog/2025/swift-justhtml-porting-html5-parser-to-swift https://ikyle.me/blog/2025/swift-justhtml-porting-html5-pars... - https://simonwillison.net/2025/Dec/15/porting-justhtml/ https://simonwillison.net/2025/Dec/15/porting-justhtml/ And code correctness: - https://en.wikipedia.org/wiki/Tree-sitter_(parser_generator) https://en.wikipedia.org/wiki/Tree-sitter_(parser_generator) - https://en.wikipedia.org/wiki/Roslyn_(compiler) https://en.wikipedia.org/wiki/Roslyn_(compiler) - https://en.wikipedia.org/wiki/Lint_(software) https://en.wikipedia.org/wiki/Lint_(software) You can make analysers that check for deeply nested code, people calling methods in the wrong order and whatever you want to check. At work we've added multiple Roslyn analysers to our build pipeline to check for invalid/inefficient code, no human will be pinged by a PR until the tests pass. And an LLM can't claim "Job's Done" before the analysers say the code is OK. And you don't need to make one yourself, there are tons you can just pick from: https://en.wikipedia.org/wiki/List_of_tools_for_static_code_analysis https://en.wikipedia.org/wiki/List_of_tools_for_static_code_...
      - gspr 9mo ago
        > It's not like testing code is a new thing. Junit is almost 30 years old today. Unit tests check whether code behaves in specific ways. They certainly are useful to weed out bugs and to ensure that changes don't have unintended side effects. > And code correctness: These are tools to check for syntactic correctness. That is, of course, not what I meant. You're completely off the mark here.
        theshrike79 9mo ago
        What did you mean then if unit tests and syntactic correctness aren't what you're looking for?
        gspr 9mo ago
- Cthulhu_ 9mo ago
  The unpredictable part isn't new - from a project manager's point of view, what's the difference between an LLM and a team of software engineers? Both, from that POV, are a black box. The "how" is not important to them, the details aren't important. What's important is that what they want is made a reality, and that customers can press on a button to add a product to their shopping cart (for example). LLMs mean software developers let go of some control of how something is built, which makes one feel uneasy because a lot of the appeal of software development is control and predictability. But this is the same process that people go through as they go from coder to lead developer or architect or project manager - letting go of control. Some thrive in their new position, having a higher overview of the job, while some really can't handle it.
  - bwestergard 9mo ago
    "But this is the same process that people go through as they go from coder to lead developer or architect or project manager - letting go of control." In those circumstances, it's delegating control. And it's difficult to judge whether the authority you delegated is being misused if you lose touch with how to do the work itself. This comparison shouldn't be pushed too far, but it's not entirely unlike a compiler developer needing to retain the ability to understand machine code instructions.
- endymion-light 9mo ago
  As someone that started off with assembly issues for a large corporation - assembly code may sometimes contain very similiar issues that mroe high-level code those, the perfection of the abstraction is not guaranteed. But yeah, there's currently a wide gap between that and a stochastic LLM.
discreteevent 9mo ago
The difference is that if you write in C you can debug in C. You don't have to debug the assembly. You can write an english wish list for an LLM but you will still have to debug the generated code. To debug it you will need to understand it.
- haspok 9mo ago
  Why would you have to debug generated code? Let the LLM debug it.
  - gspr 9mo ago
    And how do you know it did it right?
lelanthran 9mo ago
> Very few people have the expertise to write efficient assembly code, yet everyone relies on compilers and assemblers to translate high-level code to byte-level machine code. I think same concept is true here. That's a poor analogy which gets repeated in every discussion: compilers are deterministic, LLMs are not.
- haspok 9mo ago
  Why does it matter if LLMs are not deterministic? Who cares? There should be tests covering meaningful functionality, as long as the code passes the tests, ie. the externally observable behaviour is the same, I don't care. (Especially, if many tests can also be autogenerated with the LLM.)
  - lelanthran 9mo ago
    >>> Very few people have the expertise to write efficient assembly code, yet everyone relies on compilers and assemblers to translate high-level code to byte-level machine code. I think same concept is true her >> That's a poor analogy which gets repeated in every discussion: compilers are deterministic, LLMs are not. > Why does it matter if LLMs are not deterministic? Who cares? In the context of this analogy, it matters. If you're not using this analogy, then sure, only the result matters. But when the analogy being used is deterministic, then, yes, it matters. You can't very well claim "We'll compare this non-deterministic process to this other deterministic process that we know works."
- mpyne 9mo ago
  > That's a poor analogy which gets repeated in every discussion: compilers are deterministic, LLMs are not. Compilers are not used directly, they are used by human software developers who are also not deterministic. From the perspective of an organization with a business or service-based mission, they already know how to supervise non-deterministic LLMs because they already know how to supervise non-deterministic human developers.

Marazan 9mo ago

Yes, but compilers (in the main), do not have a random number generator to decide what output to produce.

feanaro 9mo ago

No, that's a completely different concept, because we have faultless machines which perfectly and deterministically translate high-level code into byte-level machine code. This is another case of (nearly) perfect abstraction. On the other hand, the whole deal of the LLM is that it does so stochastically and unpredictably.

theshrike79 9mo ago
We also have machines that can perfectly and deterministically check written code for correctness. And the stohastic LLM can use those tools to check whether its work was sufficient, if not, it will try again - without human intervention. It will repeat this loop until the deterministic checks pass.
- gspr 9mo ago
  > We also have machines that can perfectly and deterministically check written code for correctness. Please do provide a single example of this preposterous claim.
  - theshrike79 9mo ago
    It's not like testing code is a new thing. Junit is almost 30 years old today. For functionality: https://en.wikipedia.org/wiki/Unit_testing https://en.wikipedia.org/wiki/Unit_testing With robust enough test suites you can vibe code a HTML5 parser - https://ikyle.me/blog/2025/swift-justhtml-porting-html5-parser-to-swift https://ikyle.me/blog/2025/swift-justhtml-porting-html5-pars... - https://simonwillison.net/2025/Dec/15/porting-justhtml/ https://simonwillison.net/2025/Dec/15/porting-justhtml/ And code correctness: - https://en.wikipedia.org/wiki/Tree-sitter_(parser_generator) https://en.wikipedia.org/wiki/Tree-sitter_(parser_generator) - https://en.wikipedia.org/wiki/Roslyn_(compiler) https://en.wikipedia.org/wiki/Roslyn_(compiler) - https://en.wikipedia.org/wiki/Lint_(software) https://en.wikipedia.org/wiki/Lint_(software) You can make analysers that check for deeply nested code, people calling methods in the wrong order and whatever you want to check. At work we've added multiple Roslyn analysers to our build pipeline to check for invalid/inefficient code, no human will be pinged by a PR until the tests pass. And an LLM can't claim "Job's Done" before the analysers say the code is OK. And you don't need to make one yourself, there are tons you can just pick from: https://en.wikipedia.org/wiki/List_of_tools_for_static_code_analysis https://en.wikipedia.org/wiki/List_of_tools_for_static_code_...
    - gspr 9mo ago
      > It's not like testing code is a new thing. Junit is almost 30 years old today. Unit tests check whether code behaves in specific ways. They certainly are useful to weed out bugs and to ensure that changes don't have unintended side effects. > And code correctness: These are tools to check for syntactic correctness. That is, of course, not what I meant. You're completely off the mark here.
      - theshrike79 9mo ago
        What did you mean then if unit tests and syntactic correctness aren't what you're looking for?
        gspr 9mo ago
Cthulhu_ 9mo ago
The unpredictable part isn't new - from a project manager's point of view, what's the difference between an LLM and a team of software engineers? Both, from that POV, are a black box. The "how" is not important to them, the details aren't important. What's important is that what they want is made a reality, and that customers can press on a button to add a product to their shopping cart (for example). LLMs mean software developers let go of some control of how something is built, which makes one feel uneasy because a lot of the appeal of software development is control and predictability. But this is the same process that people go through as they go from coder to lead developer or architect or project manager - letting go of control. Some thrive in their new position, having a higher overview of the job, while some really can't handle it.
- bwestergard 9mo ago
  "But this is the same process that people go through as they go from coder to lead developer or architect or project manager - letting go of control." In those circumstances, it's delegating control. And it's difficult to judge whether the authority you delegated is being misused if you lose touch with how to do the work itself. This comparison shouldn't be pushed too far, but it's not entirely unlike a compiler developer needing to retain the ability to understand machine code instructions.
endymion-light 9mo ago
As someone that started off with assembly issues for a large corporation - assembly code may sometimes contain very similiar issues that mroe high-level code those, the perfection of the abstraction is not guaranteed. But yeah, there's currently a wide gap between that and a stochastic LLM.

discreteevent 9mo ago

The difference is that if you write in C you can debug in C. You don't have to debug the assembly. You can write an english wish list for an LLM but you will still have to debug the generated code. To debug it you will need to understand it.

haspok 9mo ago
Why would you have to debug generated code? Let the LLM debug it.
- gspr 9mo ago
  And how do you know it did it right?

lelanthran 9mo ago

> Very few people have the expertise to write efficient assembly code, yet everyone relies on compilers and assemblers to translate high-level code to byte-level machine code. I think same concept is true here. That's a poor analogy which gets repeated in every discussion: compilers are deterministic, LLMs are not.

haspok 9mo ago
Why does it matter if LLMs are not deterministic? Who cares? There should be tests covering meaningful functionality, as long as the code passes the tests, ie. the externally observable behaviour is the same, I don't care. (Especially, if many tests can also be autogenerated with the LLM.)
- lelanthran 9mo ago
  >>> Very few people have the expertise to write efficient assembly code, yet everyone relies on compilers and assemblers to translate high-level code to byte-level machine code. I think same concept is true her >> That's a poor analogy which gets repeated in every discussion: compilers are deterministic, LLMs are not. > Why does it matter if LLMs are not deterministic? Who cares? In the context of this analogy, it matters. If you're not using this analogy, then sure, only the result matters. But when the analogy being used is deterministic, then, yes, it matters. You can't very well claim "We'll compare this non-deterministic process to this other deterministic process that we know works."
mpyne 9mo ago
> That's a poor analogy which gets repeated in every discussion: compilers are deterministic, LLMs are not. Compilers are not used directly, they are used by human software developers who are also not deterministic. From the perspective of an organization with a business or service-based mission, they already know how to supervise non-deterministic LLMs because they already know how to supervise non-deterministic human developers.