3 ms·

Is it "anti-LLM" or is it anti "here's some code I don't understand that a machine generated for me kthxbye"? Code is not just code, it's also liability and tru

Is it "anti-LLM" or is it anti "here's some code I don't understand that a machine generated for me kthxbye"? Code is not just code, it's also liability and trust.

tossandthrow 2mo ago
You literally spend 99.999% og your time blindly trusting code - and that is only because I assume ymthat you write code yourself. For 99.999% of people, it is literally kthxbye on all code they execute on all their devices.
- deleted 2mo ago
  [deleted]
- EvanAnderson 2mo ago
  I don't think that's accurate. Until recently the code we rely on wasn't blindly trusted. People wrote and QA'd it. We may not have reviewed it personally, but that's one of the functions we outsource to software maintainers or "manufacturers". I'm not "blindly trusting" code on my computing devices. I'm trusting the vendors / maintainers to do their job. Until very recently the norm has been that the vast majority of code had human eyes and hands on it. Edit: The owners of those human eyes and hands had some type of accountability (either reputationally, in the case of free/open-source software, or occupationally, in the case of proprietary software). The LLM has no accountability as to the output it generates. The companies who make the LLMs also seem to have very little accountability, too. We've assumed a "blame the victim" stance when people use LLM-generated output in some inappropriate ways (legal briefs with "hallucinated" citations, articles "written" by LLMs). Whether that's the right location for accountability to be placed isn't for me to say, but that seems to be how it is. I'm not sure that we're applying accountability to LLM-generated code in the same way we are for, say, the LLM-generated legal brief.
  - jadar 2mo ago
    A compiler is not an LLM, and I do not want to equivocate, but there are aspects of similarity. We do not assume people read the bytes of machine code to ensure it’s correct — there could be mistakes. We also write and run automated tests to ensure the code outputted from a compiler and an LLM behaves correctly. At some point, we won’t have to literally read every byte of code that comes out because we have a reasonable assurance that it’s correct.
    - EvanAnderson 2mo ago
      That we don't have to heavily scrutinize the code generated by compilers is a result of the huge amount of human toil that went into the compilers and test suites. Compilers are deterministic mechanisms so tests can be constructed. LLMs, at least as they're currently constructed, aren't deterministic (the whole "temperature" thing). I don't see how to build a mechanistic test for something that has non-deterministic output. It feels a little bit like solving the halting problem. I have no doubt we'll move away from human code review. The idea of large amounts of software edifice being built upon foundations that no human has reviewed or, perhaps even understands, is horrifying to me, though.
    - flohofwoe 2mo ago
      Compilers also usually give you the same output for the same input. And fwiw I do spend quite a lot of time reading compiler output to check that it's not doing something stupid or unexpected (usually as part of optimization work). Also this sort of 'technological whataboutism' really isn't helpful, compilers are entirely different from LLMs. I agree that it doesn't make much sense to read or review LLM output in detail, but I also don't plan to use LLM output for anything important or mission critical. That would be irresponsible.
      - AshamedCaptain 2mo ago
        In addition, when we usually say that triggering undefined behavior on C can start a game of Tetris or format your hard disk we're usually joking (or at least exaggerating), i.e. the most common failure conditions of compilers are really limited in scope, and very likely caught by whatever testing mechanism you use for the software. No such limits for LLMs where losing all your files is about par for the course for everyone who uses them regularly.
  - tossandthrow 2mo ago
    > Until very recently the norm has been that the vast majority of code had human eyes and hands on it. What quality does it have that humans had eyes on code? Elite teams likely still produce code of better quality with a higher qa bar than agentic code. But that category is dimishing everyday. The core point is that so much trust is reduced to "Joe in cubicle". Also a lot more than what he can carry. Most of the code that is being executed on your behalf is far from written by elite teams.
    - flohofwoe 2mo ago
      It has nothing to do with 'elite teams', it has to do with decades long and very careful maintenance. It's too early to say whether LLM generated projects will ever reach that sort of maturity, most examples I've seen so far are basically "fire and forget". But lets talk again in one or two decades, maybe there will be counterexamples of successful open source projects which will be just as well llm-maintained as human-mainained. But I suspect that to reach that sort of maturity, the resulting human effort will be mostly the same (e.g. not much of a productity win - except maybe on the 'edges', e.g. maintaining the test suite, documentation, helping to analyze bugs..., e.g. these are examples where LLMs are genuinely useful and where plagiarism hardly matters).
      - tossandthrow 2mo ago
        It really is not. There are some areas that are critical and where software developers carefully will detail stear the work. But the vast amount of software written, react components and rest endpoints, are very ripe to be entirely written by agents.
        flohofwoe 2mo ago
        > But the vast amount of software written, react components and rest endpoints, are very ripe to be entirely written by agents. In that I agree, nobody should be forced to write React code manually, that's almost a human rights violation ;) REST endpoints (and the code talking to those endpoints) should be code generated anyway though, no need for LLMs, and instead of human language prompting, a precise IDL should be the spec and basis for a mechnical code generation process. That problem was solved decades ago with much more pedestrian technology. E.g. it basically comes down to "it's fine to use LLMs for software that shouldn't have been written in the first place", and funny enough that's where LLMs are really good at: creating software that has been written a million times before with only minor variations, and doing this type of work manually (cranking out one cookie cutter React webpage or REST API after another) is essentially what's called 'bullshit jobs' (which bring food on the table though, but that's another topic). PS: > and where software developers carefully will detail stear the work. ...I think the further a project evolves, the less this "detailed stearing" will be any more productive than doing the same without LLMs. The older a project, the more the work shifts from implementation to decision making, and in most cases the result of that decision is just a very tiny code change. I already see cases in my daily work when I use 'agentic workflows' where a tiny change takes longer and involves more 'collatoral updates' then just fixing that one frigging line of code by hand like in the olden days, and for LLM-generated code bases I really do prefer to not mix LLM and manual work, I think that's the worst of all options.
- flohofwoe 2mo ago
  You sound like a webdev, e.g. the "let's just install 1000s of unvetted external dependencies" is nowhere else nearly as extreme as in web development land.
  - tossandthrow 2mo ago
    [flagged]
    - flohofwoe 2mo ago
      > You literally spend 99.999% og your time blindly trusting code ... Please don't be personal ;)

Is it "anti-LLM" or is it anti "here's some code I don't understand that a machine generated for me kthxbye"? Code is not just code, it's also liability and trust.

tossandthrow 2mo ago

You literally spend 99.999% og your time blindly trusting code - and that is only because I assume ymthat you write code yourself. For 99.999% of people, it is literally kthxbye on all code they execute on all their devices.

deleted 2mo ago
[deleted]
EvanAnderson 2mo ago
I don't think that's accurate. Until recently the code we rely on wasn't blindly trusted. People wrote and QA'd it. We may not have reviewed it personally, but that's one of the functions we outsource to software maintainers or "manufacturers". I'm not "blindly trusting" code on my computing devices. I'm trusting the vendors / maintainers to do their job. Until very recently the norm has been that the vast majority of code had human eyes and hands on it. Edit: The owners of those human eyes and hands had some type of accountability (either reputationally, in the case of free/open-source software, or occupationally, in the case of proprietary software). The LLM has no accountability as to the output it generates. The companies who make the LLMs also seem to have very little accountability, too. We've assumed a "blame the victim" stance when people use LLM-generated output in some inappropriate ways (legal briefs with "hallucinated" citations, articles "written" by LLMs). Whether that's the right location for accountability to be placed isn't for me to say, but that seems to be how it is. I'm not sure that we're applying accountability to LLM-generated code in the same way we are for, say, the LLM-generated legal brief.
- jadar 2mo ago
  A compiler is not an LLM, and I do not want to equivocate, but there are aspects of similarity. We do not assume people read the bytes of machine code to ensure it’s correct — there could be mistakes. We also write and run automated tests to ensure the code outputted from a compiler and an LLM behaves correctly. At some point, we won’t have to literally read every byte of code that comes out because we have a reasonable assurance that it’s correct.
  - EvanAnderson 2mo ago
    That we don't have to heavily scrutinize the code generated by compilers is a result of the huge amount of human toil that went into the compilers and test suites. Compilers are deterministic mechanisms so tests can be constructed. LLMs, at least as they're currently constructed, aren't deterministic (the whole "temperature" thing). I don't see how to build a mechanistic test for something that has non-deterministic output. It feels a little bit like solving the halting problem. I have no doubt we'll move away from human code review. The idea of large amounts of software edifice being built upon foundations that no human has reviewed or, perhaps even understands, is horrifying to me, though.
  - flohofwoe 2mo ago
    Compilers also usually give you the same output for the same input. And fwiw I do spend quite a lot of time reading compiler output to check that it's not doing something stupid or unexpected (usually as part of optimization work). Also this sort of 'technological whataboutism' really isn't helpful, compilers are entirely different from LLMs. I agree that it doesn't make much sense to read or review LLM output in detail, but I also don't plan to use LLM output for anything important or mission critical. That would be irresponsible.
    - AshamedCaptain 2mo ago
      In addition, when we usually say that triggering undefined behavior on C can start a game of Tetris or format your hard disk we're usually joking (or at least exaggerating), i.e. the most common failure conditions of compilers are really limited in scope, and very likely caught by whatever testing mechanism you use for the software. No such limits for LLMs where losing all your files is about par for the course for everyone who uses them regularly.
- tossandthrow 2mo ago
  > Until very recently the norm has been that the vast majority of code had human eyes and hands on it. What quality does it have that humans had eyes on code? Elite teams likely still produce code of better quality with a higher qa bar than agentic code. But that category is dimishing everyday. The core point is that so much trust is reduced to "Joe in cubicle". Also a lot more than what he can carry. Most of the code that is being executed on your behalf is far from written by elite teams.
  - flohofwoe 2mo ago
    It has nothing to do with 'elite teams', it has to do with decades long and very careful maintenance. It's too early to say whether LLM generated projects will ever reach that sort of maturity, most examples I've seen so far are basically "fire and forget". But lets talk again in one or two decades, maybe there will be counterexamples of successful open source projects which will be just as well llm-maintained as human-mainained. But I suspect that to reach that sort of maturity, the resulting human effort will be mostly the same (e.g. not much of a productity win - except maybe on the 'edges', e.g. maintaining the test suite, documentation, helping to analyze bugs..., e.g. these are examples where LLMs are genuinely useful and where plagiarism hardly matters).
    - tossandthrow 2mo ago
      It really is not. There are some areas that are critical and where software developers carefully will detail stear the work. But the vast amount of software written, react components and rest endpoints, are very ripe to be entirely written by agents.
      - flohofwoe 2mo ago
        > But the vast amount of software written, react components and rest endpoints, are very ripe to be entirely written by agents. In that I agree, nobody should be forced to write React code manually, that's almost a human rights violation ;) REST endpoints (and the code talking to those endpoints) should be code generated anyway though, no need for LLMs, and instead of human language prompting, a precise IDL should be the spec and basis for a mechnical code generation process. That problem was solved decades ago with much more pedestrian technology. E.g. it basically comes down to "it's fine to use LLMs for software that shouldn't have been written in the first place", and funny enough that's where LLMs are really good at: creating software that has been written a million times before with only minor variations, and doing this type of work manually (cranking out one cookie cutter React webpage or REST API after another) is essentially what's called 'bullshit jobs' (which bring food on the table though, but that's another topic). PS: > and where software developers carefully will detail stear the work. ...I think the further a project evolves, the less this "detailed stearing" will be any more productive than doing the same without LLMs. The older a project, the more the work shifts from implementation to decision making, and in most cases the result of that decision is just a very tiny code change. I already see cases in my daily work when I use 'agentic workflows' where a tiny change takes longer and involves more 'collatoral updates' then just fixing that one frigging line of code by hand like in the olden days, and for LLM-generated code bases I really do prefer to not mix LLM and manual work, I think that's the worst of all options.
flohofwoe 2mo ago
You sound like a webdev, e.g. the "let's just install 1000s of unvetted external dependencies" is nowhere else nearly as extreme as in web development land.
- tossandthrow 2mo ago
  [flagged]
  - flohofwoe 2mo ago
    > You literally spend 99.999% og your time blindly trusting code ... Please don't be personal ;)