5 ms·

Humans don't make mistakes nearly as much, the mistakes they do make are way more predictable (they're easier to spot in code review), and they don't tend to ma

Humans don't make mistakes nearly as much, the mistakes they do make are way more predictable (they're easier to spot in code review), and they don't tend to make the kinds of catastrophic mistakes that could sink a business. They also tend to cause codebases to rapidly deteriorate, since even very disciplined reviewers can miss the kinds of strange and unpredictable stuff an LLM will do. Redundant code isn't evident in a diff, and things like tautological tests, or useless tests where they're mocking everything and only actually testing the mocks. Or they'll write a bunch of redundant code because they really just aggressively avoid code re-use unless you are very specific. The real problem is just that they don't have brains, and can't think. They generate text that is optimized to look the most right, but not to be the most right. That means they're deceptive right off the bat. When a human is wrong, it usually looks wrong. When an LLM is wrong, it's generating the most correct looking thing it possibly could while still being wrong, with no consideration for actual correctness. It has no idea what "correctness" even means, or any ideas at all, because it's a computer doing matmul. They are text summarization/regurgitation, pattern matching machines. They regurgitate summaries of things seen in their training data, and that training data was written by humans who can think. We just let ourselves get duped into believing the machine is the where the thinking is coming from and not the (likely uncompensated) author(s) whose work was regurgitated for you.

ACCount37 1y ago
>The real problem is just that they don't have brains, and can't think. That would have had more weight if you haven't just described junior developer behavior beforehand. "LLMs can't think" is anthropocentric cope. It's the old AI effect all over again - people would rather die than admit that there's very little practical difference between their own "thinking" and that of an AI chatbot.
- gnulinux996 1y ago
  > That would have had more weight if you haven't just described junior developer behavior beforehand. Effectively telling that junior developers "don't have brains" is in very bad taste and offensively wrong. > people would rather die than admit that there's very little practical difference between their own "thinking" and that of an AI chatbot. Would you like to elaborate on this? I was told that McDonalds employees would have been replaced by now, self-driving cars will be driving the streets and new medicines would have been discovered. It's been a couple of years that "AI" is out, and no singularity yet.
  - ACCount37 1y ago
    LLMs use the same type of "abstract thinking" process as humans. Which is why they can struggle with 6-digit multiplication (unlike computer code, very much like humans), but not with parsing out metaphors or describing what love is (unlike computer code, very much like humans). The capability profile of an LLM is amusingly humanlike. Setting the bar for "AI" at "singularity" is a bit like setting requirements for "fusion" at "creating a star more powerful than the Sun". Very good for dismissing all existing fusion research, but not any good for actually understanding fusion. If we had two humans, one with IQ 80 and another with IQ 120, we wouldn't say that one of them isn't "thinking". It's just that one of them is much worse at "thinking" than the other. Which is where a lot of LLMs are currently at. They are, for all intents and purposes, thinking. Are they any good at it though? Depends on what you want from them. Sometimes they're good enough, and sometimes they aren't.
    - habinero 1y ago
      > LLMs use the same type of "abstract thinking" process as humans It's surprising you say that, considering we don't actually understand the mechanisms behind how humans think. We do know that human brains are so good at patterns, they'll even see patterns and such that aren't actually there. LLMs are a pile of statistics that can mimic human speech patterns if you don't tax them too hard. Anyone who thinks otherwise is just Clever Hans-ing themselves.
      - ACCount37 1y ago
        We understand the outcomes well enough. LLMs converge onto a similar process by being trained on human-made text. Is LLM reasoning a 1:1 replica of what the human brain does? No, but it does something very similar in function. I see no reason to think that humans are anything more than "a pile of statistics that can mimic human speech patterns if you don't tax them too hard". Humans can get offended when you point it out though. It's too dismissive of their unique human gift of intelligence that a chatbot clearly doesn't have.
        1y ago
- deleted 1y ago
  [deleted]
ozim 1y ago
I don't really see this argument playing out. As much as I was trying out copilot — if I ask it to make "todo app" it will make me a "todo app" it will not hallucinate making "calculator app". Duplicated code or tautological tests are not going to sink the business. From my experience I have seen much more problems and hours burned caused by wrong application of DRY than from duplication. I can say that in regulated environments duplicated code that has no abstractions is even preferred.
alecco 1y ago
> Humans don't make mistakes nearly as much [...] Yeah, I remember how in every large corporation the specs were perfectly interpreted and with no issues, at all. Humans are great at communication and understanding each other. https://google.com/search?q=tree+swing+cartoon https://google.com/search?q=tree+swing+cartoon

ACCount37 1y ago

>The real problem is just that they don't have brains, and can't think. That would have had more weight if you haven't just described junior developer behavior beforehand. "LLMs can't think" is anthropocentric cope. It's the old AI effect all over again - people would rather die than admit that there's very little practical difference between their own "thinking" and that of an AI chatbot.

gnulinux996 1y ago
> That would have had more weight if you haven't just described junior developer behavior beforehand. Effectively telling that junior developers "don't have brains" is in very bad taste and offensively wrong. > people would rather die than admit that there's very little practical difference between their own "thinking" and that of an AI chatbot. Would you like to elaborate on this? I was told that McDonalds employees would have been replaced by now, self-driving cars will be driving the streets and new medicines would have been discovered. It's been a couple of years that "AI" is out, and no singularity yet.
- ACCount37 1y ago
  LLMs use the same type of "abstract thinking" process as humans. Which is why they can struggle with 6-digit multiplication (unlike computer code, very much like humans), but not with parsing out metaphors or describing what love is (unlike computer code, very much like humans). The capability profile of an LLM is amusingly humanlike. Setting the bar for "AI" at "singularity" is a bit like setting requirements for "fusion" at "creating a star more powerful than the Sun". Very good for dismissing all existing fusion research, but not any good for actually understanding fusion. If we had two humans, one with IQ 80 and another with IQ 120, we wouldn't say that one of them isn't "thinking". It's just that one of them is much worse at "thinking" than the other. Which is where a lot of LLMs are currently at. They are, for all intents and purposes, thinking. Are they any good at it though? Depends on what you want from them. Sometimes they're good enough, and sometimes they aren't.
  - habinero 1y ago
    > LLMs use the same type of "abstract thinking" process as humans It's surprising you say that, considering we don't actually understand the mechanisms behind how humans think. We do know that human brains are so good at patterns, they'll even see patterns and such that aren't actually there. LLMs are a pile of statistics that can mimic human speech patterns if you don't tax them too hard. Anyone who thinks otherwise is just Clever Hans-ing themselves.
    - ACCount37 1y ago
      We understand the outcomes well enough. LLMs converge onto a similar process by being trained on human-made text. Is LLM reasoning a 1:1 replica of what the human brain does? No, but it does something very similar in function. I see no reason to think that humans are anything more than "a pile of statistics that can mimic human speech patterns if you don't tax them too hard". Humans can get offended when you point it out though. It's too dismissive of their unique human gift of intelligence that a chatbot clearly doesn't have.
      - 1y ago
deleted 1y ago
[deleted]

ozim 1y ago

I don't really see this argument playing out. As much as I was trying out copilot — if I ask it to make "todo app" it will make me a "todo app" it will not hallucinate making "calculator app". Duplicated code or tautological tests are not going to sink the business. From my experience I have seen much more problems and hours burned caused by wrong application of DRY than from duplication. I can say that in regulated environments duplicated code that has no abstractions is even preferred.

alecco 1y ago

> Humans don't make mistakes nearly as much [...] Yeah, I remember how in every large corporation the specs were perfectly interpreted and with no issues, at all. Humans are great at communication and understanding each other. https://google.com/search?q=tree+swing+cartoon https://google.com/search?q=tree+swing+cartoon