5 ms·

We have a name: Large Language Models, or "Generative" AI. It doesn't think, it doesn't reason, and it doesn't listen to instructions, but it does generate pre

We have a name: Large Language Models, or "Generative" AI. It doesn't think, it doesn't reason, and it doesn't listen to instructions, but it does generate pretty good text!

chpatrick 1y ago
[citation needed] People constantly assert that LLMs don't think in some magic way that humans do think, when we don't even have any idea how that works.
- d3ckard 1y ago
  What can be asserted without proof, can be dismissed without proof. The proof burden is on AI proponents.
  - chpatrick 1y ago
    It's more that "thinking" is a vague term that we don't even understand in humans, so for me it's pretty meaningless to claim LLMs think or don't think. There's this very cliched comment to any AI HN headline which is this: "LLM's don't REALLY have <vague human behavior we don't really understand>. I know this for sure because I know both how humans work and how gigabytes of LLM weights work." or its cousin: "LLMs CAN'T possibly do <vague human behavior we don't really understand> BECAUSE they generate text one character at a time UNLIKE humans who generate text one character a time by typing with their fleshy fingers"
    - barnacs 1y ago
      To me, it's about motivation. Intelligent living beings have natural, evolutionary inputs as motivation underlying every rational thought. A biological reward system in the brain, a desire to avoid pain, hunger, boredom and sadness, seek to satisfy physiological needs, socialize, self-actualize, etc. These are the fundamental forces that drive us, even if the rational processes are capable of suppressing or delaying them to some degree. In contrast, machine learning models have a loss function or reward system purely constructed by humans to achieve a specific goal. They have no intrinsic motivations, feelings or goals. They are statistical models that approximate some mathematical function provided by humans.
      - chpatrick 1y ago
        Are any of those required for thinking?
        barnacs 1y ago
        In my view, absolutely yes. Thinking is a means to an end. It's about acting upon these motivations by abstracting, recollecting past experiences, planning, exploring, innovating. Without any motivation, there is nothing novel about the process. It really is just statistical approximation, "learning" at best, but definitely not "thinking".
        chpatrick 1y ago
        Again the problem is that what "thinking" is totally vague. To me if I can ask a computer a difficult question it hasn't seen before and it can give a correct answer, it's thinking. I don't need it to have a full and colorful human life to do that.
        barnacs 1y ago
        But it's only able to answer the question because it has been trained on all text in existence written by humans, precisely with the purpose to mimic human language use. It is the humans that produced the training data and then provided feedback in the form of reinforcement that did all the "thinking". Even if it can extrapolate to some degree (altough that's where "hallucinations" tend to become obvious), it could never, for example, invent a game like chess or a social construct like a legal system. Those require motivations like "boredom", "being social", having a "need for safety".
        chpatrick 1y ago
        Humans are also trained on data made by humans. > it could never, for example, invent a game like chess or a social construct like a legal system. Those require motivations like "boredom", "being social", having a "need for safety". That's creativity which is a different question from thinking.
    - shakna 1y ago
      Thinking is better understood than you seem to believe. We don't just study it in humans. We look at it in trees [0], for example. And whilst trees have distributed systems that ingest data from their surroundings, and use that to make choices, it isn't usually considered to be intelligence. Organizational complexity is one of the requirements for intelligence, and an LLM does not reach that threshold. They have vast amounts of data, but organizationally, they are still simple - thus "ai slop". [0] https://www.cell.com/trends/plant-science/abstract/S1360-1385(19)30126-8 https://www.cell.com/trends/plant-science/abstract/S1360-138...
      - chpatrick 1y ago
        Who says what degree of complexity is enough? Seems like deferring the problem to some other mystical arbiter. In my opinion AI slop is slop not because AIs are basic but because the prompt is minimal. A human went and put minimal effort into making something with an AI and put it online, producing slop, because the actual informational content is very low.
        shakna 1y ago
        > In my opinion AI slop is slop not because AIs are basic but because the prompt is minimal And you'd be disagreeing with the vast amount of research into AI. [0] > Moreover, they exhibit a counter-intuitive scaling limit: their reasoning effort increases with problem complexity up to a point, then declines despite having an adequate token budget. [0] https://machinelearning.apple.com/research/illusion-of-thinking https://machinelearning.apple.com/research/illusion-of-think...
        chpatrick 1y ago
        This article doesn't mention "slop" at all.
        shakna 1y ago
        But it does mention that prompt complexity is not related to the output. It does say that there is a maximal complexity that LLMs can have - which leads us back to... Intelligence requires organizational complexity that LLMs are not capable of.
    - omnicognate 1y ago
      This seems backwards to me. There's a fully understood thing (LLMs)[1] and a not-understood thing (brains)[2]. You seem to require a person to be able to fully define (presumably in some mathematical or mechanistic way) any behaviour they might observe in the not-understood thing before you will permit them to point out that the fully understood thing does not appear to exhibit that behaviour. In short you are requiring that people explain brains before you will permit them to observe that LLMs don't appear to be the same sort of thing as them. That seems rather unreasonable to me. That doesn't mean such claims don't need to made as specific as possible. Just saying something like "humans love but machines don't" isn't terribly compelling. I think mathematics is an area where it seems possible to draw a reasonably intuitively clear line. Personally, I've always considered the ability to independently contribute genuinely novel pure mathematical ideas (i.e. to perform significant independent research in pure maths) to be a likely hallmark of true human-like thinking. This is a high bar and one AI has not yet reached, despite the recent successes on the International Mathematical Olympiad [3] and various other recent claims. It isn't a moved goalpost, either - I've been saying the same thing for more than 20 years. I don't have to, and can't, define what "genuinely novel pure mathematical ideas" means, but we have a human system that recognises, verifies and rewards them so I expect us to know them when they are produced. By the way, your use of "magical" in your earlier comment, is typical of the way that argument is often presented, and I think it's telling. It's very easy to fall into the fallacy of deducing things from one's own lack of imagination. I've certainly fallen into that trap many times before. It's worth honestly considering whether your reasoning is of the form "I can't imagine there being something other than X, therefore there is nothing other than X". Personally, I think it's likely that to truly "do maths" requires something qualitatively different to a computer. Those who struggle to imagine anything other than a computer being possible often claim that that view is self-evidently wrong and mock such an imagined device as "magical", but that is not a convincing line of argument. The truth is that the physical Church-Turing thesis is a thesis, not a theorem, and a much shakier one than the original Church-Turing thesis. We have no particularly convincing reason to think such a device is impossible, and certainly no hard proof of it. [1] Individual behaviours of LLMs are "not understood" in the sense that there is typically not some neat story we can tell about how a particular behaviour arises that contains only the truly relevant information. However, on a more fundamental level LLMs are completely understood and always have been, as they are human inventions that we are able to build from scratch. [2] Anybody who thinks we understand how brains work isn't worth having this debate with until they read a bit about neuroscience and correct their misunderstanding. [3] The IMO involves problems in extremely well-trodden areas of mathematics. While the problems are carefully chosen to be novel they are problems to be solved in exam conditions, not mathematical research programs. The performance of the Google and OpenAI models on them, while impressive, is not evidence that they are capable of genuinely novel mathematical thought. What I'm looking for is the crank-the-handle-and-important-new-theorems-come-out machine that people have been trying to build since computers were invented. That isn't here yet, and if and when it arrives it really will turn maths on its head.
  - deleted 1y ago
    [deleted]
  - CamperBob2 1y ago
    The proof burden is on AI proponents. Why? Team "Stochastic Parrot" will just move the goalposts again, as they've done many times before.
- exe34 1y ago
  my favourite game is to try to get them to be more specific - every single time they manage to exclude a whole bunch of people from being "intelligent".
- mindcrime 1y ago
  > People constantly assert that LLMs don't think in some magic way that humans do think, It doesn't matter anyway. The marquee sign reads "Artificial Intelligence" not "Artificial Human Being". As long as AI displays intelligent behavior, it's "intelligent" in the relevant context. There's no basis for demanding that the mechanism be the same as what humans do. And of course it should go without saying that Artificial Intelligence exists on a continuum (just like human intelligence as far as that goes) and that we're not "there yet" as far as reaching the extreme high end of the continuum.
  - hermitcrab 1y ago
    Aircraft don't fly like birds, submarines don't swim like fish and AIs aren't going to think like a human.
    - chpatrick 1y ago
      Do you need to "think like a human" to think? Is it only thinking if you do it with a meat brain?
      - hermitcrab 1y ago
        Is the substrate important? If you made an accurate model of a human brain in software, in silicon or using water pipes and valves, would it be able to tnink? Would it be conscious? I have no idea.
        chpatrick 1y ago
        Me neither but that's why I don't like arguments that say LLM's can't do X because of their substrate, as if that was self-evident. It's like the aliens saying surely humans can't think because they're made of meat.
    - utyop22 1y ago
      Do these comparisons actually make sense though? Aircraft and submarines belong to a different category and of the same category, than AI.
- jbritton 1y ago
  I recently saw an article about LLMs and Towers of Hanoi. An LLM can write code to solve it. It can also output steps to solve it when the disk count is low like 3. It can’t give the steps when the disk count is higher. This indicates LLMs inability to reason and understand. Also see Gotham Chess and the Chatbot Championship. The Chatbots start off making good moves, but then quickly transition to making illegal moves and generally playing unbelievably poorly. They don’t understand the rules or strategy or anything.
  - leptons 1y ago
    Could the LLM "write code to solve it" if no human ever wrote code to solve it? Could it output "steps to solve it" if no human ever wrote about it before to have in its training data? The answer is no.
    - chpatrick 1y ago
      Could a human code the solution if they didn't learn to code from someone else? No. Could they do it if someone didn't tell them the rules of towers of hanoi? No. That doesn't mean much.
      - Gee101 1y ago
        It does since humans where able to invent a programming language.
        chpatrick 1y ago
        Have you tried asking a modern LLM to invent a programming language?
        CamperBob2 1y ago
        Have you? If so, how'd it go? Sounds like an interesting exercise.
- leptons 1y ago
  When I write a sentence, I do it with intent, with specific purpose in mind. When an "AI" does it, it's predicting the next word that might satisfy the input requirement. It doesn't care if the sentence it writes makes any sense, is factual, etc, so long as it is human readable and follows gramatic rules. It does not do this with any specific intent, which is why you get slop and just plain wrong output a fair amount of time. Just because it produces something that sounds correct sometimes does not mean it's doing any thinking at all. Yes, humans do actually think before they speak, LLMs do not, cannot, and will not because that is not what they are designed to do.
  - chpatrick 1y ago
    Actually LLMs crunch through half a terabyte of weights before they "speak". How are you so confident that nothing happens in that immense amount of processing that has anything to do with thinking? Modern LLMs are also trained to have an inner dialogue before they output an answer to the user. When you type the next word you also put a word that fits some requirement. That doesn't mean you're not thinking.
    - leptons 1y ago
      "crunch through half a terabyte of weights" isn't thinking. Following grammatical rules to produce a readable sentence isn't thought, it's statistics, and whether that sentence is factual or foolish isn't something the LLM cares about. If LLMs didn't so constantly produce garbage, I might agree with you more.
      - chpatrick 1y ago
        They don't follow "grammatical rules", they process inputs with an incredibly large neural net. It's like saying humans aren't really thinking because their brains are made of meat.
- elbasti 1y ago
  It's not some "magical way"--the ways in which a human thinks that an LLM doesn't are pretty obvious, and I dare say self-evidently part of what we think constitutes human intelligence: - We have a sense of time (ie, ask an LLM to follow up in 2 minutes) - We can follow negative instructions ("don't hallucinate, if you don't know the answer, say so")
  - chpatrick 1y ago
    I think plenty of people have problems with the second one but you wouldn't say that means they can't think.
    - bluefirebrand 1y ago
      We don't need to prove all humans are capable of this. We can demonstrate that some humans are, therefore humans must be capable, broadly speaking Until we see an LLM that is capable of this, then they aren't capable of it, period
      - chpatrick 1y ago
        Sometimes LLMs hallucinate or bullshit, sometimes they don't, sometimes humans hallucinate or bullshit, sometimes they don't. It's not like you can tell a human to stop being delusional on command either. I'm not really seeing the argument.
        bluefirebrand 1y ago
        If a human hallucinates or bullshits in a way that harms you or your company you can take action against them That's the difference. AI cannot be held responsible for hallucinations that cause harm, therefore it cannot be incentivized to avoid that behavior, therefore it cannot be trusted Simple as that
        chpatrick 1y ago
        The question wasn't can it be trusted, it was does it think.
- lordhumphrey 1y ago
  Yes, and the name for this behaviour is called "being scientific". Imagine a process called A, and, as you say, we've no idea how it works. Imagine, then, a new process, B, comes along. Some people know a lot about how B works, most people don't. But the people selling B, they continuously tell me it works like process A, and even resort to using various cutesy linguistic tricks to make that feel like it's the case. The people selling B even go so far as to suggest that if we don't accept a future where B takes over, we won't have a job, no matter what our poor A does. What's the rational thing to do, for a sceptical, scientific mind? Agree with the company, that process B is of course like process A, when we - as you say yourself - don't understand process A in any comprehensive way at all? Or would that be utterly nonsensical?
  - chpatrick 1y ago
    Again, I'm not claiming that LLMs can think like people (I don't know that). I just don't like that people confidently claim that they can't, just because they work differently from biological brains. That doesn't matter when it comes to the Turing test (which they passed a while ago btw), just what it says.
- mvdtnz 1y ago
  The classic God of the Gaps - we don't know how human brains think, so what LLMs do must be it!
  - chpatrick 1y ago
    I'm not saying that LLMs do anything, just that it's rich to confidently say they don't do something when we don't even understand how humans do it. It's like we're pretending cognition is a solved problem so we can make grand claims about what LLM's aren't really doing.

We have a name: Large Language Models, or "Generative" AI. It doesn't think, it doesn't reason, and it doesn't listen to instructions, but it does generate pretty good text!

chpatrick 1y ago

[citation needed] People constantly assert that LLMs don't think in some magic way that humans do think, when we don't even have any idea how that works.

d3ckard 1y ago
What can be asserted without proof, can be dismissed without proof. The proof burden is on AI proponents.
- chpatrick 1y ago
  It's more that "thinking" is a vague term that we don't even understand in humans, so for me it's pretty meaningless to claim LLMs think or don't think. There's this very cliched comment to any AI HN headline which is this: "LLM's don't REALLY have <vague human behavior we don't really understand>. I know this for sure because I know both how humans work and how gigabytes of LLM weights work." or its cousin: "LLMs CAN'T possibly do <vague human behavior we don't really understand> BECAUSE they generate text one character at a time UNLIKE humans who generate text one character a time by typing with their fleshy fingers"
  - barnacs 1y ago
    To me, it's about motivation. Intelligent living beings have natural, evolutionary inputs as motivation underlying every rational thought. A biological reward system in the brain, a desire to avoid pain, hunger, boredom and sadness, seek to satisfy physiological needs, socialize, self-actualize, etc. These are the fundamental forces that drive us, even if the rational processes are capable of suppressing or delaying them to some degree. In contrast, machine learning models have a loss function or reward system purely constructed by humans to achieve a specific goal. They have no intrinsic motivations, feelings or goals. They are statistical models that approximate some mathematical function provided by humans.
    - chpatrick 1y ago
      Are any of those required for thinking?
      - barnacs 1y ago
        In my view, absolutely yes. Thinking is a means to an end. It's about acting upon these motivations by abstracting, recollecting past experiences, planning, exploring, innovating. Without any motivation, there is nothing novel about the process. It really is just statistical approximation, "learning" at best, but definitely not "thinking".
        chpatrick 1y ago
        Again the problem is that what "thinking" is totally vague. To me if I can ask a computer a difficult question it hasn't seen before and it can give a correct answer, it's thinking. I don't need it to have a full and colorful human life to do that.
        barnacs 1y ago
        But it's only able to answer the question because it has been trained on all text in existence written by humans, precisely with the purpose to mimic human language use. It is the humans that produced the training data and then provided feedback in the form of reinforcement that did all the "thinking". Even if it can extrapolate to some degree (altough that's where "hallucinations" tend to become obvious), it could never, for example, invent a game like chess or a social construct like a legal system. Those require motivations like "boredom", "being social", having a "need for safety".
        chpatrick 1y ago
        Humans are also trained on data made by humans. > it could never, for example, invent a game like chess or a social construct like a legal system. Those require motivations like "boredom", "being social", having a "need for safety". That's creativity which is a different question from thinking.
  - shakna 1y ago
    Thinking is better understood than you seem to believe. We don't just study it in humans. We look at it in trees [0], for example. And whilst trees have distributed systems that ingest data from their surroundings, and use that to make choices, it isn't usually considered to be intelligence. Organizational complexity is one of the requirements for intelligence, and an LLM does not reach that threshold. They have vast amounts of data, but organizationally, they are still simple - thus "ai slop". [0] https://www.cell.com/trends/plant-science/abstract/S1360-1385(19)30126-8 https://www.cell.com/trends/plant-science/abstract/S1360-138...
    - chpatrick 1y ago
      Who says what degree of complexity is enough? Seems like deferring the problem to some other mystical arbiter. In my opinion AI slop is slop not because AIs are basic but because the prompt is minimal. A human went and put minimal effort into making something with an AI and put it online, producing slop, because the actual informational content is very low.
      - shakna 1y ago
        > In my opinion AI slop is slop not because AIs are basic but because the prompt is minimal And you'd be disagreeing with the vast amount of research into AI. [0] > Moreover, they exhibit a counter-intuitive scaling limit: their reasoning effort increases with problem complexity up to a point, then declines despite having an adequate token budget. [0] https://machinelearning.apple.com/research/illusion-of-thinking https://machinelearning.apple.com/research/illusion-of-think...
        chpatrick 1y ago
        This article doesn't mention "slop" at all.
        shakna 1y ago
        But it does mention that prompt complexity is not related to the output. It does say that there is a maximal complexity that LLMs can have - which leads us back to... Intelligence requires organizational complexity that LLMs are not capable of.
  - omnicognate 1y ago
    This seems backwards to me. There's a fully understood thing (LLMs)[1] and a not-understood thing (brains)[2]. You seem to require a person to be able to fully define (presumably in some mathematical or mechanistic way) any behaviour they might observe in the not-understood thing before you will permit them to point out that the fully understood thing does not appear to exhibit that behaviour. In short you are requiring that people explain brains before you will permit them to observe that LLMs don't appear to be the same sort of thing as them. That seems rather unreasonable to me. That doesn't mean such claims don't need to made as specific as possible. Just saying something like "humans love but machines don't" isn't terribly compelling. I think mathematics is an area where it seems possible to draw a reasonably intuitively clear line. Personally, I've always considered the ability to independently contribute genuinely novel pure mathematical ideas (i.e. to perform significant independent research in pure maths) to be a likely hallmark of true human-like thinking. This is a high bar and one AI has not yet reached, despite the recent successes on the International Mathematical Olympiad [3] and various other recent claims. It isn't a moved goalpost, either - I've been saying the same thing for more than 20 years. I don't have to, and can't, define what "genuinely novel pure mathematical ideas" means, but we have a human system that recognises, verifies and rewards them so I expect us to know them when they are produced. By the way, your use of "magical" in your earlier comment, is typical of the way that argument is often presented, and I think it's telling. It's very easy to fall into the fallacy of deducing things from one's own lack of imagination. I've certainly fallen into that trap many times before. It's worth honestly considering whether your reasoning is of the form "I can't imagine there being something other than X, therefore there is nothing other than X". Personally, I think it's likely that to truly "do maths" requires something qualitatively different to a computer. Those who struggle to imagine anything other than a computer being possible often claim that that view is self-evidently wrong and mock such an imagined device as "magical", but that is not a convincing line of argument. The truth is that the physical Church-Turing thesis is a thesis, not a theorem, and a much shakier one than the original Church-Turing thesis. We have no particularly convincing reason to think such a device is impossible, and certainly no hard proof of it. [1] Individual behaviours of LLMs are "not understood" in the sense that there is typically not some neat story we can tell about how a particular behaviour arises that contains only the truly relevant information. However, on a more fundamental level LLMs are completely understood and always have been, as they are human inventions that we are able to build from scratch. [2] Anybody who thinks we understand how brains work isn't worth having this debate with until they read a bit about neuroscience and correct their misunderstanding. [3] The IMO involves problems in extremely well-trodden areas of mathematics. While the problems are carefully chosen to be novel they are problems to be solved in exam conditions, not mathematical research programs. The performance of the Google and OpenAI models on them, while impressive, is not evidence that they are capable of genuinely novel mathematical thought. What I'm looking for is the crank-the-handle-and-important-new-theorems-come-out machine that people have been trying to build since computers were invented. That isn't here yet, and if and when it arrives it really will turn maths on its head.
- deleted 1y ago
  [deleted]
- CamperBob2 1y ago
  The proof burden is on AI proponents. Why? Team "Stochastic Parrot" will just move the goalposts again, as they've done many times before.
exe34 1y ago
my favourite game is to try to get them to be more specific - every single time they manage to exclude a whole bunch of people from being "intelligent".
mindcrime 1y ago
> People constantly assert that LLMs don't think in some magic way that humans do think, It doesn't matter anyway. The marquee sign reads "Artificial Intelligence" not "Artificial Human Being". As long as AI displays intelligent behavior, it's "intelligent" in the relevant context. There's no basis for demanding that the mechanism be the same as what humans do. And of course it should go without saying that Artificial Intelligence exists on a continuum (just like human intelligence as far as that goes) and that we're not "there yet" as far as reaching the extreme high end of the continuum.
- hermitcrab 1y ago
  Aircraft don't fly like birds, submarines don't swim like fish and AIs aren't going to think like a human.
  - chpatrick 1y ago
    Do you need to "think like a human" to think? Is it only thinking if you do it with a meat brain?
    - hermitcrab 1y ago
      Is the substrate important? If you made an accurate model of a human brain in software, in silicon or using water pipes and valves, would it be able to tnink? Would it be conscious? I have no idea.
      - chpatrick 1y ago
        Me neither but that's why I don't like arguments that say LLM's can't do X because of their substrate, as if that was self-evident. It's like the aliens saying surely humans can't think because they're made of meat.
  - utyop22 1y ago
    Do these comparisons actually make sense though? Aircraft and submarines belong to a different category and of the same category, than AI.
jbritton 1y ago
I recently saw an article about LLMs and Towers of Hanoi. An LLM can write code to solve it. It can also output steps to solve it when the disk count is low like 3. It can’t give the steps when the disk count is higher. This indicates LLMs inability to reason and understand. Also see Gotham Chess and the Chatbot Championship. The Chatbots start off making good moves, but then quickly transition to making illegal moves and generally playing unbelievably poorly. They don’t understand the rules or strategy or anything.
- leptons 1y ago
  Could the LLM "write code to solve it" if no human ever wrote code to solve it? Could it output "steps to solve it" if no human ever wrote about it before to have in its training data? The answer is no.
  - chpatrick 1y ago
    Could a human code the solution if they didn't learn to code from someone else? No. Could they do it if someone didn't tell them the rules of towers of hanoi? No. That doesn't mean much.
    - Gee101 1y ago
      It does since humans where able to invent a programming language.
      - chpatrick 1y ago
        Have you tried asking a modern LLM to invent a programming language?
        CamperBob2 1y ago
        Have you? If so, how'd it go? Sounds like an interesting exercise.
leptons 1y ago
When I write a sentence, I do it with intent, with specific purpose in mind. When an "AI" does it, it's predicting the next word that might satisfy the input requirement. It doesn't care if the sentence it writes makes any sense, is factual, etc, so long as it is human readable and follows gramatic rules. It does not do this with any specific intent, which is why you get slop and just plain wrong output a fair amount of time. Just because it produces something that sounds correct sometimes does not mean it's doing any thinking at all. Yes, humans do actually think before they speak, LLMs do not, cannot, and will not because that is not what they are designed to do.
- chpatrick 1y ago
  Actually LLMs crunch through half a terabyte of weights before they "speak". How are you so confident that nothing happens in that immense amount of processing that has anything to do with thinking? Modern LLMs are also trained to have an inner dialogue before they output an answer to the user. When you type the next word you also put a word that fits some requirement. That doesn't mean you're not thinking.
  - leptons 1y ago
    "crunch through half a terabyte of weights" isn't thinking. Following grammatical rules to produce a readable sentence isn't thought, it's statistics, and whether that sentence is factual or foolish isn't something the LLM cares about. If LLMs didn't so constantly produce garbage, I might agree with you more.
    - chpatrick 1y ago
      They don't follow "grammatical rules", they process inputs with an incredibly large neural net. It's like saying humans aren't really thinking because their brains are made of meat.
elbasti 1y ago
It's not some "magical way"--the ways in which a human thinks that an LLM doesn't are pretty obvious, and I dare say self-evidently part of what we think constitutes human intelligence: - We have a sense of time (ie, ask an LLM to follow up in 2 minutes) - We can follow negative instructions ("don't hallucinate, if you don't know the answer, say so")
- chpatrick 1y ago
  I think plenty of people have problems with the second one but you wouldn't say that means they can't think.
  - bluefirebrand 1y ago
    We don't need to prove all humans are capable of this. We can demonstrate that some humans are, therefore humans must be capable, broadly speaking Until we see an LLM that is capable of this, then they aren't capable of it, period
    - chpatrick 1y ago
      Sometimes LLMs hallucinate or bullshit, sometimes they don't, sometimes humans hallucinate or bullshit, sometimes they don't. It's not like you can tell a human to stop being delusional on command either. I'm not really seeing the argument.
      - bluefirebrand 1y ago
        If a human hallucinates or bullshits in a way that harms you or your company you can take action against them That's the difference. AI cannot be held responsible for hallucinations that cause harm, therefore it cannot be incentivized to avoid that behavior, therefore it cannot be trusted Simple as that
        chpatrick 1y ago
        The question wasn't can it be trusted, it was does it think.
lordhumphrey 1y ago
Yes, and the name for this behaviour is called "being scientific". Imagine a process called A, and, as you say, we've no idea how it works. Imagine, then, a new process, B, comes along. Some people know a lot about how B works, most people don't. But the people selling B, they continuously tell me it works like process A, and even resort to using various cutesy linguistic tricks to make that feel like it's the case. The people selling B even go so far as to suggest that if we don't accept a future where B takes over, we won't have a job, no matter what our poor A does. What's the rational thing to do, for a sceptical, scientific mind? Agree with the company, that process B is of course like process A, when we - as you say yourself - don't understand process A in any comprehensive way at all? Or would that be utterly nonsensical?
- chpatrick 1y ago
  Again, I'm not claiming that LLMs can think like people (I don't know that). I just don't like that people confidently claim that they can't, just because they work differently from biological brains. That doesn't matter when it comes to the Turing test (which they passed a while ago btw), just what it says.
mvdtnz 1y ago
The classic God of the Gaps - we don't know how human brains think, so what LLMs do must be it!
- chpatrick 1y ago
  I'm not saying that LLMs do anything, just that it's rich to confidently say they don't do something when we don't even understand how humans do it. It's like we're pretending cognition is a solved problem so we can make grand claims about what LLM's aren't really doing.