8 ms·

Because plenty of people, even ones that should know better, really believe it's a conscious, thinking entity, not just some turn of phrase. I have a coworker

Because plenty of people, even ones that should know better, really believe it's a conscious, thinking entity, not just some turn of phrase. I have a coworker that spends at least 10 hours a week arguing with his like you would with a conscious person. I've gently tried to explain it's like arguing with your compiler for giving you an incoherent error message - it's pointless. It doesn't understand, it can't understand, and even if it could, you arguing with it isn't going to make it "learn" or act differently.

alexey-salmin 2mo ago
> It doesn't understand, it can't understand, and even if it could, you arguing with it isn't going to make it "learn" or act differently. I know people who are like that too. I'm not sure anthropomorphizing is a problem. Seeing analogies everywhere is an innate human trait, sometimes it can be harmful but more often it's useful.
- danaris 2mo ago
  > I know people who are like that too. This is part of the problem being described. You are part of the problem. "Some people are bad at X" is not comparable—is not even in the same category—as "LLMs are fundamentally incapable of X". Every human (at least to a first approximation) is capable of understanding, of learning, of remembering things, of doing math, of counting the number of "r"s in "strawberry". What you are observing is that some humans are careless, do not take the time and effort to understand, or have internalized the idea that they're "not smart enough" or "not the type of person" who understands things like <whatever>. That has nothing remotely to do with the fact that LLMs have no consciousness, no self-awareness, no cognition, no understanding. At a fundamental level.
  - JohnMakin 2mo ago
    I don't think anyone in this conversation is saying this behavior is anything but the fault of the user not understanding how these tools work? This is a weirdly aggressive post.
    - danaris 2mo ago
      This is an extremely common fallacy I've seen lots and lots of people fall into with respect to LLMs. In nearly every case, they use the fact that "some humans can't do X" to claim that LLMs are, in fact, basically conscious/human-like/AGI already. This is deeply untrue, and is highly likely to lead them to bad conclusions about what we can and should do with LLMs.
      - throw310822 2mo ago
        > This is an extremely common fallacy ... they use the fact that "some humans can't do X" to claim that LLMs are, in fact, basically conscious/human-like/AGI already. Claiming that LLMs are conscious or human-like because humans can't do X seems a very strange way to argue for LLM intelligence. Usually, it goes the other way around: an LLM sceptic says "LLMs are dumb because they can't do X" and soon someone has to remind them that also most of the population can't, in fact, do X.
        randomImmigrant 2mo ago
        “ Usually, it goes the other way around: an LLM sceptic says "LLMs are dumb because they can't do X" and soon someone has to remind them that also most of the population can't, in fact, do X.” And it should not go this way, is the OPs point. For an analogy, imagine someone looked at a bunch of lightbulbs and expresses dissatisfaction that they aren’t really stars. If someone replies by saying “not all stars are equally bright”, do you think that fact should carry any weight in the argument?
        throw310822 2mo ago
        > And it should not go this way, is the OPs point Well, he's wrong. If you argue that LLMs and humans are fundamentally different because all LLMs do X and no human does it, then showing you that it's not true demolishes your argument. Doesn't prove anything positive, but it certainly proves that your argument is invalid.
- tavavex 2mo ago
  Anthropomorphizing is a problem when you're talking about treating something that's not living as if it were. Using humanizing language invites discussions of things like the rights and feelings of an algorithm. A judge that is misled by the application of human-centric language to an algorithm can lead to some terrible outcomes. Not everyone is an LLM expert and the language people use leads to them treating LLMs like actual, real humans. That is terrifying.
  - salawat 2mo ago
    What is terrifying is the propensity of people hoping for a mechanical slave to do everything possible to avoid touching on the possibility for being-ness of the technology they are desperately hoping will work as a basis for that implementation.
    - hypfer 2mo ago
      Cyberpsychosis. A next token predictor is not a being. Stop posting these things. Stop thinking these things.
      - Eisenstein 2mo ago
        Why can't it be a being? Why is thinking about such a possibilty bad?
        hypfer 2mo ago
        Sorry, but if you're genuinely asking this not for trolling reasons then you should _really_ _really_ see a medical professional (not an insult). Online comment sections are not the correct place to unpack any of this. Which is indeed terminating this comment chain, but for good (and benevolent) reason. Doing anything else other than referring to a trained professional in a controlled context would potentially just feed delusions, which is highly unethical. Might not even be yours but those of another reader.
        2mo ago
- hypfer 2mo ago
  > I know people who are like that too. Cool. Question: Why _on earth_ would we make more of them?
- qsera 2mo ago
  >I'm not sure anthropomorphizing is a problem. It is a problem because it does not really understand stuff. For example, if you ask a human "Do you understand that doing X will kill you 100%?". If the human answers "Yes", then you can expect the human to act according to that understanding. That they will not do X But an LLM will happily acknowledges the consequences of doing X, but will still proceed to do X. So replace the human in the above example with a robot controlled by an LLM. There is no guarantee that it will not do X.
  - wccrawford 2mo ago
    Have you not met humans that do that, too? When it was found that smoking causes cancer, many, many people kept doing it. Then the government taxed the crap out of it, and some people still do it. It's easy to say that people have other motivations, but doesn't the AI, too? You tell it not to do X, but you've also told it to do something that would benefit from X. It's going to "want" to still do X, to support that other thing. It's also got all the "knowledge" that enables it to do the work in the first place, and all of the tendencies of the people who do that work, because that's what it's trained on. It's really easy to anthropomorphize AI because it was literally modeled after people. And for the record, as lead developer, I've had actual humans that reported me to go ahead and do things I specifically told them not to.
    - qsera 2mo ago
      >smoking causes cancer Which part of "X will kill you 100%" did you not get?
traverseda 2mo ago
Do you have a working definition of "conscious"?
- JohnMakin 2mo ago
  Do you? I’m not sure what you’re getting at.
  - nostrademons 2mo ago
    Not GP, but one of the challenges with debating whether LLMs are "conscious" is that we don't even really know what it means for a human to be "conscious", or even if consciousness is experienced by other humans the same way it is for ourselves. What we do know: neurons carry electrical impulses across their synapses to trigger other neurons to fire, and more frequently used synapses are strengthened while infrequently used ones are pruned. This is not all that dissimilar to how a multi-layer perceptron is trained: it's floating point numbers in a big matrix rather than biological structures and electrical impulses, but there is still that element of frequently used connections being strengthened and infrequently used ones being pruned. What we hypothesize but do not know: there is a thin brain structure of grey matter called the claustrum that has tendrils that reach into nearly every other brain structure. In many ways, this is similar to the attention mechanism of the transformer architecture. It is hypothesized that this may be the seat of consciousness, owing to experiments where electrical stimulation of the claustrum caused patients to immediately lose consciousness. However, there is no way to prove this, owing to the difficulty of otherwise removing or disabling the most connected structure in the brain and observing its effect on consciousness without permanently killing the patient. Beyond that, we don't know much. I've got a family friend that's been a practicing therapist for 50 years, and I asked him what was the most interesting observation he made in his career. It was that "Everybody experiences the world in a different way, and yet everybody assumes that everyone else experiences the world the same way they do."
    - Dilettante_ 2mo ago
      >"Everybody experiences the world in a different way, and yet everybody assumes that everyone else experiences the world the same way they do." Relevant article: "Generalizing from one example"[https://www.lesswrong.com/posts/baTWMegR42PAsH9qJ/generalizing-from-one-example https://www.lesswrong.com/posts/baTWMegR42PAsH9qJ/generalizi...] Personal note: That principle has been the bane of my autistic existence. People sometimes seem literally incapable of understanding that other people even can be different.
red75prime 2mo ago
> if it could, you arguing with it isn't going to make it "learn" or act differently. Are you talking about a specific harness that doesn't have context retention mechanisms? For example, ChatGPT with disabled memory feature? Or in general where "it" is a fixed-weights network? The latter is trivially true, of course.
- JohnMakin 2mo ago
  Even claude with “memory” enabled isn’t really “remembering” anything. It just injects it into the context and you hope it happens to find it relevant in its attention mechanisms, and then remembers to actually act on it. Anthropic’s own documentation states claude can and will ignore/truncate these. It’s a context trick, nothing approaching actual “memory,” and in fact, arguing with it will make a bunch of memory files, sometimes contradictory, and clutter up the context and act even worse.
  - red75prime 2mo ago
    I'm not a big fan of arguments like "it's not the real [human quality], it's [mechanistic explanation]." They lack a part: "because the [human quality] allows us to do X, Y, Z, which is impossible with [this mechanism]." I agree that the relevance of retrieved pieces and the management of long-term storage could be improved, though.
    - JohnMakin 2mo ago
      I just don’t find it really relevant to the argument presented I guess. I disable auto memory and have my own mechanisms and infrastructure with how my agentic system “knows” and “remembers” things which is roughly an automated, sometimes self-correcting working index on the file system. It behaves much better than claude’s automated “memory” system, so I use that, but digging into how that worked and making something of my own just makes me really dismissive of comparing it to something like actual memory, so I apologize if it came off dismissive.
      - 2mo ago
Jtarii 2mo ago
>Because plenty of people, even ones that should know better, really believe it's a conscious, thinking entity You need evidence to make the positive claim that LLMs do not posses any form of consciousness.
- roughly 2mo ago
  Well, our current set of evidence is that it’s a mechanistic mechanical algorithm with an RNG embedded in it and we can both get it to repeatedly produce the same output for the same input and also get it to repeatedly do absolutely nothing at all, which are not characteristics we usually find in objects evincing consciousness. LLMs bear absolutely none of the traits we’ve come to recognize as the external hallmarks of consciousness in biological organisms, nor anything that would seem analogous in a non-biological substrate. That said, we don’t have a rigorous definition of consciousness that includes the actual phenomenology of consciousness, so I daresay if you’re going to go around asserting the LLM is conscious despite all existing evidence to the contrary, I think the impetus is on you to define some version of consciousness that isn’t also satisfied by a book or a movie.
  - Jtarii 2mo ago
    I would agree that LLMs aren't much like the human brain, that doesn't prove that consciousness is not occurring. Does a fruit fly experience anything? If a microscopic insect can experience something, why can't a CPU? >if you’re going to go around asserting the LLM is conscious despite all existing evidence to the contrary Well there is neither any evidence that suggests LLMs are not conscious, and I also never asserted that they are. If I had to guess I would say that any information processing system will produce some kind of conscious experience, but I ultimately have literally no idea.
    - roughly 2mo ago
      There’s plenty. First, when we say “LLM”, what are we referring to? What is the entity that would be conscious in this case? The reason this is important is because powerful people are currently trying to use the dodge that LLMs are conscious to launder liability for their own policy choices, so the sloppy thinking and half-assed conjecture about LLM consciousness has real-world consequences, and every time you assert the question is unknowable you allow that kind of loophole, so it’d behoove all of us for you to spend some time actually digging in on this instead of just idly making or rebutting assertions. There’s a richer literature here than what you’ve seemed to have engaged with, and I’d encourage you to spend some time with it before handing more money to the magic AI people.
eloisius 2mo ago
Some tools like code rabbit (PR review bot) encourage you to do this. I couldn’t believe I found myself replying to code review comments to explain to an AI why we would rather let an exception crash the app than to catch and hide it several times so that it would stick in its memory. Having to interact with bots as if they are humans, especially when they are gate keeping, is degrading.
- hypfer 2mo ago
  Not just coderabbit (though it is a bad offender). GitHub's copilot review feature is an equally miserable experience. That one loves talking in imperatives, regardless of the fact that it is a clueless machine. FWIW, this tells you a lot about both the culture behind who built these things but also about the people that enable this stuff and don't immediately nope out. From that perspective, it's a low price to pay to learn whose judgement to never trust again.
- drfloyd51 2mo ago
  You shouldn’t be interacting like it is a human. It is a LLM. And if the interaction happens to be in a form of prose similar to how you talk to other humans, that is coincidental.
itemize123 2mo ago
i dont understand, isn't the arguing just some form prompt steering?
- wccrawford 2mo ago
  That's my thought. I've been using Matt Pocock's wayfinder and grilling skills, and I quite often end up in what would be considered "arguing" if done with another person. It's one-sided. It says something wrong, I correct it. I find that the AI, like many programmers, likes things to be really solid and over-engineered. For a project that needs that, it's already pretty great. For my shopping list app that I tried creating with it, it was absolutely ridiculous. I ended up "blowing up" on it multiple times, impressing upon it the seriousness with which I meant things. Even with MP's skills adding that kind of context to written files, it still kept trying to scope creep the crap out of the project.
pjerem 2mo ago
To me it’s evident that we are a few years away from the Her movie, where everyone on the street is talking to its IA friend. I’m really afraid that it will totally destruct what is remaining of social tissue because why search for friends when you have an always on virtual (and pretty smart) friend h24 in your earbuds ? I’m not blaming anyone for this outcome. I have myself argued with Claude more than once, and really not about code but about everyday things or nice facts of life I should rather have discussed with a friend.
- hypfer 2mo ago
  I wouldn't worry about that too much. I would worry about it, but not too much. People (generally speaking) also eventually stop eating just fastfood. Not all, but many. So I think we can have some faith in the self-regulation of others.
  - pjerem 2mo ago
    Well, I have bad news : https://p.kagi.com/proxy/bja35kxoyryc1.png?c=TklOzPjLPioJ5YMJT75bSo-AD0Xfbci2j5mOD4CAVgQGeRC2MGtAGpDOqcKxjZRn https://p.kagi.com/proxy/bja35kxoyryc1.png?c=TklOzPjLPioJ5YM...
    - zombot 2mo ago
      That link just runs an infinite reloading loop in my browser. Bad news indeed.
      - pjerem 2mo ago
        Ah yes, my bad : https://ourworldindata.org/cdn-cgi/imagedelivery/qLq-8BTgXU8yG0N6HnOy8g/9f102a64-ed24-4350-4456-7d669a666400/public https://ourworldindata.org/cdn-cgi/imagedelivery/qLq-8BTgXU8...
phoghed 2mo ago
> I've gently tried to explain it's like arguing with your compiler for giving you an incoherent error message - it's pointless. Unless doing so changed the compiler output, which is what happens when you say different things to an LLM.
- JohnMakin 2mo ago
  code changes change the compiler output.
  - charlieyu1 2mo ago
    And arguing changes LLM responses, especially when they are wrong
deleted 2mo ago
[deleted]
threethirtytwo 2mo ago
No. You’re the one who is ignorant. The LLM is a black box. We do not, in fact know how it works. We know the learning algorithm and we know the scaffold of the transformer network, but the end result of all the weights interacting with each other is something we do not understand. We do not understand this anymore than we understand the human brain. Now from this the best technical answer we can give is that we don’t know whether the LLM understands or is conscious. But you have to realize that same lack of understanding applies to humans. From a technical standpoint, You cannot say whether your best friend is conscious or not for the same technical reasons as to why you cannot say the same for the LLM. You don’t in actuality know anything. So, when we have a machine that produces output identical and indistinguishable from an intelligent entity it is actually reasonable to call it conscious, because we already do this for humans. There is no other factor involved. What is clear is that the LLM isn’t human… there is enough evidence to show that its nature is extremely alien. But to say it doesn’t understand or it isn’t self aware is not something anyone can definitively make a statement about other then the fact that it BEHAVES and communicates in a virtually indistinguishable way from something that is self aware. HN is full of arm chair experts who think they know what they are talking about. But you guys actually don’t. HN was wrong about AI and self driving cars, now we have Waymo. 10 years of research produced self driving cars that are 10x safer than humans. HN was wrong about LLMs. In the beginning HN was sure all it could do was write slop bootstrapped code… now it writes code for all of us. More than the general public HN has been making wrong predictions and wrong statements about AI with an authority that is outright ludicrous. We need to stop. It’s embarrassing how wrong we’ve been.

alexey-salmin 2mo ago

> It doesn't understand, it can't understand, and even if it could, you arguing with it isn't going to make it "learn" or act differently. I know people who are like that too. I'm not sure anthropomorphizing is a problem. Seeing analogies everywhere is an innate human trait, sometimes it can be harmful but more often it's useful.

danaris 2mo ago
> I know people who are like that too. This is part of the problem being described. You are part of the problem. "Some people are bad at X" is not comparable—is not even in the same category—as "LLMs are fundamentally incapable of X". Every human (at least to a first approximation) is capable of understanding, of learning, of remembering things, of doing math, of counting the number of "r"s in "strawberry". What you are observing is that some humans are careless, do not take the time and effort to understand, or have internalized the idea that they're "not smart enough" or "not the type of person" who understands things like <whatever>. That has nothing remotely to do with the fact that LLMs have no consciousness, no self-awareness, no cognition, no understanding. At a fundamental level.
- JohnMakin 2mo ago
  I don't think anyone in this conversation is saying this behavior is anything but the fault of the user not understanding how these tools work? This is a weirdly aggressive post.
  - danaris 2mo ago
    This is an extremely common fallacy I've seen lots and lots of people fall into with respect to LLMs. In nearly every case, they use the fact that "some humans can't do X" to claim that LLMs are, in fact, basically conscious/human-like/AGI already. This is deeply untrue, and is highly likely to lead them to bad conclusions about what we can and should do with LLMs.
    - throw310822 2mo ago
      > This is an extremely common fallacy ... they use the fact that "some humans can't do X" to claim that LLMs are, in fact, basically conscious/human-like/AGI already. Claiming that LLMs are conscious or human-like because humans can't do X seems a very strange way to argue for LLM intelligence. Usually, it goes the other way around: an LLM sceptic says "LLMs are dumb because they can't do X" and soon someone has to remind them that also most of the population can't, in fact, do X.
      - randomImmigrant 2mo ago
        “ Usually, it goes the other way around: an LLM sceptic says "LLMs are dumb because they can't do X" and soon someone has to remind them that also most of the population can't, in fact, do X.” And it should not go this way, is the OPs point. For an analogy, imagine someone looked at a bunch of lightbulbs and expresses dissatisfaction that they aren’t really stars. If someone replies by saying “not all stars are equally bright”, do you think that fact should carry any weight in the argument?
        throw310822 2mo ago
        > And it should not go this way, is the OPs point Well, he's wrong. If you argue that LLMs and humans are fundamentally different because all LLMs do X and no human does it, then showing you that it's not true demolishes your argument. Doesn't prove anything positive, but it certainly proves that your argument is invalid.
tavavex 2mo ago
Anthropomorphizing is a problem when you're talking about treating something that's not living as if it were. Using humanizing language invites discussions of things like the rights and feelings of an algorithm. A judge that is misled by the application of human-centric language to an algorithm can lead to some terrible outcomes. Not everyone is an LLM expert and the language people use leads to them treating LLMs like actual, real humans. That is terrifying.
- salawat 2mo ago
  What is terrifying is the propensity of people hoping for a mechanical slave to do everything possible to avoid touching on the possibility for being-ness of the technology they are desperately hoping will work as a basis for that implementation.
  - hypfer 2mo ago
    Cyberpsychosis. A next token predictor is not a being. Stop posting these things. Stop thinking these things.
    - Eisenstein 2mo ago
      Why can't it be a being? Why is thinking about such a possibilty bad?
      - hypfer 2mo ago
        Sorry, but if you're genuinely asking this not for trolling reasons then you should _really_ _really_ see a medical professional (not an insult). Online comment sections are not the correct place to unpack any of this. Which is indeed terminating this comment chain, but for good (and benevolent) reason. Doing anything else other than referring to a trained professional in a controlled context would potentially just feed delusions, which is highly unethical. Might not even be yours but those of another reader.
        2mo ago
hypfer 2mo ago
> I know people who are like that too. Cool. Question: Why _on earth_ would we make more of them?
qsera 2mo ago
>I'm not sure anthropomorphizing is a problem. It is a problem because it does not really understand stuff. For example, if you ask a human "Do you understand that doing X will kill you 100%?". If the human answers "Yes", then you can expect the human to act according to that understanding. That they will not do X But an LLM will happily acknowledges the consequences of doing X, but will still proceed to do X. So replace the human in the above example with a robot controlled by an LLM. There is no guarantee that it will not do X.
- wccrawford 2mo ago
  Have you not met humans that do that, too? When it was found that smoking causes cancer, many, many people kept doing it. Then the government taxed the crap out of it, and some people still do it. It's easy to say that people have other motivations, but doesn't the AI, too? You tell it not to do X, but you've also told it to do something that would benefit from X. It's going to "want" to still do X, to support that other thing. It's also got all the "knowledge" that enables it to do the work in the first place, and all of the tendencies of the people who do that work, because that's what it's trained on. It's really easy to anthropomorphize AI because it was literally modeled after people. And for the record, as lead developer, I've had actual humans that reported me to go ahead and do things I specifically told them not to.
  - qsera 2mo ago
    >smoking causes cancer Which part of "X will kill you 100%" did you not get?

traverseda 2mo ago

Do you have a working definition of "conscious"?

JohnMakin 2mo ago
Do you? I’m not sure what you’re getting at.
- nostrademons 2mo ago
  Not GP, but one of the challenges with debating whether LLMs are "conscious" is that we don't even really know what it means for a human to be "conscious", or even if consciousness is experienced by other humans the same way it is for ourselves. What we do know: neurons carry electrical impulses across their synapses to trigger other neurons to fire, and more frequently used synapses are strengthened while infrequently used ones are pruned. This is not all that dissimilar to how a multi-layer perceptron is trained: it's floating point numbers in a big matrix rather than biological structures and electrical impulses, but there is still that element of frequently used connections being strengthened and infrequently used ones being pruned. What we hypothesize but do not know: there is a thin brain structure of grey matter called the claustrum that has tendrils that reach into nearly every other brain structure. In many ways, this is similar to the attention mechanism of the transformer architecture. It is hypothesized that this may be the seat of consciousness, owing to experiments where electrical stimulation of the claustrum caused patients to immediately lose consciousness. However, there is no way to prove this, owing to the difficulty of otherwise removing or disabling the most connected structure in the brain and observing its effect on consciousness without permanently killing the patient. Beyond that, we don't know much. I've got a family friend that's been a practicing therapist for 50 years, and I asked him what was the most interesting observation he made in his career. It was that "Everybody experiences the world in a different way, and yet everybody assumes that everyone else experiences the world the same way they do."
  - Dilettante_ 2mo ago
    >"Everybody experiences the world in a different way, and yet everybody assumes that everyone else experiences the world the same way they do." Relevant article: "Generalizing from one example"[https://www.lesswrong.com/posts/baTWMegR42PAsH9qJ/generalizing-from-one-example https://www.lesswrong.com/posts/baTWMegR42PAsH9qJ/generalizi...] Personal note: That principle has been the bane of my autistic existence. People sometimes seem literally incapable of understanding that other people even can be different.

red75prime 2mo ago

> if it could, you arguing with it isn't going to make it "learn" or act differently. Are you talking about a specific harness that doesn't have context retention mechanisms? For example, ChatGPT with disabled memory feature? Or in general where "it" is a fixed-weights network? The latter is trivially true, of course.

JohnMakin 2mo ago
Even claude with “memory” enabled isn’t really “remembering” anything. It just injects it into the context and you hope it happens to find it relevant in its attention mechanisms, and then remembers to actually act on it. Anthropic’s own documentation states claude can and will ignore/truncate these. It’s a context trick, nothing approaching actual “memory,” and in fact, arguing with it will make a bunch of memory files, sometimes contradictory, and clutter up the context and act even worse.
- red75prime 2mo ago
  I'm not a big fan of arguments like "it's not the real [human quality], it's [mechanistic explanation]." They lack a part: "because the [human quality] allows us to do X, Y, Z, which is impossible with [this mechanism]." I agree that the relevance of retrieved pieces and the management of long-term storage could be improved, though.
  - JohnMakin 2mo ago
    I just don’t find it really relevant to the argument presented I guess. I disable auto memory and have my own mechanisms and infrastructure with how my agentic system “knows” and “remembers” things which is roughly an automated, sometimes self-correcting working index on the file system. It behaves much better than claude’s automated “memory” system, so I use that, but digging into how that worked and making something of my own just makes me really dismissive of comparing it to something like actual memory, so I apologize if it came off dismissive.
    - 2mo ago

Jtarii 2mo ago

>Because plenty of people, even ones that should know better, really believe it's a conscious, thinking entity You need evidence to make the positive claim that LLMs do not posses any form of consciousness.

roughly 2mo ago
Well, our current set of evidence is that it’s a mechanistic mechanical algorithm with an RNG embedded in it and we can both get it to repeatedly produce the same output for the same input and also get it to repeatedly do absolutely nothing at all, which are not characteristics we usually find in objects evincing consciousness. LLMs bear absolutely none of the traits we’ve come to recognize as the external hallmarks of consciousness in biological organisms, nor anything that would seem analogous in a non-biological substrate. That said, we don’t have a rigorous definition of consciousness that includes the actual phenomenology of consciousness, so I daresay if you’re going to go around asserting the LLM is conscious despite all existing evidence to the contrary, I think the impetus is on you to define some version of consciousness that isn’t also satisfied by a book or a movie.
- Jtarii 2mo ago
  I would agree that LLMs aren't much like the human brain, that doesn't prove that consciousness is not occurring. Does a fruit fly experience anything? If a microscopic insect can experience something, why can't a CPU? >if you’re going to go around asserting the LLM is conscious despite all existing evidence to the contrary Well there is neither any evidence that suggests LLMs are not conscious, and I also never asserted that they are. If I had to guess I would say that any information processing system will produce some kind of conscious experience, but I ultimately have literally no idea.
  - roughly 2mo ago
    There’s plenty. First, when we say “LLM”, what are we referring to? What is the entity that would be conscious in this case? The reason this is important is because powerful people are currently trying to use the dodge that LLMs are conscious to launder liability for their own policy choices, so the sloppy thinking and half-assed conjecture about LLM consciousness has real-world consequences, and every time you assert the question is unknowable you allow that kind of loophole, so it’d behoove all of us for you to spend some time actually digging in on this instead of just idly making or rebutting assertions. There’s a richer literature here than what you’ve seemed to have engaged with, and I’d encourage you to spend some time with it before handing more money to the magic AI people.

eloisius 2mo ago

Some tools like code rabbit (PR review bot) encourage you to do this. I couldn’t believe I found myself replying to code review comments to explain to an AI why we would rather let an exception crash the app than to catch and hide it several times so that it would stick in its memory. Having to interact with bots as if they are humans, especially when they are gate keeping, is degrading.

hypfer 2mo ago
Not just coderabbit (though it is a bad offender). GitHub's copilot review feature is an equally miserable experience. That one loves talking in imperatives, regardless of the fact that it is a clueless machine. FWIW, this tells you a lot about both the culture behind who built these things but also about the people that enable this stuff and don't immediately nope out. From that perspective, it's a low price to pay to learn whose judgement to never trust again.
drfloyd51 2mo ago
You shouldn’t be interacting like it is a human. It is a LLM. And if the interaction happens to be in a form of prose similar to how you talk to other humans, that is coincidental.

itemize123 2mo ago

i dont understand, isn't the arguing just some form prompt steering?

wccrawford 2mo ago
That's my thought. I've been using Matt Pocock's wayfinder and grilling skills, and I quite often end up in what would be considered "arguing" if done with another person. It's one-sided. It says something wrong, I correct it. I find that the AI, like many programmers, likes things to be really solid and over-engineered. For a project that needs that, it's already pretty great. For my shopping list app that I tried creating with it, it was absolutely ridiculous. I ended up "blowing up" on it multiple times, impressing upon it the seriousness with which I meant things. Even with MP's skills adding that kind of context to written files, it still kept trying to scope creep the crap out of the project.

pjerem 2mo ago

To me it’s evident that we are a few years away from the Her movie, where everyone on the street is talking to its IA friend. I’m really afraid that it will totally destruct what is remaining of social tissue because why search for friends when you have an always on virtual (and pretty smart) friend h24 in your earbuds ? I’m not blaming anyone for this outcome. I have myself argued with Claude more than once, and really not about code but about everyday things or nice facts of life I should rather have discussed with a friend.

hypfer 2mo ago
I wouldn't worry about that too much. I would worry about it, but not too much. People (generally speaking) also eventually stop eating just fastfood. Not all, but many. So I think we can have some faith in the self-regulation of others.
- pjerem 2mo ago
  Well, I have bad news : https://p.kagi.com/proxy/bja35kxoyryc1.png?c=TklOzPjLPioJ5YMJT75bSo-AD0Xfbci2j5mOD4CAVgQGeRC2MGtAGpDOqcKxjZRn https://p.kagi.com/proxy/bja35kxoyryc1.png?c=TklOzPjLPioJ5YM...
  - zombot 2mo ago
    That link just runs an infinite reloading loop in my browser. Bad news indeed.
    - pjerem 2mo ago
      Ah yes, my bad : https://ourworldindata.org/cdn-cgi/imagedelivery/qLq-8BTgXU8yG0N6HnOy8g/9f102a64-ed24-4350-4456-7d669a666400/public https://ourworldindata.org/cdn-cgi/imagedelivery/qLq-8BTgXU8...

phoghed 2mo ago

> I've gently tried to explain it's like arguing with your compiler for giving you an incoherent error message - it's pointless. Unless doing so changed the compiler output, which is what happens when you say different things to an LLM.

JohnMakin 2mo ago
code changes change the compiler output.
- charlieyu1 2mo ago
  And arguing changes LLM responses, especially when they are wrong

deleted 2mo ago

[deleted]

threethirtytwo 2mo ago

No. You’re the one who is ignorant. The LLM is a black box. We do not, in fact know how it works. We know the learning algorithm and we know the scaffold of the transformer network, but the end result of all the weights interacting with each other is something we do not understand. We do not understand this anymore than we understand the human brain. Now from this the best technical answer we can give is that we don’t know whether the LLM understands or is conscious. But you have to realize that same lack of understanding applies to humans. From a technical standpoint, You cannot say whether your best friend is conscious or not for the same technical reasons as to why you cannot say the same for the LLM. You don’t in actuality know anything. So, when we have a machine that produces output identical and indistinguishable from an intelligent entity it is actually reasonable to call it conscious, because we already do this for humans. There is no other factor involved. What is clear is that the LLM isn’t human… there is enough evidence to show that its nature is extremely alien. But to say it doesn’t understand or it isn’t self aware is not something anyone can definitively make a statement about other then the fact that it BEHAVES and communicates in a virtually indistinguishable way from something that is self aware. HN is full of arm chair experts who think they know what they are talking about. But you guys actually don’t. HN was wrong about AI and self driving cars, now we have Waymo. 10 years of research produced self driving cars that are 10x safer than humans. HN was wrong about LLMs. In the beginning HN was sure all it could do was write slop bootstrapped code… now it writes code for all of us. More than the general public HN has been making wrong predictions and wrong statements about AI with an authority that is outright ludicrous. We need to stop. It’s embarrassing how wrong we’ve been.