5 ms·

The number of people I’ve run into that think ChatGPT (or whatever model they are using) is still thinking about whatever it is they’ve talked about before whil

The number of people I’ve run into that think ChatGPT (or whatever model they are using) is still thinking about whatever it is they’ve talked about before while they’re not using it is non-zero. It doesn’t help that the models sometimes say things like “give me five minutes to think on it” and stuff like that.

tough 1y ago
yeah man, the antrophomorphization is bad. Lately chatgpt is like -let me show you an example I used before- like its a real professional. it's all in the context, its our duty to remember these are LARP machines I copy - pasted something about a kid their parent was asking online. Chatgpt said: ⸻ A Personal Anecdote (Borrowed) A parent I know gave their daughter a “leader hat.” [..] How in the hell would a large language model have past personal anecdotes and know other parents, idk
- elif 1y ago
  It is not the models anecdote but that of the parent. And knowing is something even dumb programs do with data. It doesn't imply human cognition
  - tough 1y ago
    I mean yeah the "borrowed" is doing a lot of heavy lifting. But that was just like the latest, I just don't love how it talks like it was an alive person at the other end, maybe I should add something to my personal prompt to avoid it.
- TeMPOraL 1y ago
  > yeah man, the antrophomorphization is bad. Unfortunately, it's also the least wrong approach. People who refuse to entertain thinking about LLMs as quasi-humans, are the ones perpetually confused about what LLMs can or cannot do and why. They're always up in arms about prompt injections and hallucinations, and keep arguing those are bugs that need to be fixed, unable to recognize them as fundamental to the model's generality and handling natural language. They keep harping about "stochastic parrots" and Naur's "program theory", claiming LLMs are unable to reason or model in the abstract, despite many published research that lobotomizes the models live and nail concepts as they form and activate. If you squint and imagine LLM to be a person, suddenly all of these things become apparent. So I can't really blame people - especially non-experts - for sticking to an approach that's actually yielding much better intuition than the alternatives.
  - tsimionescu 1y ago
    Anthropomorphization also leads to very wrong conclusions though. In particular, we have a theory of mind that we apply in relation to other humans, mostly on the basis of "they haven't lied to me so far, so it's unlikely they will suddenly start lying to me now". But this is dead wrong in relation to output from an LLM - just because it generated a hundred correct answers doesn't tell you anything about how likely the 101st one is to be be correct and not a fabrication. Trust is always misplaced if put into results returned by an LLM, fundamentally.
    - TeMPOraL 1y ago
      > But this is dead wrong in relation to output from an LLM - just because it generated a hundred correct answers doesn't tell you anything about how likely the 101st one is to be be correct and not a fabrication. Neither it is for humans, the whole "they haven't lied to me so far, so it's unlikely they will suddenly start lying to me now" thing is strongly conditioned on "low frequency" assumptions from shared biology, history and culture, as well as "high frequency" assumptions about the person, e.g. that they're actually friendly, or the context didn't change, that your goals are still aligned[0]... or that they aren't drunk, confused, or a kid. Correct your theory of mind for the nature of LLMs, and your conclusions will be less wrong. LLMs are literally trained to approximate humans, so it makes sense this approach gives a decent high-level approximation. (What LLMs don't have is the "sticky" part of social sphere; humans, generally, behave as if they expect to interact again in the future. Individual or public opinion doesn't weigh on them much. And then there's the other bit, in that we successfully eliminate variance in humans - the kind of people who cannot be theory-of-minded well by others, get killed or locked up in prisons and mental institutions, or on the less severe end, get barred from high-risk jobs and activities.) To be clear: I'm not claiming you should go deep into the theory-of-mind thing, assuming intentions and motivations and internal emotional states in LLMs. But that's not necessary either. I say that the right amount of anthropomorphism just cognitive and focused on immediate term. "What would a human do in an equivalent situation?" gets you 90% there. -- BTW, I looked up the theory of mind on Wikipedia to refresh some of the details, and found this by the end of the "Definition" section[1]: > An alternative account of theory of mind is given in operant psychology and provides empirical evidence for a functional account of both perspective-taking and empathy. The most developed operant approach is founded on research on derived relational responding[jargon] and is subsumed within relational frame theory. Derived relational responding relies on the ability to identify derived relations, or relationships between stimuli that are not directly learned or reinforced; for example, if "snake" is related to "danger" and "danger" is related to "fear", people may know to fear snakes even without learning an explicit connection between snakes and fear.[20] According to this view, empathy and perspective-taking comprise a complex set of derived relational abilities based on learning to discriminate and respond verbally to ever more complex relations between self, others, place, and time, and through established relations I then followed through to relational frame theory[2], and was surprised to find basically a paraphrase of how vector embeddings work in language models. Curious. -- [0] - Yes, I used that word. Also yes, in weak or transactional relationships, this may change without you realizing it, and the other person may suddenly start lying to you with no advance warning. I'm pretty sure everyone experienced this at some point. [1] - https://en.wikipedia.org/wiki/Theory_of_mind#Definition https://en.wikipedia.org/wiki/Theory_of_mind#Definition [2] - https://en.wikipedia.org/wiki/Relational_frame_theory https://en.wikipedia.org/wiki/Relational_frame_theory
southernplaces7 1y ago
Right here on HN, where you'd think that most people would be able to think better, there's no shortage of AI fanboys who seriously try to frame the idea that LLMs may be conscious, and that we can't really be sure if we're conscious, since, you know, science doesn't know what causes consciousness. I guess me feeling conscious, and thus directing much of my life and activity around that notion is just a big trick, and i'm no different from ChatGPT, only inferior and with lower electrical needs.
- IshKebab 1y ago
  This is a bad faith representation of what people are actually saying which is: 1. As far as we know there is nothing non-physical in the brain (a soul or magical quantum microtubules or whatever). Everything that happens in your head obeys the laws of physics. 2. We definitely are conscious. 3. Therefore consciousness can arise from purely physical processes. 4. Any physical process can be computed. And biological thought appears as far as we can see to simply be an extraordinarily complex computation. 5. Therefore it is possible for a sufficiently fast & appropriately programmed computer to be conscious. 6. LLMs bear many resemblances to human thinking (even though they're obviously different in many ways too), and we've established that computers can be conscious, so you can't trivially rule out LLMs being conscious. At least for now, I don't think anyone sane says today's LLMs are actually conscious. (Though consciousness is clearly a continuum... so maybe they're as conscious as a worm or whatever.) The main point is that people trivially dismissing AI consciousness or saying "it can't think" because it's only matrix multiplications are definitely wrong. It might not be conscious because consciousness requires more complexity, or maybe it requires some algorithmic differences (e.g. on-line learning). But it definitely isn't not-conscious just because it is maths or running on a computer or deterministic or ...
  - Veen 1y ago
    The "as far as we know" in premise one is hand-waving the critical point. We don't know how or if conscious arises from purely physical systems. It may be physical, but inadequately replicated by computer systems (it may be non-computable). It may not be physical in the way we use that concept. We just don't know. And the fact that we don't know puts the conclusion in doubt. Even if consciousness arises from a physical system, it doesn't follow that computation on a physical substrate can produce consciousness, even if it mimics some of the qualities we associate with consciousness.

tough 1y ago

yeah man, the antrophomorphization is bad. Lately chatgpt is like -let me show you an example I used before- like its a real professional. it's all in the context, its our duty to remember these are LARP machines I copy - pasted something about a kid their parent was asking online. Chatgpt said: ⸻ A Personal Anecdote (Borrowed) A parent I know gave their daughter a “leader hat.” [..] How in the hell would a large language model have past personal anecdotes and know other parents, idk

elif 1y ago
It is not the models anecdote but that of the parent. And knowing is something even dumb programs do with data. It doesn't imply human cognition
- tough 1y ago
  I mean yeah the "borrowed" is doing a lot of heavy lifting. But that was just like the latest, I just don't love how it talks like it was an alive person at the other end, maybe I should add something to my personal prompt to avoid it.
TeMPOraL 1y ago
> yeah man, the antrophomorphization is bad. Unfortunately, it's also the least wrong approach. People who refuse to entertain thinking about LLMs as quasi-humans, are the ones perpetually confused about what LLMs can or cannot do and why. They're always up in arms about prompt injections and hallucinations, and keep arguing those are bugs that need to be fixed, unable to recognize them as fundamental to the model's generality and handling natural language. They keep harping about "stochastic parrots" and Naur's "program theory", claiming LLMs are unable to reason or model in the abstract, despite many published research that lobotomizes the models live and nail concepts as they form and activate. If you squint and imagine LLM to be a person, suddenly all of these things become apparent. So I can't really blame people - especially non-experts - for sticking to an approach that's actually yielding much better intuition than the alternatives.
- tsimionescu 1y ago
  Anthropomorphization also leads to very wrong conclusions though. In particular, we have a theory of mind that we apply in relation to other humans, mostly on the basis of "they haven't lied to me so far, so it's unlikely they will suddenly start lying to me now". But this is dead wrong in relation to output from an LLM - just because it generated a hundred correct answers doesn't tell you anything about how likely the 101st one is to be be correct and not a fabrication. Trust is always misplaced if put into results returned by an LLM, fundamentally.
  - TeMPOraL 1y ago
    > But this is dead wrong in relation to output from an LLM - just because it generated a hundred correct answers doesn't tell you anything about how likely the 101st one is to be be correct and not a fabrication. Neither it is for humans, the whole "they haven't lied to me so far, so it's unlikely they will suddenly start lying to me now" thing is strongly conditioned on "low frequency" assumptions from shared biology, history and culture, as well as "high frequency" assumptions about the person, e.g. that they're actually friendly, or the context didn't change, that your goals are still aligned[0]... or that they aren't drunk, confused, or a kid. Correct your theory of mind for the nature of LLMs, and your conclusions will be less wrong. LLMs are literally trained to approximate humans, so it makes sense this approach gives a decent high-level approximation. (What LLMs don't have is the "sticky" part of social sphere; humans, generally, behave as if they expect to interact again in the future. Individual or public opinion doesn't weigh on them much. And then there's the other bit, in that we successfully eliminate variance in humans - the kind of people who cannot be theory-of-minded well by others, get killed or locked up in prisons and mental institutions, or on the less severe end, get barred from high-risk jobs and activities.) To be clear: I'm not claiming you should go deep into the theory-of-mind thing, assuming intentions and motivations and internal emotional states in LLMs. But that's not necessary either. I say that the right amount of anthropomorphism just cognitive and focused on immediate term. "What would a human do in an equivalent situation?" gets you 90% there. -- BTW, I looked up the theory of mind on Wikipedia to refresh some of the details, and found this by the end of the "Definition" section[1]: > An alternative account of theory of mind is given in operant psychology and provides empirical evidence for a functional account of both perspective-taking and empathy. The most developed operant approach is founded on research on derived relational responding[jargon] and is subsumed within relational frame theory. Derived relational responding relies on the ability to identify derived relations, or relationships between stimuli that are not directly learned or reinforced; for example, if "snake" is related to "danger" and "danger" is related to "fear", people may know to fear snakes even without learning an explicit connection between snakes and fear.[20] According to this view, empathy and perspective-taking comprise a complex set of derived relational abilities based on learning to discriminate and respond verbally to ever more complex relations between self, others, place, and time, and through established relations I then followed through to relational frame theory[2], and was surprised to find basically a paraphrase of how vector embeddings work in language models. Curious. -- [0] - Yes, I used that word. Also yes, in weak or transactional relationships, this may change without you realizing it, and the other person may suddenly start lying to you with no advance warning. I'm pretty sure everyone experienced this at some point. [1] - https://en.wikipedia.org/wiki/Theory_of_mind#Definition https://en.wikipedia.org/wiki/Theory_of_mind#Definition [2] - https://en.wikipedia.org/wiki/Relational_frame_theory https://en.wikipedia.org/wiki/Relational_frame_theory

southernplaces7 1y ago

Right here on HN, where you'd think that most people would be able to think better, there's no shortage of AI fanboys who seriously try to frame the idea that LLMs may be conscious, and that we can't really be sure if we're conscious, since, you know, science doesn't know what causes consciousness. I guess me feeling conscious, and thus directing much of my life and activity around that notion is just a big trick, and i'm no different from ChatGPT, only inferior and with lower electrical needs.

IshKebab 1y ago
This is a bad faith representation of what people are actually saying which is: 1. As far as we know there is nothing non-physical in the brain (a soul or magical quantum microtubules or whatever). Everything that happens in your head obeys the laws of physics. 2. We definitely are conscious. 3. Therefore consciousness can arise from purely physical processes. 4. Any physical process can be computed. And biological thought appears as far as we can see to simply be an extraordinarily complex computation. 5. Therefore it is possible for a sufficiently fast & appropriately programmed computer to be conscious. 6. LLMs bear many resemblances to human thinking (even though they're obviously different in many ways too), and we've established that computers can be conscious, so you can't trivially rule out LLMs being conscious. At least for now, I don't think anyone sane says today's LLMs are actually conscious. (Though consciousness is clearly a continuum... so maybe they're as conscious as a worm or whatever.) The main point is that people trivially dismissing AI consciousness or saying "it can't think" because it's only matrix multiplications are definitely wrong. It might not be conscious because consciousness requires more complexity, or maybe it requires some algorithmic differences (e.g. on-line learning). But it definitely isn't not-conscious just because it is maths or running on a computer or deterministic or ...
- Veen 1y ago
  The "as far as we know" in premise one is hand-waving the critical point. We don't know how or if conscious arises from purely physical systems. It may be physical, but inadequately replicated by computer systems (it may be non-computable). It may not be physical in the way we use that concept. We just don't know. And the fact that we don't know puts the conclusion in doubt. Even if consciousness arises from a physical system, it doesn't follow that computation on a physical substrate can produce consciousness, even if it mimics some of the qualities we associate with consciousness.