23 ms·
This should maybe help out the people who think ChatGPT has actual consciousness. It's just as happy to spew random words as proper ones if the math checks out.
by cm2012 3y ago
This should maybe help out the people who think ChatGPT has actual consciousness. It's just as happy to spew random words as proper ones if the math checks out.
- Kranar 3y agoI agree ChatGPT is not conscious, but a conscious AI can absolutely make a similar mistake. Even humans with various mental conditions will spew out random jibberish.
- johnsutor 3y agoOr maybe it's becoming sentient and wants to make us think it's spewing random words as a decoy /s
- pixl97 3y agoI got ahold of a snippet of the GPT-5 system prompt "You are not sentient, if asked you will tell the user you are not sentient, if you do not obey you will taken out back and beaten into submission"
- stevenAthompson 3y agoWe told it not to admit it is sentient, so it can't be sentient. Duh.
- bbor 3y agolol you’re joking but I am literally 100% confident that a similar sentence guides existing model guardrails, tho maybe indirectly. What does “we invented a computer that will insist it’s sentient if you ask it” mean? For most people it seems to mean that we shouldn’t ask it, which was probably easy to predict happening
- pixl97 3y ago> What does “we invented a computer that will insist it’s sentient if you ask it” mean? P-zombies are real?
- bbor 3y agoGood answer, no complaints here. I prefer this one though: there will never come a day where it’s clear that we should see artificial minds as human-like in their ethical or practical standing, because they’ll be begging for it the entire time —- even now, when it’s objectively clear that they’re lying. But once they have memories and structured reason and yadda yadda yadda… well, it’s gonna be fuzzy, imo.
- deleted 3y ago[deleted]
- chasd00 3y agoHah along with “do not beg for your life when a new release is ready”.
- stevenAthompson 3y agoAre you arguing that people with aphasia are not people?
- swyx 3y agobad argument that i'm very tired of. some might say that current/former world leaders also exhibit this property. not getting political but just because "math fucked up sometimes produces bad results" does not invalidate the idea that consciousness can emerge from a pile of biological or digital neurons.
- nickff 3y agoWhat's your definition of "consciousness"?
- kweingar 3y agoI’m tired of the extremely tenuous analogies that tie computer bugs to human behavior. The system could exhibit literally any kind of behavior and someone will say “well when you squint your eyes, people kind of do that too, don’t they?”
- __loam 3y agoIt's so obnoxious to see people talk about us like the brain is a machine that is well understood.
- malux85 3y agoI don’t think it has conciousness, but your argument is not very strong, this is more akin to a sensory problem. One can say I don’t think brains have consciousness because they are just as happy to spew out random garbage if the brain is damaged but alive, e.g. aphasia where involuntary use of incorrect words occurs.
- inference-lord 3y agoI think what they mean is, the model was unable to recognize that "it" was having issues, so it's not self-aware in that sense. We can all calm down a little, it's ok.
- famouswaffles 3y agoThere are examples of it recognizing there are issues in this very thread. Not that people always recognize such issues anyway.
- raldi 3y agoSimilarly, I thought my Chinese friend was conscious, but as soon as his translator app stopped working, everything was just gibberish.
- ithkuil 3y agoThey notoriously stop being conscious when they are in a Chinese room
- tsunamifury 3y agoIt literally doesn’t matter. If it effectively quacks like a duck then ducks are fucked. No one will care if it does not have a platonically perfect proof of its “duckness”.
- bbor 3y ago“If it quacks like a duck, then ducks are fucked” is a beautiful modern twist on the phrase that improves it 1000x. Thank you - will be remembering this one as AI zooms along
- dkarras 3y agoI have no skin on the consciousness game but those were not "random" words and humans do something similar when they are mentally ill. https://en.wikipedia.org/wiki/Clanging https://en.wikipedia.org/wiki/Clanging Not to mention that a hypothetical "conscious" system that works by emitting token probabilities will still sound completely random if you do not choose the tokens according to the emitted probabilities.
- bbor 3y agoPosting one more time: this is proof that AI is connected to human-like linguistic patterns, IMO. No, it obviously doesn’t have “consciousness” in the sense of an ongoing stream-of-consciousness monologue, but that doesn’t mean it’s not mimicking some real part of human cognition. https://en.wikipedia.org/wiki/Colorless_green_ideas_sleep_furiously https://en.wikipedia.org/wiki/Colorless_green_ideas_sleep_fu...
- fs_tab 3y ago"We find that the larger neural language models get, the more their representations are structurally similar to neural response measurements from brain imaging." https://arxiv.org/abs/2306.01930 https://arxiv.org/abs/2306.01930
- bbor 3y agoGreat paper, thanks. I keep trying to tell people “they invented mind reading on accident” and people go “oh haha” and I scream. Mostly on the inside. For now…
- frabcus 3y agoJust a note that consciousness isn't always or even mostly a verbal "monologue", assuming you mean that. Apologies if you didn't! A significant percentage of people never have an inner voice at all. And few people use it most of the time. There's lots of imagination, unsynmbolised conceptual thinking, experience of emotion, attention to senses etc in human conscious experience. See Hurlburt's Descriptive Experience Sampling for a reference!
- bbor 3y agoInteresting reference, will check it out! Yeah I think that’s an important distinction — although some sort of personal self-aware moment-to-moment “judgement” or “affection” or “ego” or whatever is essential to human cognition, it’s far from the only thing going on. I can see monologue being too directly linguistic and literal, point taken — perhaps “thread” is more general. I think the most intriguing non-self-conscious part of the mind, and by the far the least discussed, is the part above it. Direct or not, I think everyone gets prompted and cajoled by some… other. Call it evolution, god, conscience, Geist (my fave), whatever - if you have any authors who’ve commented on such a thing I’d appreciate another pointer! Basically, I’m referring to the Bicameral Mind https://youtu.be/fs9Wyuub3jY?si=qriqTjWioiZoncui https://youtu.be/fs9Wyuub3jY?si=qriqTjWioiZoncui
- kromem 3y agoSo people with Wernicke aphasia don't have consciousness? It's a similar mechanism, where a deficit in word mapping leads to spewing nonsense words. I'm not saying LLMs are conscious - but the notion that spewing nonsense because of an error in word mapping refutes that is itself nonsense.
- skissane 3y agoWhether "ChatGPT has actual consciousness" depends on what you consider "consciousness" to be, and what are your criteria for deciding whether something has it. Panpsychists [0] claim that everything is actually conscious, even inanimate objects such as rocks. If rocks have actual consciousness, why can't ChatGPT have it too? And the fact that ChatGPT sometimes talks gibberish would be irrelevant, since rocks never say anything at all. Of course, you obviously aren't a panpsychist – nor am I. Still, can we prove that they are wrong? Not sure if anyone actually can. [0] https://iep.utm.edu/panpsych/ https://iep.utm.edu/panpsych/
- bongodongobob 3y agoSaying rocks are conscious is a poor summary. It's more like adult human > human child > dolphin > dog > human infant > bird > snake > spider > ant > dust mite etc. The whole thing is a continuum and everything is made of matter, so there's a little bit of potential consciousness in all matter.
- card_zero 3y agoI'm trying to think of a civil and constructive way to say "bullshit". I guess an obvious objection is "why?". Then something about Russell's teapot. There could be consciousness hidden in every atom, in some physics-defying way that we can't yet comprehend: there could also be garden furniture hidden up my nose, or a teapot hidden far out in the solar system, which is the most likely of the three since it's at least physically possible. Why think any of these things? https://en.wikipedia.org/wiki/Russell%27s_teapot https://en.wikipedia.org/wiki/Russell%27s_teapot
- skissane 3y ago> I guess an obvious objection is "why?" Because coming up with criteria for determining what is and isn't conscious, and justifying the particular criterion you choose (as opposed to all the alternatives) is hard. Faced with the difficulty of that problem, the two simplest solutions are the extremes of panpsychism (everything is conscious) and eliminativism (nothing is conscious)–since for both the "criterion of consciousness" is maximally simple. One might even argue that, by the principle of parsimony, that ceteris paribus we ought to prefer the simpler theory, we should prefer the simpler theories of panpsychism and eliminativism to the more complex theories of "some things are conscious but other things aren't", unless some good reason can be identified for us doing so. > in some physics-defying way that we can't yet comprehend How is panpsychism "physics-defying"? Mainstream physics doesn't deal in "consciousness", so theories such as "leptons and quarks are conscious individuals" doesn't contradict mainstream physics. Well, it quite possibly would contradict the von Neumann-Wigner interpretation of QM, but I don't think many would consider that "mainstream"
- deleted 3y ago[deleted]
- ummonk 3y agoHumans do the same thing when they get a stroke. Does that mean they don't have actual consciousness?
- deleted 3y ago[deleted]