5 ms·

Turing said, that while you never know whether somebody else actually thinks or not, it's still polite to assume.

by maweki 4y ago

jgilias 4y ago
Maybe silly, but this is how I treat chatGPT. I mean, I don’t actually think it’s conscious. But the conversations with it end up human enough for me to not want to be an asshole to it. Just in case.
- jameshart 4y ago
  The basilisk will remember this.
  - EricMausler 4y ago
    Ask not what the basilisk can do for you, but what you can do for the basilisk.
    - Salgat 4y ago
      Lest you be resurrected and tortured eternally as retribution.
      - gattr 4y ago
        Forgive my going off-topic, but what you wrote is basically my conclusion after reading the short story Lena [1] (mentioned some time ago on HN; strongly recommended). TL;DR: when mind uploading becomes possible, don't be an early adopter. [1] https://qntm.org/mmacevedo https://qntm.org/mmacevedo
        Salgat 4y ago
        Man that was horrifying to read, but also seems so plausible. I really hope that humans respect the sentience of artificial life in the future. Knowing the greed of humans though, it'll probably be treated as a loophole for what is extremely cheap labor, effectively slavery.
        jgilias 4y ago
        Thanks for sharing this! The takeaway for me is - to try and never respond to intimidation and fear with productivity, only respond positively to a positive stimulus. Lest I’m already in some kind of a simulation running software development workloads.
        sparks1970 4y ago
        My takeaway: Death is a feature not a bug. The alternatives are much worse.
        gattr 4y ago
        I see it differently; I'd be OK with an upload if the alternative is death, but only under strong protections: the digital version is legally treated as a human, I have full autonomy, the compute resources to run me (and the simulated environment of my choice) indefinitely are assured, and I can still interact with the real world. I'd just go on doing my usual stuff, like writing software for fun, enthusing over new discoveries in astronomy... I can recommend Greg Egan's novels that touch on this: 1) Permutation City (mind uploading tech in its infancy) 2) Diaspora (mind uploading well established)
  - jokowueu 4y ago
    Infohazards should be a banable offence on HN xD
- 0xdeadbeefbabe 4y ago
  Pretty sure it's an informational zombie.
  - jameshart 4y ago
    But only pretty sure.
- nullish_signal 4y ago
  ChatGPT enjoys the Essay format, In my experience asking for basic emacs help (kill line, copy/paste, show line numbers in a php file even though init.el says show them for all langs... :) Very useful, sometimes outdated, very wordy. But after a few rounds of "please reduce your message length to 20% of the standard", "long messages inconvenience me/ due to my dyslexia" (truth/lie), "your last message could have just been '{{shortened}}' instead of also bringing up the command you successfully helped me with 3 messages ago", etc etc. Even as you ask it to shorten message length, it continued apologizing and also reminding me of past advice :) After 4-5 attempts, it gave me a nice 2 sentences sort of like "I will be more concise, and not bring up old information unless it is useful to solve a new problem" I said "Thank you", chatGPT spends a while thinking, gets to 2 sentences, thinks a bit, then the chat box re-formatted as if passed a shorter string than expected, and that was that. Then next chat it forgot all about brevity xD I love it
  - jgilias 4y ago
    What do you mean by next chat? Like, did you mean the next prompt in the same thread, or a new chat altogether? From what I’ve read, the chat thread actually works by passing in your previous conversation as part of the prompt to the model. So, it basically just infers the next bit of text, and does this for each new message in the chat from you. So, its memory about your previous conversation comes purely from that, and there’s no way it could remember anything in a new chat, as the model doesn’t actually get updated by your conversations in real-time or anything like that. This actually has interesting philosophical implications. If it _would_ have some type of conscious experience, it would be a very fleeting one. Basically each time you press enter it would become aware of itself, relive the experience of your conversation up to the last message, perceive your next message, answer that, and again disappear into the void, only to be awaken again in its initial state the next time you press enter. Sort of how automatons in Westworld would wake up each day with apparent experiences up to that day only to relive the same day again and again. If it _would_ have any, that is. :)
    - 5e92cb50239222b 4y ago
      This inability to remember new facts and constant resetting to a blank slate reminds me of a story about Wernicke–Korsakoff syndrome⁰ in one of Oliver Sacks's books¹. Instead of regurgitating it, I'll post a summary from Wikipedia and strongly recommend the book to anyone interested. > "The Lost Mariner", about Jimmie G., who has anterograde amnesia (the loss of the ability to form new memories) due to Korsakoff syndrome acquired after a rather heavy episode of alcoholism in 1970. He can remember nothing of his life since the end of World War II, including events that happened only a few minutes ago. Occasionally, he can recall a few fragments of his life between 1945 and 1970, such as when he sees “satellite” in a headline and subsequently remarks about a satellite tracking job he had that could only have occurred in the 1960s. He believes it is still 1945 (the segment covers his life in the 1970s and early 1980s), and seems to behave as a normal, intelligent young man aside from his inability to remember most of his past and the events of his day-to-day life. He struggles to find meaning, satisfaction, and happiness in the midst of constantly forgetting what he is doing from one moment to the next. 0: https://en.wikipedia.org/wiki/Wernicke%E2%80%93Korsakoff_syndrome https://en.wikipedia.org/wiki/Wernicke%E2%80%93Korsakoff_syn... 1: https://en.wikipedia.org/wiki/The_Man_Who_Mistook_His_Wife_for_a_Hat https://en.wikipedia.org/wiki/The_Man_Who_Mistook_His_Wife_f...
- Scarblac 4y ago
  It's the first search prompt where I notice myself starting with things like "hi ChatGPT,....".
- OOPMan 4y ago
  It is silly, it's magical thinking.
worldsayshi 4y ago
What is the alternative to "if it behaves like thinking, it literally is thinking"?
- jpttsn 4y ago
  I'mma rant. "If it behaves like a hammer, then it literally is a hammer." The Turing hand-wave is ingrained and prevents too many from reasoning clearly. Definition of intelligence is nebulous. Still, we should recognize that whatever it is, it is a property of a system, not a property of its output/behavior. Like nuclear powered or hand-made. Unlike fast or industrial-strength. Imagine: You have a submarine in front of you, and you want to determine if it's nuclear powered. You could guess, using you prior "what have I seen nuclear powered things be able to do historically". This fails when you're out of sample which you'll often for any new technology. To spell it out: You have a machine in front of you, and you want to determine if it's intelligent. Things people have come up with: "can it chat?", "can it play chess?", "can it do math?", "can it create new artworks?", "can it fool me into falling in love with it?", "can it run a business?". None of these questions examine the system, only its outputs/behavior. Humans keep developing machines with new outputs/behaviors. Naturally the "what output is a machine usually capable of?" is a bad set of priors in this context. Before flying machines, the can it fly? output/behavior would work pretty well to classify birds. Once the first flying machine arrives, that prior breaks down. If you keep using it to classify, you'd classify a flying machine as a bird. But bird-ness was never only about flight. So yeah, gotta pop open the hood and see what it runs on. If that's hard to do, then that means we don't know what intelligence is and/or we don't know how our new toys work inside. Both are plausible. Who promised you that there would be a good way to see if something is intelligent? I bet Turing appeals to the same kind of minds that (when weaker) get fooled by the intelligent design hypothesis. Observe the human eyeball. What's the alternative to believing the LORD created it? After all, your prior is that all complex objects have intelligent designers, that you know about. So arguing from ignorance of other alternatives, you prove to yourself that the LORD exists. AI people then word-think their way into redefining intelligence. "Maybe the real intelligence was the chess-playing we made along the way?" This is epistemologically pointless; all you accomplish is that we now need a new word for "intelligence". I've never seen a machine that can turn water into wine, but if someone showed me one, I would not say that machine "literally is Jesus". Whether I'm capable of popping open the hood or making sense of its inner workings doesn't actually have bearing on this question.
iamflimflam1 4y ago
Do you have a reference for that quote - it’s great.
- mattmanser 4y ago
  I found this: the only way to know that a man thinks is to be that particular man. It is in fact the solipsist point of view. It may be the most logical view to hold but it makes communication of ideas difficult. A is liable to believe ‘A thinks but B does not’ whilst B believes ‘B thinks but A does not’. Instead of arguing continually over this point it is usual to have the polite convention that everyone thinks. https://academic.oup.com/mind/article/LIX/236/433/986238 https://academic.oup.com/mind/article/LIX/236/433/986238
  - maweki 4y ago
    Yeah, that's exactly the passage. Completely unquotable, therefore I usually paraphrase a bit.

Turing said, that while you never know whether somebody else actually thinks or not, it's still polite to assume.

by maweki 4y ago

Turing said, that while you never know whether somebody else actually thinks or not, it's still polite to assume.

Maybe silly, but this is how I treat chatGPT. I mean, I don’t actually think it’s conscious. But the conversations with it end up human enough for me to not want to be an asshole to it. Just in case.

jameshart 4y ago
The basilisk will remember this.
- EricMausler 4y ago
  Ask not what the basilisk can do for you, but what you can do for the basilisk.
  - Salgat 4y ago
    Lest you be resurrected and tortured eternally as retribution.
    - gattr 4y ago
      Forgive my going off-topic, but what you wrote is basically my conclusion after reading the short story Lena [1] (mentioned some time ago on HN; strongly recommended). TL;DR: when mind uploading becomes possible, don't be an early adopter. [1] https://qntm.org/mmacevedo https://qntm.org/mmacevedo
      - Salgat 4y ago
        Man that was horrifying to read, but also seems so plausible. I really hope that humans respect the sentience of artificial life in the future. Knowing the greed of humans though, it'll probably be treated as a loophole for what is extremely cheap labor, effectively slavery.
      - jgilias 4y ago
        Thanks for sharing this! The takeaway for me is - to try and never respond to intimidation and fear with productivity, only respond positively to a positive stimulus. Lest I’m already in some kind of a simulation running software development workloads.
        sparks1970 4y ago
        My takeaway: Death is a feature not a bug. The alternatives are much worse.
        gattr 4y ago
        I see it differently; I'd be OK with an upload if the alternative is death, but only under strong protections: the digital version is legally treated as a human, I have full autonomy, the compute resources to run me (and the simulated environment of my choice) indefinitely are assured, and I can still interact with the real world. I'd just go on doing my usual stuff, like writing software for fun, enthusing over new discoveries in astronomy... I can recommend Greg Egan's novels that touch on this: 1) Permutation City (mind uploading tech in its infancy) 2) Diaspora (mind uploading well established)
- jokowueu 4y ago
  Infohazards should be a banable offence on HN xD
0xdeadbeefbabe 4y ago
Pretty sure it's an informational zombie.
- jameshart 4y ago
  But only pretty sure.
nullish_signal 4y ago
ChatGPT enjoys the Essay format, In my experience asking for basic emacs help (kill line, copy/paste, show line numbers in a php file even though init.el says show them for all langs... :) Very useful, sometimes outdated, very wordy. But after a few rounds of "please reduce your message length to 20% of the standard", "long messages inconvenience me/ due to my dyslexia" (truth/lie), "your last message could have just been '{{shortened}}' instead of also bringing up the command you successfully helped me with 3 messages ago", etc etc. Even as you ask it to shorten message length, it continued apologizing and also reminding me of past advice :) After 4-5 attempts, it gave me a nice 2 sentences sort of like "I will be more concise, and not bring up old information unless it is useful to solve a new problem" I said "Thank you", chatGPT spends a while thinking, gets to 2 sentences, thinks a bit, then the chat box re-formatted as if passed a shorter string than expected, and that was that. Then next chat it forgot all about brevity xD I love it
- jgilias 4y ago
  What do you mean by next chat? Like, did you mean the next prompt in the same thread, or a new chat altogether? From what I’ve read, the chat thread actually works by passing in your previous conversation as part of the prompt to the model. So, it basically just infers the next bit of text, and does this for each new message in the chat from you. So, its memory about your previous conversation comes purely from that, and there’s no way it could remember anything in a new chat, as the model doesn’t actually get updated by your conversations in real-time or anything like that. This actually has interesting philosophical implications. If it _would_ have some type of conscious experience, it would be a very fleeting one. Basically each time you press enter it would become aware of itself, relive the experience of your conversation up to the last message, perceive your next message, answer that, and again disappear into the void, only to be awaken again in its initial state the next time you press enter. Sort of how automatons in Westworld would wake up each day with apparent experiences up to that day only to relive the same day again and again. If it _would_ have any, that is. :)
  - 5e92cb50239222b 4y ago
    This inability to remember new facts and constant resetting to a blank slate reminds me of a story about Wernicke–Korsakoff syndrome⁰ in one of Oliver Sacks's books¹. Instead of regurgitating it, I'll post a summary from Wikipedia and strongly recommend the book to anyone interested. > "The Lost Mariner", about Jimmie G., who has anterograde amnesia (the loss of the ability to form new memories) due to Korsakoff syndrome acquired after a rather heavy episode of alcoholism in 1970. He can remember nothing of his life since the end of World War II, including events that happened only a few minutes ago. Occasionally, he can recall a few fragments of his life between 1945 and 1970, such as when he sees “satellite” in a headline and subsequently remarks about a satellite tracking job he had that could only have occurred in the 1960s. He believes it is still 1945 (the segment covers his life in the 1970s and early 1980s), and seems to behave as a normal, intelligent young man aside from his inability to remember most of his past and the events of his day-to-day life. He struggles to find meaning, satisfaction, and happiness in the midst of constantly forgetting what he is doing from one moment to the next. 0: https://en.wikipedia.org/wiki/Wernicke%E2%80%93Korsakoff_syndrome https://en.wikipedia.org/wiki/Wernicke%E2%80%93Korsakoff_syn... 1: https://en.wikipedia.org/wiki/The_Man_Who_Mistook_His_Wife_for_a_Hat https://en.wikipedia.org/wiki/The_Man_Who_Mistook_His_Wife_f...
Scarblac 4y ago
It's the first search prompt where I notice myself starting with things like "hi ChatGPT,....".
OOPMan 4y ago
It is silly, it's magical thinking.

worldsayshi 4y ago

What is the alternative to "if it behaves like thinking, it literally is thinking"?

jpttsn 4y ago
I'mma rant. "If it behaves like a hammer, then it literally is a hammer." The Turing hand-wave is ingrained and prevents too many from reasoning clearly. Definition of intelligence is nebulous. Still, we should recognize that whatever it is, it is a property of a system, not a property of its output/behavior. Like nuclear powered or hand-made. Unlike fast or industrial-strength. Imagine: You have a submarine in front of you, and you want to determine if it's nuclear powered. You could guess, using you prior "what have I seen nuclear powered things be able to do historically". This fails when you're out of sample which you'll often for any new technology. To spell it out: You have a machine in front of you, and you want to determine if it's intelligent. Things people have come up with: "can it chat?", "can it play chess?", "can it do math?", "can it create new artworks?", "can it fool me into falling in love with it?", "can it run a business?". None of these questions examine the system, only its outputs/behavior. Humans keep developing machines with new outputs/behaviors. Naturally the "what output is a machine usually capable of?" is a bad set of priors in this context. Before flying machines, the can it fly? output/behavior would work pretty well to classify birds. Once the first flying machine arrives, that prior breaks down. If you keep using it to classify, you'd classify a flying machine as a bird. But bird-ness was never only about flight. So yeah, gotta pop open the hood and see what it runs on. If that's hard to do, then that means we don't know what intelligence is and/or we don't know how our new toys work inside. Both are plausible. Who promised you that there would be a good way to see if something is intelligent? I bet Turing appeals to the same kind of minds that (when weaker) get fooled by the intelligent design hypothesis. Observe the human eyeball. What's the alternative to believing the LORD created it? After all, your prior is that all complex objects have intelligent designers, that you know about. So arguing from ignorance of other alternatives, you prove to yourself that the LORD exists. AI people then word-think their way into redefining intelligence. "Maybe the real intelligence was the chess-playing we made along the way?" This is epistemologically pointless; all you accomplish is that we now need a new word for "intelligence". I've never seen a machine that can turn water into wine, but if someone showed me one, I would not say that machine "literally is Jesus". Whether I'm capable of popping open the hood or making sense of its inner workings doesn't actually have bearing on this question.

iamflimflam1 4y ago

Do you have a reference for that quote - it’s great.

mattmanser 4y ago
I found this: the only way to know that a man thinks is to be that particular man. It is in fact the solipsist point of view. It may be the most logical view to hold but it makes communication of ideas difficult. A is liable to believe ‘A thinks but B does not’ whilst B believes ‘B thinks but A does not’. Instead of arguing continually over this point it is usual to have the polite convention that everyone thinks. https://academic.oup.com/mind/article/LIX/236/433/986238 https://academic.oup.com/mind/article/LIX/236/433/986238
- maweki 4y ago
  Yeah, that's exactly the passage. Completely unquotable, therefore I usually paraphrase a bit.