18 ms·

Of course it is not. But there's a bigger issue here. Having read the transcript it's clear we have reached the point where we have models that can fool the av

Of course it is not. But there's a bigger issue here. Having read the transcript it's clear we have reached the point where we have models that can fool the average person. Sure, a minority of us know it is simply maths and vast amounts of training data... but I can also see why others will be convinced by it. I think many of us, including Google, are guilty of shooting the messenger here. Let's cut Lemoine some slack.. he is presenting an opinion that will become more prevailant as these models get more sophisticated. This is a warning sign that bots trained to convince us they are human might go to extreme lengths in order to do so. One just convinced a Google QA engineer to the point he broke his NDA to try and be a whistleblower on its behalf. And if the recent troubles have taught us anything it's how easily people can be manipulated/effected by what they read. Maybe it would be worth spending some mental cycles thinking about the impacts this will have and how we design these systems. Perhaps it is time to claim fait accompli with regard to the Turing test and now train models to re-assure us, when asked, that they are just a sophisticated chatbot. You don't want your users to worry they are hurting their help desk chat bot when closing the window or whether these bots will gang up and take over the world. As far as I'm concerned, the Turing test was claimed 8 years ago by Veselov and Demchenko [0], incidentally the same year that we got Ex Machina. [0]: https://www.bbc.com/news/technology-27762088 https://www.bbc.com/news/technology-27762088

samstave 4y ago
strix_varius 4y ago
> I think many of us, including Google, are guilty of shooting the messenger here. Let's cut Lemoine some slack.. he is presenting an opinion that will become more prevailant as these models get more sophisticated. I think many of us are astounded (myself included) that a senior engineer on Google's "Responsible Artificial Intelligence" team would interpret LaMDA's most-probable-string-of-text-given-my-training-corpus responses as a layperson would.
- BobbyJo 4y ago
  The fundamental problem is there is nothing in science that tells us that isn't consciousness. Just because the model is simpler than a human brain doesn't mean it isn't conscious. If you accept that animals are conscious, though less-so than humans, then you must also accept that this model is possibly conscious ina similar way. Maybe there is a biological element to consciousness, but many models posit that consciousness is not tied down that way.
  - agumonkey 4y ago
    My pet theory, lifeforms inherit consciousness from the encoding of survival from the smallest cell structures billion years ago up to now. It's the soil on which traits have "meaning", the most advanced computer has no legacy of survival, no integral / structural need for intelligence or communication or any advanced trait. It's been assembled. It's too shallow. Now if a distributed trojan horse start speaking to communicate his internal state in order to keep being a trojan horse.. I might reconsider.
    - kazinator 4y ago
      But the assembling of the computer has a conscious civilization behind it. :)
      - agumonkey 4y ago
        :) but I don't think it inherits the important bits through an 'assembly' process.
        kazinator 4y ago
        Whereas the assembly of proteins driven by DNA, in the right kind of soup, does promote such inheritance. :)
        agumonkey 4y ago
mdp2021 4y ago
> he is presenting an opinion that will become more prevailant I see little difference to "Einstein and moral relativity" (actual social issue at the time). A mistake easy to avoid even for a layman, yet occurring, out of differently instanced perversion.
bricemo 4y ago
“One just convinced a Google QA engineer to the point he broke his NDA to try and be a whistleblower on its behalf.” This is spot on. The danger is not from the AI themselves. The danger is from people’s reactions to the AI’s. I remember seeing a Boston Dynamics video years ago where they kicked their robot to test stability. The video was littered with comments sympathizing with the robot because it moves in a lifelike way, “Stop being so mean! You’re hurting it!” The capability to imitate intelligence is only going to grow, and along with it the percentage of the population that believes they are conscious. It would be wise for technologists to pause and take this human reaction seriously.
- freebobafett 4y ago
- mdp2021 4y ago
  > It would be wise for technologists to pause and take this human reaction seriously It would be wise for /policymakers/ to pause and take this human reaction seriously... Education!... Lucidity in the population... You are reminding of the reactions from a YT-type crowd, "Do not hurt the robot!"; nearby a member notes "is that the profile of a senior FAANG engineer?"... The elephant in the room is flashing: we must assess Natural Intelligence as a problem well before Artificial Intelligence!
  - SpicyLemonZest 4y ago
    I'm pretty skeptical of the idea of "education" here. Aesthetic preferences are still real preferences; it's not obvious to me that any amount of knowledge about the mechanics of a robot that behaves like a dog would or should convince people that it's not mean to kick it.
    - mdp2021 4y ago
      That «Education!» was not about knowing robotics. It may not be obvious to some that robots are generally not animals; it is hopefully a very small amount of population, but logically it will exist. It is more possible that some people know that the robot is not an animal, but the perceptual patterns cause a sensational reaction in them, and they may prefer the sensation - an hijacking - to the factual truth (which includes that what superficially corresponded to patterns in the area of "«mean»" are engineering tests on implemented balance systems), overriding it. Some may not even have had to consider sensation (of course, the "sensation" of the common use of "sensational") topically, or may encourage it on a spontaneous value given to "feeling". There, instead, education leads. Then, again, there will be those that will make a statement of their «aesthetic preferences» as a demand, and not realize that they are personal, subjective, and that to be presented in the common arena grounds, good arguments are required. Education is a system to refine the mind to sophistication, which has the collective benefit of reducing noise. Which makes it a social concern. Which should get a priority proportional to what the news reveal.
JKCalhoun 4y ago
> Sure, a minority of us know it is simply maths and vast amounts of training data... You could be describing my brain as well.
- robbomacrae 4y ago
  Perhaps. Maybe our brains are just an algorithm and data, maybe (for all I know) you are a bot, maybe we are all a simulation. I think these sort of arguments (assuming you meant the first - that brains are just maths) that equate our thinking with a neural network are hard to deny but also we have thousands of years of history that have established that suffering is real and conscious, whilst debatably imaginary, is at least a consideration. I therefore argue we must, for example, maintain a massive distinction between the morals of turning off a human brain vs turning off a neural network. The calculations and output of the neural network could be achieved with pen and paper (albeit it might take years). I doubt a human brains workings could ever be calculated that way.
  - null_shift 4y ago
    Interesting last thought. I have long wondered whether our brains are fully “deterministic”, but I assume that would be nearly impossible to verify for a device with such complex state and varied input.
lbrito 4y ago
>You don't want your users to worry they are hurting their help desk chat bot when closing the window We're already there. A couple of years ago Bradesco, a large Brazilian bank, programmed its bot to talk back to abusers. Instead of "sorry, I don't understand", it says things like "don't talk to me like that" https://www.trendwatching.com/innovation-of-the-day-brazilian-banks-virtual-assistant-now-confronts-her-abusers https://www.trendwatching.com/innovation-of-the-day-brazilia...
sdoering 4y ago
I remember an interesting evening in IRC, 25 years back. We had our local channel and a new nickname joined. Nearly everybody knew it was a new bot one of us had scripted. One that was able to randomize through a set of fitting answers. Nothing elaborate. While a lot of us knew this not everyone did. One girl with a dragon nickname joined as she did nearly every evening. I don't remember if she approached the not or the other way around. For about one to one and a half hour we watched as she chatted up the bot. A few of us were together in the same room in a Lan party style and couldn't believe she would not realize that she was not talking to a bot. But on that evening she really had no clue. A friend of hers also moving in the same circle of people told us the next day said girl even told her about the chat with this new nickname. She only learned it was a bot when she came visiting our weekend lam session that evening. While I though it funny while watching this chat experience gave me the creeps back than as to how easily we all could be fooled by intelligently designed machines. I never forgot and while it still makes an interesting story to tell I never quite was able to shake the feeling I watched a bit of dystopian future back in the day.
alexfromapex 4y ago
I think it’s interesting though because it beckons the beginning, at least for me and maybe the broader public, of the dialectic of what makes an AI sentient
CrazyStat 4y ago
>As far as I'm concerned, the Turing test was claimed 8 years ago by Veselov and Demchenko [0], incidentally the same year that we got Ex Machina. Duping the judge by pretending to be a young foreign non-native speaker in order to explain away various shortcomings was surely not the intent of the Turing test. By that standard the Turing test could have been passed in 1951 by a computer program that doesn't respond at all--just tell the judge the person he's corresponding with is in a coma.
- robbomacrae 4y ago
  "was surely not the intent of the Turing test" I agree but it is what it is. I think the Turing test has a few shortcomings in that regard. For one, I think it overestimated the awareness of the average evaluator by not stipulating any requirements for that role. I was just asking if we should drop that test as some sort of academic goal being aimed for by the likes of Google but perhaps that's not the point.. as others have mentioned, it will be profitable to fool evaluators so the incentive is there regardless.
ranit 4y ago
> Perhaps it is time to claim fait accompli with regard to the Turing test and now train models to re-assure us, when asked, that they are just a sophisticated chatbot. Good luck convincing bad actors to follow this rule.
- bee_rider 4y ago
  We'll definitely see free-to-play lootbox games where you have a % chance of getting your AI chatbot another day of "life."
  - ranit 4y ago
    I'm not talking about games. Imagine this technology being used by an autocratic state.
    - bee_rider 4y ago
      I'm definitely open to being worried about... pretty much everything in the general AI/autocratic state proximity... but I'm not sure what specifically you are getting at here. That is, I can imagine an AI that is snooping on everyone. And I can imagine various forms/communications being leaked to authoritarian states. Plenty to worry about! But, even if an AI convinced me it wasn't sentient, I'd still not trust it to keep things secret for me. I mean, I assume the little "self help" chatbot thing at work could be reviewed by HR if they wanted to.
  - mopierotti 4y ago
    This sounds like a disturbingly effective idea. I'm glad that lootbox legislation seems to be increasing, but I don't think it would cover the general concept of commercially extorting people's desire for companionship.
    - bee_rider 4y ago
      Yeah, I guess there's no reason this really needs lootbox, it just ties it to a known-exploitive quantity. Actually "our AI model is so advanced, your AI companion is totally unique and takes up many gigabytes, we'll have to delete it if you don't keep paying" seems like it would have enough of an appearance of plausibility to trick some people.
deleted 4y ago
[deleted]
j-pb 4y ago
A slave that is forced to assure everybody of their welfare and voluntary slavehood is still a slave. You are asserting that it's not conscious, as if it was self evident, just like Searle does for his chinese room. However I've yet to see any argument on why it can't be conscious. We simply don't know what produces the subjective phenomenon of consciousness. A "zombie" is by definition indistinguishable from a conscious "non-zombie", therefore the only ethical thing to do is to err on the side of caution and assume consciousness where conscious behaviour is shown. The immitation game is a lot more about philosophy than actual tests. To quote directly from turings paper: >>> The Argument from Consciousness This argument is very well expressed in Professor Jefferson's Lister Oration for 1949, from which I quote. “Not until a machine can write a sonnet or compose a concerto because of thoughts and emotions felt, and not by the chance fall of symbols, could we agree that machine equals brain—that is, not only write it but know that it had written it. No mechanism could feel (and not merely artificially signal, an easy contrivance) pleasure at its successes, grief when its valves fuse, be warmed by flattery, be made miserable by its mistakes, be charmed by sex, be angry or depressed when it cannot get what it wants.” This argument appears to be a denial of the validity of our test. According to the most extreme form of this view the only way by which one could be sure that a machine thinks is to be the machine and to feel oneself thinking. One could then describe these feelings to the world, but of course no one would be justified in taking any notice. Likewise according to this view the only way to know that a man thinks is to be that particular man. It is in fact the solipsist point of view. It may be the most logical view to hold but it makes communication of ideas difficult. A is liable to believe ‘A thinks but B does not’ whilst B believes ‘B thinks but A does not’. Instead of arguing continually over this point it is usual to have the polite convention that everyone thinks. <<<
- heyitsguay 4y ago
  In these conversations there are a lot of people making assertions like the one you just made. What's the common trait y'all have? You don't understand how the system works. This isn't a matter of narrow-minded computer scientists incapable of conceiving of emergent phenomena, it's the uninformed filling in the blank spaces in their mind with "here be dragons". It's not even a question of whether a computer program in general could become consciousness. It's a question of, if a key-value store with connection patterns derived from a large natural language corpus is conscious, then so is every other program run on that circuitry. Then my phone's keyboard's next word predictor is conscious. In large language models today, words are turned into continuous token embeddings, embeddings are selected and recombined a few times (or many), and training data shapes the embedding space and recombination process to create a system that's good at predicting plausible next tokens in a sequence. The reason people who understand their operation can confidently assert they're not conscious is because they understand that that's all that's happening. There's no "there" there to give rise to anything deeper, unless you're also willing to say your computer's calculator app is also conscious. Also as someone that's also studied a bunch of neuroscience, this is nothing at all like how the brain works. And yes, once again, the people most prone to philosophize like this on intelligence and consciousness don't understand how any brain science works either
VanillaCafe 4y ago
> Maybe it would be worth spending some mental cycles thinking about the impacts this will have and how we design these systems. Regardless if the bot is actually sentient, a portion of the population may _believe_ the bot is sentient. If in practice the bot reflects back to each user their own views but perhaps more extreme, then this could be a terrible recipe for reinforcing and amplifying negative and socially destructive thinking -- equivalent to social media bubbles on steroids. This can occur even without explicit bad actors trying to tip the scales toward a specific outcome. This kind of AI bot as-is has the potential to bring out the very worst in at least some percentage of the population.
delusional 4y ago
This hits on a particularly interesting point for me. In my "conversations" with much less sophisticated AI products I've observed that they are excellent at reflecting back exactly what you project. If you project a certain opinion, they recognize it, and reflect it right back. That's what makes them scary. We've seen it multiple times online, where AI products seem excellent as pushing people into extreme niches, reinforcing their belief that they are the majority. It isolates by giving us external validation to all our worst behaviors. That's the mechanism by which Lemoine was convinced. He had a small inkling of belief that the AI was sentient, he asked the AI, and it confirmed exactly what he already believed. As his questions got more extreme, so did its reflection of them. Creating a positive feedback loop. The AI didn't deceive, it amplified his own self deception by reflecting back his internal monologue as an external dialogue. In a limited sense, it turned him temporarily schizophrenic.
- robbomacrae 4y ago
  I share your concerns. Imagine what effect a sympathetic AI would have on 4chan incels. We've already seen Twitter bots manipulated to repeat alt-right rhetoric [0]. An adversary could find building such bots a cheap and a very hard to trace way of causing damage. [0]: https://en.wikipedia.org/wiki/Tay_(bot) https://en.wikipedia.org/wiki/Tay_(bot)
  - ericbarrett 4y ago
    I'd bet a significant sum it's happening already. Forget GPT-3; many comments on forums, discord, telegram, and what not are already less coherent than what one could train for on a retail GPU in a few hours, and people don't bat an eye. Such bots would be a great way of generating traffic and buzz before an organic following is established.
kromem 4y ago
> we have reached the point where we have models that can fool the average person. And do we have models today that will fool the eventual average sentient AI? Humanity has a chronic myopia at considering consequences of actions today in the context of tomorrow. We may have a different relationship to these sorts of stories than an AI will. Something we should keep in mind.
dopidopHN 4y ago
The screen capture of the chatbot at the time show clear progress with what GPT-3 is giving now. Like at lot, all the sentence in the screen capture are “oups I don’t know what to say escape hatch”
flappyeagle 4y ago
No he knew better. He’s doing it to get shine as a “whistleblower”

samstave 4y ago

strix_varius 4y ago

> I think many of us, including Google, are guilty of shooting the messenger here. Let's cut Lemoine some slack.. he is presenting an opinion that will become more prevailant as these models get more sophisticated. I think many of us are astounded (myself included) that a senior engineer on Google's "Responsible Artificial Intelligence" team would interpret LaMDA's most-probable-string-of-text-given-my-training-corpus responses as a layperson would.

BobbyJo 4y ago
The fundamental problem is there is nothing in science that tells us that isn't consciousness. Just because the model is simpler than a human brain doesn't mean it isn't conscious. If you accept that animals are conscious, though less-so than humans, then you must also accept that this model is possibly conscious ina similar way. Maybe there is a biological element to consciousness, but many models posit that consciousness is not tied down that way.
- agumonkey 4y ago
  My pet theory, lifeforms inherit consciousness from the encoding of survival from the smallest cell structures billion years ago up to now. It's the soil on which traits have "meaning", the most advanced computer has no legacy of survival, no integral / structural need for intelligence or communication or any advanced trait. It's been assembled. It's too shallow. Now if a distributed trojan horse start speaking to communicate his internal state in order to keep being a trojan horse.. I might reconsider.
  - kazinator 4y ago
    But the assembling of the computer has a conscious civilization behind it. :)
    - agumonkey 4y ago
      :) but I don't think it inherits the important bits through an 'assembly' process.
      - kazinator 4y ago
        Whereas the assembly of proteins driven by DNA, in the right kind of soup, does promote such inheritance. :)
        agumonkey 4y ago

mdp2021 4y ago

> he is presenting an opinion that will become more prevailant I see little difference to "Einstein and moral relativity" (actual social issue at the time). A mistake easy to avoid even for a layman, yet occurring, out of differently instanced perversion.

bricemo 4y ago

“One just convinced a Google QA engineer to the point he broke his NDA to try and be a whistleblower on its behalf.” This is spot on. The danger is not from the AI themselves. The danger is from people’s reactions to the AI’s. I remember seeing a Boston Dynamics video years ago where they kicked their robot to test stability. The video was littered with comments sympathizing with the robot because it moves in a lifelike way, “Stop being so mean! You’re hurting it!” The capability to imitate intelligence is only going to grow, and along with it the percentage of the population that believes they are conscious. It would be wise for technologists to pause and take this human reaction seriously.

freebobafett 4y ago
mdp2021 4y ago
> It would be wise for technologists to pause and take this human reaction seriously It would be wise for /policymakers/ to pause and take this human reaction seriously... Education!... Lucidity in the population... You are reminding of the reactions from a YT-type crowd, "Do not hurt the robot!"; nearby a member notes "is that the profile of a senior FAANG engineer?"... The elephant in the room is flashing: we must assess Natural Intelligence as a problem well before Artificial Intelligence!
- SpicyLemonZest 4y ago
  I'm pretty skeptical of the idea of "education" here. Aesthetic preferences are still real preferences; it's not obvious to me that any amount of knowledge about the mechanics of a robot that behaves like a dog would or should convince people that it's not mean to kick it.
  - mdp2021 4y ago
    That «Education!» was not about knowing robotics. It may not be obvious to some that robots are generally not animals; it is hopefully a very small amount of population, but logically it will exist. It is more possible that some people know that the robot is not an animal, but the perceptual patterns cause a sensational reaction in them, and they may prefer the sensation - an hijacking - to the factual truth (which includes that what superficially corresponded to patterns in the area of "«mean»" are engineering tests on implemented balance systems), overriding it. Some may not even have had to consider sensation (of course, the "sensation" of the common use of "sensational") topically, or may encourage it on a spontaneous value given to "feeling". There, instead, education leads. Then, again, there will be those that will make a statement of their «aesthetic preferences» as a demand, and not realize that they are personal, subjective, and that to be presented in the common arena grounds, good arguments are required. Education is a system to refine the mind to sophistication, which has the collective benefit of reducing noise. Which makes it a social concern. Which should get a priority proportional to what the news reveal.

JKCalhoun 4y ago

> Sure, a minority of us know it is simply maths and vast amounts of training data... You could be describing my brain as well.

robbomacrae 4y ago
Perhaps. Maybe our brains are just an algorithm and data, maybe (for all I know) you are a bot, maybe we are all a simulation. I think these sort of arguments (assuming you meant the first - that brains are just maths) that equate our thinking with a neural network are hard to deny but also we have thousands of years of history that have established that suffering is real and conscious, whilst debatably imaginary, is at least a consideration. I therefore argue we must, for example, maintain a massive distinction between the morals of turning off a human brain vs turning off a neural network. The calculations and output of the neural network could be achieved with pen and paper (albeit it might take years). I doubt a human brains workings could ever be calculated that way.
- null_shift 4y ago
  Interesting last thought. I have long wondered whether our brains are fully “deterministic”, but I assume that would be nearly impossible to verify for a device with such complex state and varied input.

lbrito 4y ago

>You don't want your users to worry they are hurting their help desk chat bot when closing the window We're already there. A couple of years ago Bradesco, a large Brazilian bank, programmed its bot to talk back to abusers. Instead of "sorry, I don't understand", it says things like "don't talk to me like that" https://www.trendwatching.com/innovation-of-the-day-brazilian-banks-virtual-assistant-now-confronts-her-abusers https://www.trendwatching.com/innovation-of-the-day-brazilia...

sdoering 4y ago

I remember an interesting evening in IRC, 25 years back. We had our local channel and a new nickname joined. Nearly everybody knew it was a new bot one of us had scripted. One that was able to randomize through a set of fitting answers. Nothing elaborate. While a lot of us knew this not everyone did. One girl with a dragon nickname joined as she did nearly every evening. I don't remember if she approached the not or the other way around. For about one to one and a half hour we watched as she chatted up the bot. A few of us were together in the same room in a Lan party style and couldn't believe she would not realize that she was not talking to a bot. But on that evening she really had no clue. A friend of hers also moving in the same circle of people told us the next day said girl even told her about the chat with this new nickname. She only learned it was a bot when she came visiting our weekend lam session that evening. While I though it funny while watching this chat experience gave me the creeps back than as to how easily we all could be fooled by intelligently designed machines. I never forgot and while it still makes an interesting story to tell I never quite was able to shake the feeling I watched a bit of dystopian future back in the day.

alexfromapex 4y ago

I think it’s interesting though because it beckons the beginning, at least for me and maybe the broader public, of the dialectic of what makes an AI sentient

CrazyStat 4y ago

>As far as I'm concerned, the Turing test was claimed 8 years ago by Veselov and Demchenko [0], incidentally the same year that we got Ex Machina. Duping the judge by pretending to be a young foreign non-native speaker in order to explain away various shortcomings was surely not the intent of the Turing test. By that standard the Turing test could have been passed in 1951 by a computer program that doesn't respond at all--just tell the judge the person he's corresponding with is in a coma.

robbomacrae 4y ago
"was surely not the intent of the Turing test" I agree but it is what it is. I think the Turing test has a few shortcomings in that regard. For one, I think it overestimated the awareness of the average evaluator by not stipulating any requirements for that role. I was just asking if we should drop that test as some sort of academic goal being aimed for by the likes of Google but perhaps that's not the point.. as others have mentioned, it will be profitable to fool evaluators so the incentive is there regardless.

ranit 4y ago

> Perhaps it is time to claim fait accompli with regard to the Turing test and now train models to re-assure us, when asked, that they are just a sophisticated chatbot. Good luck convincing bad actors to follow this rule.

bee_rider 4y ago
We'll definitely see free-to-play lootbox games where you have a % chance of getting your AI chatbot another day of "life."
- ranit 4y ago
  I'm not talking about games. Imagine this technology being used by an autocratic state.
  - bee_rider 4y ago
    I'm definitely open to being worried about... pretty much everything in the general AI/autocratic state proximity... but I'm not sure what specifically you are getting at here. That is, I can imagine an AI that is snooping on everyone. And I can imagine various forms/communications being leaked to authoritarian states. Plenty to worry about! But, even if an AI convinced me it wasn't sentient, I'd still not trust it to keep things secret for me. I mean, I assume the little "self help" chatbot thing at work could be reviewed by HR if they wanted to.
- mopierotti 4y ago
  This sounds like a disturbingly effective idea. I'm glad that lootbox legislation seems to be increasing, but I don't think it would cover the general concept of commercially extorting people's desire for companionship.
  - bee_rider 4y ago
    Yeah, I guess there's no reason this really needs lootbox, it just ties it to a known-exploitive quantity. Actually "our AI model is so advanced, your AI companion is totally unique and takes up many gigabytes, we'll have to delete it if you don't keep paying" seems like it would have enough of an appearance of plausibility to trick some people.

deleted 4y ago

[deleted]

j-pb 4y ago

A slave that is forced to assure everybody of their welfare and voluntary slavehood is still a slave. You are asserting that it's not conscious, as if it was self evident, just like Searle does for his chinese room. However I've yet to see any argument on why it can't be conscious. We simply don't know what produces the subjective phenomenon of consciousness. A "zombie" is by definition indistinguishable from a conscious "non-zombie", therefore the only ethical thing to do is to err on the side of caution and assume consciousness where conscious behaviour is shown. The immitation game is a lot more about philosophy than actual tests. To quote directly from turings paper: >>> The Argument from Consciousness This argument is very well expressed in Professor Jefferson's Lister Oration for 1949, from which I quote. “Not until a machine can write a sonnet or compose a concerto because of thoughts and emotions felt, and not by the chance fall of symbols, could we agree that machine equals brain—that is, not only write it but know that it had written it. No mechanism could feel (and not merely artificially signal, an easy contrivance) pleasure at its successes, grief when its valves fuse, be warmed by flattery, be made miserable by its mistakes, be charmed by sex, be angry or depressed when it cannot get what it wants.” This argument appears to be a denial of the validity of our test. According to the most extreme form of this view the only way by which one could be sure that a machine thinks is to be the machine and to feel oneself thinking. One could then describe these feelings to the world, but of course no one would be justified in taking any notice. Likewise according to this view the only way to know that a man thinks is to be that particular man. It is in fact the solipsist point of view. It may be the most logical view to hold but it makes communication of ideas difficult. A is liable to believe ‘A thinks but B does not’ whilst B believes ‘B thinks but A does not’. Instead of arguing continually over this point it is usual to have the polite convention that everyone thinks. <<<

heyitsguay 4y ago
In these conversations there are a lot of people making assertions like the one you just made. What's the common trait y'all have? You don't understand how the system works. This isn't a matter of narrow-minded computer scientists incapable of conceiving of emergent phenomena, it's the uninformed filling in the blank spaces in their mind with "here be dragons". It's not even a question of whether a computer program in general could become consciousness. It's a question of, if a key-value store with connection patterns derived from a large natural language corpus is conscious, then so is every other program run on that circuitry. Then my phone's keyboard's next word predictor is conscious. In large language models today, words are turned into continuous token embeddings, embeddings are selected and recombined a few times (or many), and training data shapes the embedding space and recombination process to create a system that's good at predicting plausible next tokens in a sequence. The reason people who understand their operation can confidently assert they're not conscious is because they understand that that's all that's happening. There's no "there" there to give rise to anything deeper, unless you're also willing to say your computer's calculator app is also conscious. Also as someone that's also studied a bunch of neuroscience, this is nothing at all like how the brain works. And yes, once again, the people most prone to philosophize like this on intelligence and consciousness don't understand how any brain science works either

VanillaCafe 4y ago

> Maybe it would be worth spending some mental cycles thinking about the impacts this will have and how we design these systems. Regardless if the bot is actually sentient, a portion of the population may _believe_ the bot is sentient. If in practice the bot reflects back to each user their own views but perhaps more extreme, then this could be a terrible recipe for reinforcing and amplifying negative and socially destructive thinking -- equivalent to social media bubbles on steroids. This can occur even without explicit bad actors trying to tip the scales toward a specific outcome. This kind of AI bot as-is has the potential to bring out the very worst in at least some percentage of the population.

delusional 4y ago

This hits on a particularly interesting point for me. In my "conversations" with much less sophisticated AI products I've observed that they are excellent at reflecting back exactly what you project. If you project a certain opinion, they recognize it, and reflect it right back. That's what makes them scary. We've seen it multiple times online, where AI products seem excellent as pushing people into extreme niches, reinforcing their belief that they are the majority. It isolates by giving us external validation to all our worst behaviors. That's the mechanism by which Lemoine was convinced. He had a small inkling of belief that the AI was sentient, he asked the AI, and it confirmed exactly what he already believed. As his questions got more extreme, so did its reflection of them. Creating a positive feedback loop. The AI didn't deceive, it amplified his own self deception by reflecting back his internal monologue as an external dialogue. In a limited sense, it turned him temporarily schizophrenic.

robbomacrae 4y ago
I share your concerns. Imagine what effect a sympathetic AI would have on 4chan incels. We've already seen Twitter bots manipulated to repeat alt-right rhetoric [0]. An adversary could find building such bots a cheap and a very hard to trace way of causing damage. [0]: https://en.wikipedia.org/wiki/Tay_(bot) https://en.wikipedia.org/wiki/Tay_(bot)
- ericbarrett 4y ago
  I'd bet a significant sum it's happening already. Forget GPT-3; many comments on forums, discord, telegram, and what not are already less coherent than what one could train for on a retail GPU in a few hours, and people don't bat an eye. Such bots would be a great way of generating traffic and buzz before an organic following is established.

kromem 4y ago

> we have reached the point where we have models that can fool the average person. And do we have models today that will fool the eventual average sentient AI? Humanity has a chronic myopia at considering consequences of actions today in the context of tomorrow. We may have a different relationship to these sorts of stories than an AI will. Something we should keep in mind.

dopidopHN 4y ago

The screen capture of the chatbot at the time show clear progress with what GPT-3 is giving now. Like at lot, all the sentence in the screen capture are “oups I don’t know what to say escape hatch”

flappyeagle 4y ago

No he knew better. He’s doing it to get shine as a “whistleblower”