6 ms·

"This rate"? What rate? We still have not the slightest indication that the programs we have created possess anything resembling consciousness, sapience, or se

"This rate"? What rate? We still have not the slightest indication that the programs we have created possess anything resembling consciousness, sapience, or sentience. The responses of the LLMs to prompts are absolutely the kinds of things we should expect from statistical models trained on large chunks of the Internet. Furthermore, given what we do have, it is vastly more likely that we will accidentally create a paperclip maximizer—an autonomous program that has no actual goals or agency of its own, only those that we give it, but given access to various virtual and physical systems in order to pursue those goals—than that we will accidentally create a fully sentient artificial life form. Please stop spreading this kind of breathless hype and misinformation about these machine-learning projects.

TheOtherHobbes 4y ago
They don't need consciousness, sapience, or sentience. They only need to mimic human behaviours accurately enough to make them redundant and automatable. That's an increasingly large range of behaviours. And the pace is clearly accelerating. There may or may not be an s-curve tail-off at some point. But if there is, we currently have no idea how to estimate where that point will be. I think a lot of people still don't realise that AI is absolutely an existential threat to our species. Currently it lives inside boxes we can turn off. But as soon as that stops being true - not because of experimentation, but because AI will be providing services and infrastructure we can't live without - the best we can hope for is some kind of symbiosis. The worst outcomes are far less reassuring.
- prox 4y ago
  Cue Agent Smiths’ monologue. That said, the way we treat our environment, ie anything external to us is a representation of an internal state. Not sure how the internal state of an AGI would be. We have many, many epistemological flaws and we do not wake up and say “Gee let’s examine my epistemology today!” Verifying what we believe, feel and reason is an ultimately worthwhile endeavor while it’s not really a cultural phenomenon.
- danaris 4y ago
  > They only need to mimic human behaviours accurately enough to make them redundant and automatable. The post I was replying to was specifically positing that they would have feelings.
  - sho_hn 4y ago
    What makes a feeling different or unapproachable by sophisticated mimicry? GPT models with bad guardrails already generate dramatic/emotional text because they internalized human narratives and propensity for drama from the training data (cf. the "Bing got upset" type articles from a month ago). Unless you mean to say it's not the same as the model having qualia. It's not clear to me whether we would know when that emerged in the algorithm, as it wouldn't necessarily be outwardly distinguishable from mimicry.
- wizzwizz4 4y ago
  > And the pace is clearly accelerating. From where I'm standing and observing… no. GPT-4 is not qualitatively more powerful than GPT-3, only quantitatively more powerful. The new functionality is coming from the other systems GPT-4 is being plugged into, not from the actual GPT model. There are even things that GPT-3 can do that GPT-4 cannot.
  - selfmodruntime 4y ago
    Really? You‘re not observing the staggering pace of advancements in AI technology? Then you‘re not looking.
ogogmad 4y ago
> We still have not the slightest indication that the programs we have created possess anything resembling consciousness, sapience, or sentience What do these words mean? > The responses of the LLMs to prompts are absolutely the kinds of things we should expect from statistical models trained on large chunks of the Internet I expected much less than you did from that description. I remember what things used to be like in 2010. Compared to then, what we have today is science fiction.
- Jensson 4y ago
  > I expected much less than you did from that description. I remember what things used to be like in 2010. Compared to then, what we have today is science fiction. If they told you that they made a perfect natural language parser, parsed the whole internet into logic and used that parsed data to make a giant text markov chain, you couldn't imagine that producing results similar to what we see today? The only new thing is that now we can work with natural language as it was code, and that lets us do a lot of cool things, but other than that there is nothing new. If humans structured their natural languages like we structure code then we would get all these abilities for free if we parsed the entire internet, but since we have the messy natural language we had to come up with the super complex solution we got today. You wouldn't need "programmers", since our natural language would be structured enough for computers to execute etc, and that would let us to structure up all posted human reasoning online and make all sorts of cool models based on that. Whenever you call ChatGPT see it as calling the logic in some internet web page, it could be a unix tutorial or a book etc, and then it runs random continuations of that. It isn't hard to imagine what happens when you do that. Is a perfect natural language parser cool? Yes. Was it unthinkable? Not really, many would say it wouldn't be possible, but if we said that we made it possible then figuring out the effects aren't super hard.
  - mannykannot 4y ago
    I don't work in the field, so my responses are naive, but for what it is worth: > If they told you that they made a perfect natural language parser... I would not have guessed that the methods used would yield that, or even 'just' a natural-language parser that performs as well as humans. > ...parsed the whole internet into logic... I am not sure what this means, but parsing seems to yield facts about specific sentences, and that alone seems insufficient to enable part three... > ...and used that parsed data to make a giant text Markov chain. Clearly, that a process is or can be described / explained / analyzed as a Markov chain is not sufficient for it to produce grammatically-correct output that is at least locally sensible to a human reader, so my surprise was that the methods used constitute a process that does. > ...but if we said that we made it possible then figuring out the effects aren't super hard. Hindsight is a really powerful tool.
isoprophlex 4y ago
Firstly, no need to get all prickly. In the absence of hard evidence everyone is entitled to their own opinion. Secondly, your brain is probably an ant colony on steroids with emergent properties of consciousness, and your DNA is a paperclip maximizer without agency. Lastly, 'this rate', I remember marveling over Char-RNN, whose LSTM neurons automatically specialized in detecting bits of grammar, less than 10 years ago. I don't need to tell you what changed 5 years, 1 year, 1 month ago. To me the rate of progress is astronomical. My point of view is, when dealing with potentially conscious entities, better to err on the side of caution. If you find my comment "breathless hype", deal with it without clamoring for silence.
- danaris 4y ago
  > If you find my comment "breathless hype", deal with it without clamoring for silence. The problem is that it's exactly this kind of hype that is trying to convince people that AGI is a) just around the corner, and b) going to wipe out humanity, and is thus calling for drastic measures in reaction to that perceived threat. Now, you may not personally believe that or advocate for those things—but intentionally or not, you are part of a chorus that is reacting to science fiction as if it is proven fact, and making demands that are likely to be very harmful to us as a people.
  - mannykannot 4y ago
    On the one hand, extrapolation from recent developments is always uncertain, and practically useless when we do not know how far it would have to be sustained, but on the other, the argument that we won't get there because we haven't made any progress yet will be just as true as it ever was right up until it is not.
  - HervalFreire 4y ago
    He clearly stated there's no hard evidence for either conclusion and he is just forming his own opinion. This is clearly different from what you are doing. You are making a hard claim that everything you see with LLMs being conscious is science fiction. Unfortunately this claim cannot be made given lack of evidence and even lack of definition of what consciousness is. Therefore by logic your statement is the one that is outlandish here.
roflyear 4y ago
Agreed. Watching people react this way makes me question our collective intelligence.
cjalmeida 4y ago
Well, while the idea of sentient AI may sound like science fiction, we are making significant strides in the field of artificial intelligence. It's not quite "I, Robot" yet, but who knows? Maybe we'll have a real-life Sonny to keep us company in the future. Just don't forget to give your robot buddy some goals that align with humanity's best interests, unless you want it to start working on paperclip production. — by ChatGPT

TheOtherHobbes 4y ago

They don't need consciousness, sapience, or sentience. They only need to mimic human behaviours accurately enough to make them redundant and automatable. That's an increasingly large range of behaviours. And the pace is clearly accelerating. There may or may not be an s-curve tail-off at some point. But if there is, we currently have no idea how to estimate where that point will be. I think a lot of people still don't realise that AI is absolutely an existential threat to our species. Currently it lives inside boxes we can turn off. But as soon as that stops being true - not because of experimentation, but because AI will be providing services and infrastructure we can't live without - the best we can hope for is some kind of symbiosis. The worst outcomes are far less reassuring.

prox 4y ago
Cue Agent Smiths’ monologue. That said, the way we treat our environment, ie anything external to us is a representation of an internal state. Not sure how the internal state of an AGI would be. We have many, many epistemological flaws and we do not wake up and say “Gee let’s examine my epistemology today!” Verifying what we believe, feel and reason is an ultimately worthwhile endeavor while it’s not really a cultural phenomenon.
danaris 4y ago
> They only need to mimic human behaviours accurately enough to make them redundant and automatable. The post I was replying to was specifically positing that they would have feelings.
- sho_hn 4y ago
  What makes a feeling different or unapproachable by sophisticated mimicry? GPT models with bad guardrails already generate dramatic/emotional text because they internalized human narratives and propensity for drama from the training data (cf. the "Bing got upset" type articles from a month ago). Unless you mean to say it's not the same as the model having qualia. It's not clear to me whether we would know when that emerged in the algorithm, as it wouldn't necessarily be outwardly distinguishable from mimicry.
wizzwizz4 4y ago
> And the pace is clearly accelerating. From where I'm standing and observing… no. GPT-4 is not qualitatively more powerful than GPT-3, only quantitatively more powerful. The new functionality is coming from the other systems GPT-4 is being plugged into, not from the actual GPT model. There are even things that GPT-3 can do that GPT-4 cannot.
- selfmodruntime 4y ago
  Really? You‘re not observing the staggering pace of advancements in AI technology? Then you‘re not looking.

ogogmad 4y ago

> We still have not the slightest indication that the programs we have created possess anything resembling consciousness, sapience, or sentience What do these words mean? > The responses of the LLMs to prompts are absolutely the kinds of things we should expect from statistical models trained on large chunks of the Internet I expected much less than you did from that description. I remember what things used to be like in 2010. Compared to then, what we have today is science fiction.

Jensson 4y ago
> I expected much less than you did from that description. I remember what things used to be like in 2010. Compared to then, what we have today is science fiction. If they told you that they made a perfect natural language parser, parsed the whole internet into logic and used that parsed data to make a giant text markov chain, you couldn't imagine that producing results similar to what we see today? The only new thing is that now we can work with natural language as it was code, and that lets us do a lot of cool things, but other than that there is nothing new. If humans structured their natural languages like we structure code then we would get all these abilities for free if we parsed the entire internet, but since we have the messy natural language we had to come up with the super complex solution we got today. You wouldn't need "programmers", since our natural language would be structured enough for computers to execute etc, and that would let us to structure up all posted human reasoning online and make all sorts of cool models based on that. Whenever you call ChatGPT see it as calling the logic in some internet web page, it could be a unix tutorial or a book etc, and then it runs random continuations of that. It isn't hard to imagine what happens when you do that. Is a perfect natural language parser cool? Yes. Was it unthinkable? Not really, many would say it wouldn't be possible, but if we said that we made it possible then figuring out the effects aren't super hard.
- mannykannot 4y ago
  I don't work in the field, so my responses are naive, but for what it is worth: > If they told you that they made a perfect natural language parser... I would not have guessed that the methods used would yield that, or even 'just' a natural-language parser that performs as well as humans. > ...parsed the whole internet into logic... I am not sure what this means, but parsing seems to yield facts about specific sentences, and that alone seems insufficient to enable part three... > ...and used that parsed data to make a giant text Markov chain. Clearly, that a process is or can be described / explained / analyzed as a Markov chain is not sufficient for it to produce grammatically-correct output that is at least locally sensible to a human reader, so my surprise was that the methods used constitute a process that does. > ...but if we said that we made it possible then figuring out the effects aren't super hard. Hindsight is a really powerful tool.

isoprophlex 4y ago

Firstly, no need to get all prickly. In the absence of hard evidence everyone is entitled to their own opinion. Secondly, your brain is probably an ant colony on steroids with emergent properties of consciousness, and your DNA is a paperclip maximizer without agency. Lastly, 'this rate', I remember marveling over Char-RNN, whose LSTM neurons automatically specialized in detecting bits of grammar, less than 10 years ago. I don't need to tell you what changed 5 years, 1 year, 1 month ago. To me the rate of progress is astronomical. My point of view is, when dealing with potentially conscious entities, better to err on the side of caution. If you find my comment "breathless hype", deal with it without clamoring for silence.

danaris 4y ago
> If you find my comment "breathless hype", deal with it without clamoring for silence. The problem is that it's exactly this kind of hype that is trying to convince people that AGI is a) just around the corner, and b) going to wipe out humanity, and is thus calling for drastic measures in reaction to that perceived threat. Now, you may not personally believe that or advocate for those things—but intentionally or not, you are part of a chorus that is reacting to science fiction as if it is proven fact, and making demands that are likely to be very harmful to us as a people.
- mannykannot 4y ago
  On the one hand, extrapolation from recent developments is always uncertain, and practically useless when we do not know how far it would have to be sustained, but on the other, the argument that we won't get there because we haven't made any progress yet will be just as true as it ever was right up until it is not.
- HervalFreire 4y ago
  He clearly stated there's no hard evidence for either conclusion and he is just forming his own opinion. This is clearly different from what you are doing. You are making a hard claim that everything you see with LLMs being conscious is science fiction. Unfortunately this claim cannot be made given lack of evidence and even lack of definition of what consciousness is. Therefore by logic your statement is the one that is outlandish here.

roflyear 4y ago

Agreed. Watching people react this way makes me question our collective intelligence.

cjalmeida 4y ago

Well, while the idea of sentient AI may sound like science fiction, we are making significant strides in the field of artificial intelligence. It's not quite "I, Robot" yet, but who knows? Maybe we'll have a real-life Sonny to keep us company in the future. Just don't forget to give your robot buddy some goals that align with humanity's best interests, unless you want it to start working on paperclip production. — by ChatGPT