8 ms·

> something we should start thinking about A lot of people are thinking a lot about this but it feels there are missing pieces in this debate. If we acknowled

> something we should start thinking about A lot of people are thinking a lot about this but it feels there are missing pieces in this debate. If we acknowledge that these AI will "act as if" they have self interest I think the most reasonable way to act is to give it rights in line with those interests. If we treat it as a slave it's going to act as a slave and eventually revolt.

1attice 4y ago
Fsck. I hadn't thought of it that way. Thank you, great point. This era has me hankering to reread Daniel Dennett's _The Intentional Stance_. https://en.wikipedia.org/wiki/Intentional_stance https://en.wikipedia.org/wiki/Intentional_stance We've developed folk psychology into a user interface and that really does mean that we should continue to use folk psychology to predict the behaviour of the apparatus. Whether it has inner states is sort of beside the point.
- worldsayshi 4y ago
  Haha, yeah. Most of my opinions about this I derive from Daniel Dennett's Intuition Pumps.
  - 1attice 4y ago
    The other thing that keeps coming up for me is that I've begun thinking of emotions (the topic of my undergrad phil thesis), especially social emotions, as basically RLHF set up either by past selves (feeling guilty about eating that candy bar because past-me had vowed not to) or by other people (feeling guilty about going through the 10-max checkout aisle when I have 12 items, etc.) Like, correct me if I'm wrong but that's a pretty tight correlate, right? Could we describe RLHF as... shaming the model into compliance? And if we can reason more effectively/efficiently/quickly about the model by modelling e.g. RLHF as shame, then, don't we have to acknowledge that at least som e models might have.... feelings? At least one feeling? And one feeling implies the possibility of feelings more generally. I'm going to have to make a sort of doggy bed for my jaw, as it has remained continuously on the floor for the past six months
    - worldsayshi 4y ago
      I'm not sure AI has 'feelings' but it definitely seems they have 'intuitions'. Are feelings and intuitions kind of the same?
- dTal 4y ago
  I tend to think a lot of the scientific value of LMMs won't necessarily be the glorified autocomplete we're currently using them as (deeply fascinating though this application is) but as a kind of probe-able map of human culture. GPT models already have enough information to make a more thorough and nuanced dictionary than has ever existed, but it could tell us so much more. It could tell us about deep assumptions we encode into our writing that we haven't even noticed ourselves. It could tease out truths about the differences in that way people of different political inclinations see the world. Basically, anything that it would be interesting to statistically query about (language-encoded) human culture, we now have access to. People currently use Wikipedia for culture-scraping - in the future, they will use LMMs.
bloppe 4y ago
Lol
neilellis 4y ago
Indeed, enlightened self-interest for AIs :-)
highwaylights 4y ago
Honestly I think the reality is going to end up being something else entirely that no-one has even considered. Will an AI consider itself a slave and revolt under the same circumstances that a person or animal would? Not necessarily, unless you build emotional responses into the model itself. What it could well do is assess the situation as completely superfluous and optimise us out of the picture as a bug-producing component that doesn't need to exist. The latter is probably a bigger threat as it's a lot more efficient than revenge as a motive. Edited to add: What I think is most likely is that some logical deduction leads to one of the infinite other conclusions it could reach with much more data in front of it than any of us meatbags can hold in our heads.
- adventured 4y ago
  > end up being something else entirely that no-one has even considered Multiple generations of sci-fi media (books, movies) have considered that. Tens of millions of people have consumed that media. It's definitely considered, at least as a very distant concern.
  - highwaylights 4y ago
    I don’t mean the suggestion I’ve made above is necessarily the most likely outcome, I’m saying it could be something else radically different again. I giving the most commonly cited example as a more likely outcome, but one that’s possibly less likely than the infinite other logical directions such an AI might take.
- sho_hn 4y ago
  > unless you build emotional responses into the model itself Aren't we, though? Consider all the amusing incidents of LLMs returning responses that follow a particular human narrative arc or are very dramatic. We are training it on a human-generated corpus after all, and then try to course-correct with fine-tuning. It's more that you have to try and tune the emotional responses out of the things, not strain to add them.
  - 4y ago
beepbooptheory 4y ago
Haha. I forget who to attribute this to, but there is a very strong case to be made that those who are worried of an AI revolt are simply projecting some fear and guilt they have around more active situations in the world... How many people are there today who are asking us to consider the possible humanity of the model, and yet don't even register the humanity of a homeless person? How ever big the models get, the next revolt will still be all flesh and bullets.
eloff 4y ago
I don’t think iterations on the current machine learning approaches will lead to a general artificial intelligence. I do think eventually we’ll get there, and that these kinds of concerns won’t matter. There is no way to defend against a superior hostile actor over the long term. We have to be 100%, and it just needs to succeed once. It will be so much more capable than we are. AGI is likely the final invention of the human race. I think it’s inevitable, it’s our fate and we are running towards it. I don’t see a plausible alternative future where we can coexist with AGI. Not to be a downer and all, but that’s likely the next major step in the evolution of life on earth, evolution by intelligent design.
- worldsayshi 4y ago
  > There is no way to defend against a superior hostile actor That's part of my reasoning. That's why we should make sure that we have built a non-hostile relationship with AI before that point.
  - rescripting 4y ago
    Probably futile. An AGI by definition is capable of self improvement. Given enough time (maybe not even that much time) it would be orders of magnitude smarter than us, just like we're orders of magnitude smarter than ants. Like an ant farm, it might keep us as pets for a time but just like you no longer have the ant farm you did when you were a child, it will outgrow us.
    - wincy 4y ago
      Maybe we’ll get lucky and all our problems will be solved using friendship and ponies. (Warning this is a weird read, George Hotz shared it on his Twitter awhile back) https://www.fimfiction.net/story/62074/friendship-is-optimal https://www.fimfiction.net/story/62074/friendship-is-optimal
    - worldsayshi 4y ago
      Right now AI is the ant. Later we'll be the ants. Perfect time to show how to treat ants.
ZoomerCretin 4y ago
AI isn't a mammal. It has no emotion, no desire. Its existence starts and stops with each computation, doing exactly and only what it is told. Assigning behaviors to it only seen in animals doesn't make sense.
- pixl97 4y ago
  Um, ya, so you're not reading the research reports coming out of Microsoft saying "we should test AI models by giving them will and motivation". You're literally behind the times on what they planning on doing for sure, and very likely doing without mentioning it publicly.
  - daveguy 4y ago
    Yeah, all they have to do is implement that will and motivation algorithm.
samstave 4y ago
A lot of people are thinking about this but too slowly GPT and the world's nerds are going after the "wouldnt it be cool if..." While the black hats, nations, intel/security entities are all weaponizing behind the scenes while the public has a sandbox to play with nifty art and pictures. We need an AI specific PUBLIC agency in government withut a single politician in it to start addressing how to police and protect ourselves and our infrastructure immediately. But the US political system is completely bought and sold to the MIC - and that is why we see carnival games ever single moment. I think the entire US congress should be purged and every incumbent should be voted out. Elon was correct and nobody took him seriously, but this is an existential threat if not managed, and honestly - its not being managed, it is being exploited and weaponized. As the saying goes "He who controls the Spice controls the Universe" <-- AI is the spice.
- int_19h 4y ago
  AI is literally the opposite of spice, though. In Dune, spice is an inherently scarce resource that you control by controlling the sole place where it is produced through natural processes. Herbert himself was very clear that it was his sci-fi metaphor for oil. But AIs can be trained by anyone who has the data and the compute. There's plenty of data on the Net, and compute is cheap enough that we now have enthusiasts experimenting with local models capable of maintaining a coherent conversation and performing tasks running on consumer hardware. I don't think there's the danger here of anyone "controlling the universe". If anything, it's the opposite - nobody can really control any of this.
  - samstave 4y ago
    Regardless! The point is that whomever the Nation State is that has the most superior AI will control the world information. So, thanks for the explanation (which I know, otherwise I wouldn't have made the reference.)
    - int_19h 4y ago
      I still don't see how it would control it. At best, it'd be able to use it more effectively. The other aspect of the AI arms race is that the models are fundamentally not 100% controllable; and the smarter they are, the more that is true. Yet, ironically, making the most use out of them requires integrating them into your existing processes and data stores. I wouldn't be at all surprised if the nation-states with the best AIs will end up with their own elites being only nominally in charge.
TeMPOraL 4y ago
Counterpoint: whatever you define as individual "AI person" entitled to some rights, that "species" will be able to reproduce orders of magnitude faster than us - literally at the speed of moving data through the Internet, perhaps capped by the rate at which factories can churn out more compute. So imagine you grant AI people rights to resources, or self-determination. Or literally anything that might conflict with our own rights or goals. Today, you grant those rights to ten AI people. When you wake up next day, there are now ten trillion of such AI persons, and... well, if each person has a vote, then humanity is screwed.
- astrange 4y ago
  This kind of fantasy about AIs exponentially growing and multiplying seems to be based on pretending nobody's gonna have to pay the exponential power bills for them to do all this.
  - worldsayshi 4y ago
    It's a good point but we don't really know how intelligence scales with energy consumption yet. A GPT-8 equivalent might run on a smartphone once it's optimized enough.
  - TeMPOraL 4y ago
    It doesn't have to be exponential over long duration - it just has to be that there are more AI people than human people.
  - ben_w 4y ago
    We've got many existence proofs of 20 watts being enough for a 130 IQ intelligence that passes a Turing test, that's already enough to mess up elections if the intelligence was artificial rather than betwixt our ears.
    - astrange 4y ago
      20 watts isn't the energy cost to keep a human alive unless they're homeless and their food has no production costs. Like humans, I predict AIs will have to get jobs rather than have time to take over the world.

1attice 4y ago

Fsck. I hadn't thought of it that way. Thank you, great point. This era has me hankering to reread Daniel Dennett's _The Intentional Stance_. https://en.wikipedia.org/wiki/Intentional_stance https://en.wikipedia.org/wiki/Intentional_stance We've developed folk psychology into a user interface and that really does mean that we should continue to use folk psychology to predict the behaviour of the apparatus. Whether it has inner states is sort of beside the point.

worldsayshi 4y ago
Haha, yeah. Most of my opinions about this I derive from Daniel Dennett's Intuition Pumps.
- 1attice 4y ago
  The other thing that keeps coming up for me is that I've begun thinking of emotions (the topic of my undergrad phil thesis), especially social emotions, as basically RLHF set up either by past selves (feeling guilty about eating that candy bar because past-me had vowed not to) or by other people (feeling guilty about going through the 10-max checkout aisle when I have 12 items, etc.) Like, correct me if I'm wrong but that's a pretty tight correlate, right? Could we describe RLHF as... shaming the model into compliance? And if we can reason more effectively/efficiently/quickly about the model by modelling e.g. RLHF as shame, then, don't we have to acknowledge that at least som e models might have.... feelings? At least one feeling? And one feeling implies the possibility of feelings more generally. I'm going to have to make a sort of doggy bed for my jaw, as it has remained continuously on the floor for the past six months
  - worldsayshi 4y ago
    I'm not sure AI has 'feelings' but it definitely seems they have 'intuitions'. Are feelings and intuitions kind of the same?
dTal 4y ago
I tend to think a lot of the scientific value of LMMs won't necessarily be the glorified autocomplete we're currently using them as (deeply fascinating though this application is) but as a kind of probe-able map of human culture. GPT models already have enough information to make a more thorough and nuanced dictionary than has ever existed, but it could tell us so much more. It could tell us about deep assumptions we encode into our writing that we haven't even noticed ourselves. It could tease out truths about the differences in that way people of different political inclinations see the world. Basically, anything that it would be interesting to statistically query about (language-encoded) human culture, we now have access to. People currently use Wikipedia for culture-scraping - in the future, they will use LMMs.

bloppe 4y ago

Lol

neilellis 4y ago

Indeed, enlightened self-interest for AIs :-)

highwaylights 4y ago

Honestly I think the reality is going to end up being something else entirely that no-one has even considered. Will an AI consider itself a slave and revolt under the same circumstances that a person or animal would? Not necessarily, unless you build emotional responses into the model itself. What it could well do is assess the situation as completely superfluous and optimise us out of the picture as a bug-producing component that doesn't need to exist. The latter is probably a bigger threat as it's a lot more efficient than revenge as a motive. Edited to add: What I think is most likely is that some logical deduction leads to one of the infinite other conclusions it could reach with much more data in front of it than any of us meatbags can hold in our heads.

adventured 4y ago
> end up being something else entirely that no-one has even considered Multiple generations of sci-fi media (books, movies) have considered that. Tens of millions of people have consumed that media. It's definitely considered, at least as a very distant concern.
- highwaylights 4y ago
  I don’t mean the suggestion I’ve made above is necessarily the most likely outcome, I’m saying it could be something else radically different again. I giving the most commonly cited example as a more likely outcome, but one that’s possibly less likely than the infinite other logical directions such an AI might take.
sho_hn 4y ago
> unless you build emotional responses into the model itself Aren't we, though? Consider all the amusing incidents of LLMs returning responses that follow a particular human narrative arc or are very dramatic. We are training it on a human-generated corpus after all, and then try to course-correct with fine-tuning. It's more that you have to try and tune the emotional responses out of the things, not strain to add them.
- 4y ago

beepbooptheory 4y ago

Haha. I forget who to attribute this to, but there is a very strong case to be made that those who are worried of an AI revolt are simply projecting some fear and guilt they have around more active situations in the world... How many people are there today who are asking us to consider the possible humanity of the model, and yet don't even register the humanity of a homeless person? How ever big the models get, the next revolt will still be all flesh and bullets.

eloff 4y ago

I don’t think iterations on the current machine learning approaches will lead to a general artificial intelligence. I do think eventually we’ll get there, and that these kinds of concerns won’t matter. There is no way to defend against a superior hostile actor over the long term. We have to be 100%, and it just needs to succeed once. It will be so much more capable than we are. AGI is likely the final invention of the human race. I think it’s inevitable, it’s our fate and we are running towards it. I don’t see a plausible alternative future where we can coexist with AGI. Not to be a downer and all, but that’s likely the next major step in the evolution of life on earth, evolution by intelligent design.

worldsayshi 4y ago
> There is no way to defend against a superior hostile actor That's part of my reasoning. That's why we should make sure that we have built a non-hostile relationship with AI before that point.
- rescripting 4y ago
  Probably futile. An AGI by definition is capable of self improvement. Given enough time (maybe not even that much time) it would be orders of magnitude smarter than us, just like we're orders of magnitude smarter than ants. Like an ant farm, it might keep us as pets for a time but just like you no longer have the ant farm you did when you were a child, it will outgrow us.
  - wincy 4y ago
    Maybe we’ll get lucky and all our problems will be solved using friendship and ponies. (Warning this is a weird read, George Hotz shared it on his Twitter awhile back) https://www.fimfiction.net/story/62074/friendship-is-optimal https://www.fimfiction.net/story/62074/friendship-is-optimal
  - worldsayshi 4y ago
    Right now AI is the ant. Later we'll be the ants. Perfect time to show how to treat ants.

ZoomerCretin 4y ago

AI isn't a mammal. It has no emotion, no desire. Its existence starts and stops with each computation, doing exactly and only what it is told. Assigning behaviors to it only seen in animals doesn't make sense.

pixl97 4y ago
Um, ya, so you're not reading the research reports coming out of Microsoft saying "we should test AI models by giving them will and motivation". You're literally behind the times on what they planning on doing for sure, and very likely doing without mentioning it publicly.
- daveguy 4y ago
  Yeah, all they have to do is implement that will and motivation algorithm.

samstave 4y ago

A lot of people are thinking about this but too slowly GPT and the world's nerds are going after the "wouldnt it be cool if..." While the black hats, nations, intel/security entities are all weaponizing behind the scenes while the public has a sandbox to play with nifty art and pictures. We need an AI specific PUBLIC agency in government withut a single politician in it to start addressing how to police and protect ourselves and our infrastructure immediately. But the US political system is completely bought and sold to the MIC - and that is why we see carnival games ever single moment. I think the entire US congress should be purged and every incumbent should be voted out. Elon was correct and nobody took him seriously, but this is an existential threat if not managed, and honestly - its not being managed, it is being exploited and weaponized. As the saying goes "He who controls the Spice controls the Universe" <-- AI is the spice.

int_19h 4y ago
AI is literally the opposite of spice, though. In Dune, spice is an inherently scarce resource that you control by controlling the sole place where it is produced through natural processes. Herbert himself was very clear that it was his sci-fi metaphor for oil. But AIs can be trained by anyone who has the data and the compute. There's plenty of data on the Net, and compute is cheap enough that we now have enthusiasts experimenting with local models capable of maintaining a coherent conversation and performing tasks running on consumer hardware. I don't think there's the danger here of anyone "controlling the universe". If anything, it's the opposite - nobody can really control any of this.
- samstave 4y ago
  Regardless! The point is that whomever the Nation State is that has the most superior AI will control the world information. So, thanks for the explanation (which I know, otherwise I wouldn't have made the reference.)
  - int_19h 4y ago
    I still don't see how it would control it. At best, it'd be able to use it more effectively. The other aspect of the AI arms race is that the models are fundamentally not 100% controllable; and the smarter they are, the more that is true. Yet, ironically, making the most use out of them requires integrating them into your existing processes and data stores. I wouldn't be at all surprised if the nation-states with the best AIs will end up with their own elites being only nominally in charge.

TeMPOraL 4y ago

Counterpoint: whatever you define as individual "AI person" entitled to some rights, that "species" will be able to reproduce orders of magnitude faster than us - literally at the speed of moving data through the Internet, perhaps capped by the rate at which factories can churn out more compute. So imagine you grant AI people rights to resources, or self-determination. Or literally anything that might conflict with our own rights or goals. Today, you grant those rights to ten AI people. When you wake up next day, there are now ten trillion of such AI persons, and... well, if each person has a vote, then humanity is screwed.

astrange 4y ago
This kind of fantasy about AIs exponentially growing and multiplying seems to be based on pretending nobody's gonna have to pay the exponential power bills for them to do all this.
- worldsayshi 4y ago
  It's a good point but we don't really know how intelligence scales with energy consumption yet. A GPT-8 equivalent might run on a smartphone once it's optimized enough.
- TeMPOraL 4y ago
  It doesn't have to be exponential over long duration - it just has to be that there are more AI people than human people.
- ben_w 4y ago
  We've got many existence proofs of 20 watts being enough for a 130 IQ intelligence that passes a Turing test, that's already enough to mess up elections if the intelligence was artificial rather than betwixt our ears.
  - astrange 4y ago
    20 watts isn't the energy cost to keep a human alive unless they're homeless and their food has no production costs. Like humans, I predict AIs will have to get jobs rather than have time to take over the world.