3 ms·

Humans evolved unsupervised. AI is highly supervised. The idea that an AI will enslave us all is as absurd as suggesting “computers” will enslave us all merely

Humans evolved unsupervised. AI is highly supervised. The idea that an AI will enslave us all is as absurd as suggesting “computers” will enslave us all merely because they exist. Models are designed and operated by people for specific use cases. The real risks are people using this new tool for evil, not the tool itself. AI sentience is a seductive concept being used by self professed experts to draw attention to themselves and by mega corps to throw up competitive barriers to entry.

sorokod 3y ago
A fair amount of effort is spent on autonomous models, e.g. driving. Who knows what the military are up to.
circuit10 3y ago
It’s almost impossible to supervise something more intelligent than you because you can’t tell why it’s doing things. For now it’s easy to supervise them because AIs are way less intelligent than humans (though even now it’s hard to tell exactly why they’re doing things), but in the future it probably won’t be
- z3c0 3y ago
  but in the future it probably won’t be I see this parroted so often, and I have to ask: Why? What is there outside of the world of SciFi that makes the AGI of the future so nebulous, when humans would have presumably advanced to a point to be able to create the intelligence to begin with. Emergent properties are often as unexpected as they are bizarre, but they are not unexplainable, especially when you understand the underpinning systems.
  - circuit10 3y ago
    We can’t even fully explain how our own brains work, never mind a system that’s completely alien to us and that would have to be more complex. We can’t even explain how current LLMs work internally. Maybe we’ll make some breakthrough if we put enough resources into it but if people keep denying the problem there will never be enough resources out into it
    - AuthorizedCust 3y ago
      > We can’t even explain how current LLMs work internally. You sure can. They are just not simple explanations yet. But that’s the common course of inventions, which in foresight are mind-bogglingly complex, in hindsight pretty straightforward.
      - circuit10 3y ago
        You can explain the high level concepts but it’s really difficult to say “this group of neurons does this specific thing and that’s why this output was produced”, though OpenAI did make some progress in getting GPT-4 to explain what each neuron in GPT-2 is correlated to but we can also find what human brain regions are correlated to but that doesn’t necessarily explain the system as a whole and how everything interacts
        wizzwizz4 3y ago
        > but it’s really difficult to say “this group of neurons does this specific thing and that’s why this output was produced”, That's because that's not how brains work. > though OpenAI did make some progress in getting GPT-4 to explain what each neuron in GPT-2 is correlated to The work contained novel-to-me, somewhat impressive accomplishments, but this presentation of it was pure hype. They could have done the same thing without GPT-4 involved at all (and, in fact, they basically did… then they plugged it into GPT-4 to get a less-accurate-but-Englishy output instead).
        circuit10 3y ago
        When I said about a group of neurons I was talking about LLMs, but some of the same ideas probably apply. Yes, it’s probably not as simple as that, and that’s why we can’t understand them. I think they just used GPT-4 to help automate it on a large scale, which could be important to help understand the whole system especially for larger models
        wizzwizz4 3y ago
- jahewson 3y ago
  The government seems to manage the task just fine every day.
  - milsorgen 3y ago
    Does it?
- adsfgiodsnrio 3y ago
  "Supervised" does not mean the models need babysitting; it refers to the fundamental way the systems learn. Our most successful machine learning models all require some answers to be provided to them in order to infer the rules. Without being given explicit feedback they can't learn anything at all. Humans also do best with supervised learning. This is why we have schools. But humans are capable of unsupervised learning and use it all the time. A human can learn patterns even in completely unstructured information. A human is also able to create their own feedback by testing their beliefs against the world.
  - circuit10 3y ago
    Oh, sorry, I’m not that familiar with the terminology (I still feel like my argument is valid despite me not being an expert though because I heard all this from people who know a lot more than me about it). One problem with that kind of feedback is that it incentives the AI to make us think it solved the problem when it didn’t, for example by hallucinating convincing information. That means it specifically learns how to lie to us so it doesn’t really help Also I guess giving feedback is sort of like babysitting, but I did interpret it the wrong way
    - wizzwizz4 3y ago
      > One problem with that kind of feedback is that it incentives the AI to make us think it solved the problem when it didn’t, Supervised learning is: "here's the task" … "here's the expected solution" *adjusts model parameters to bring it closer to the expected solution*. What you're describing is specification hacking, which only occurs in a different kind of AI system: https://vkrakovna.wordpress.com/2018/04/02/specification-gaming-examples-in-ai/ https://vkrakovna.wordpress.com/2018/04/02/specification-gam... In theory, it could occur with feedback-based fine-tuning, but I doubt it'd result in anything impressive happening.
      - circuit10 3y ago
        Oh, that seems less problematic (though not completely free of problems), but also less powerful because it can’t really exceed human performance
flagrant_taco 3y ago
Where is the real supervision of AI though? Even those that are developing and managing it make it clear that they really have no insights into how or what the AI has learned. If we can't peak behind the curtain to see what's really going on, how can we really supervise it? Ever since GPT 3.5 dropped and people really started talking again about whether these AI are sentient, I've wondered if researchers are leaving on quantum theory to write it off as "it can't be sentient until we look to see if it is sentient"
JonChesterfield 3y ago
Your line of reasoning would be more comforting if the military wasn't enthusiastically arming such machine intelligence as we've managed to construct so far. Enslave us all feels unlikely as the benefit to said machine is unclear, kill us all is totally plausible.
DirkH 3y ago
Nobody said the AI needs to be sentient to enslave us all. Highly supervised AI systems mess up all the time and we often have no idea why. It "enslaving" us or whatever the overly extreme paperclip analogy is that is the current hot thought-experiment of the week is usually just talking about a mismatch between what we want and what the AI does. This is already a huge problem and will simply become a bigger problem as AI gets more powerful and complex. Imagine you asked GPT-7 to execute your business plan. What it does is so complex you can't know whether its plan will violate any laws. Let's say you're not an evil corporation and want to do good, but that it is impossible to compete without also using AI. At some point these systems may well become so complex we have no idea what they're actually doing. But empirically they seem to do what we want most of the time so we don't care and use them anyway. The problem is not only people using this new tool for evil. The problem is also the tool itself. Because it might be defective.

sorokod 3y ago

A fair amount of effort is spent on autonomous models, e.g. driving. Who knows what the military are up to.

circuit10 3y ago

It’s almost impossible to supervise something more intelligent than you because you can’t tell why it’s doing things. For now it’s easy to supervise them because AIs are way less intelligent than humans (though even now it’s hard to tell exactly why they’re doing things), but in the future it probably won’t be

z3c0 3y ago
but in the future it probably won’t be I see this parroted so often, and I have to ask: Why? What is there outside of the world of SciFi that makes the AGI of the future so nebulous, when humans would have presumably advanced to a point to be able to create the intelligence to begin with. Emergent properties are often as unexpected as they are bizarre, but they are not unexplainable, especially when you understand the underpinning systems.
- circuit10 3y ago
  We can’t even fully explain how our own brains work, never mind a system that’s completely alien to us and that would have to be more complex. We can’t even explain how current LLMs work internally. Maybe we’ll make some breakthrough if we put enough resources into it but if people keep denying the problem there will never be enough resources out into it
  - AuthorizedCust 3y ago
    > We can’t even explain how current LLMs work internally. You sure can. They are just not simple explanations yet. But that’s the common course of inventions, which in foresight are mind-bogglingly complex, in hindsight pretty straightforward.
    - circuit10 3y ago
      You can explain the high level concepts but it’s really difficult to say “this group of neurons does this specific thing and that’s why this output was produced”, though OpenAI did make some progress in getting GPT-4 to explain what each neuron in GPT-2 is correlated to but we can also find what human brain regions are correlated to but that doesn’t necessarily explain the system as a whole and how everything interacts
      - wizzwizz4 3y ago
        > but it’s really difficult to say “this group of neurons does this specific thing and that’s why this output was produced”, That's because that's not how brains work. > though OpenAI did make some progress in getting GPT-4 to explain what each neuron in GPT-2 is correlated to The work contained novel-to-me, somewhat impressive accomplishments, but this presentation of it was pure hype. They could have done the same thing without GPT-4 involved at all (and, in fact, they basically did… then they plugged it into GPT-4 to get a less-accurate-but-Englishy output instead).
        circuit10 3y ago
        When I said about a group of neurons I was talking about LLMs, but some of the same ideas probably apply. Yes, it’s probably not as simple as that, and that’s why we can’t understand them. I think they just used GPT-4 to help automate it on a large scale, which could be important to help understand the whole system especially for larger models
        wizzwizz4 3y ago
jahewson 3y ago
The government seems to manage the task just fine every day.
- milsorgen 3y ago
  Does it?
adsfgiodsnrio 3y ago
"Supervised" does not mean the models need babysitting; it refers to the fundamental way the systems learn. Our most successful machine learning models all require some answers to be provided to them in order to infer the rules. Without being given explicit feedback they can't learn anything at all. Humans also do best with supervised learning. This is why we have schools. But humans are capable of unsupervised learning and use it all the time. A human can learn patterns even in completely unstructured information. A human is also able to create their own feedback by testing their beliefs against the world.
- circuit10 3y ago
  Oh, sorry, I’m not that familiar with the terminology (I still feel like my argument is valid despite me not being an expert though because I heard all this from people who know a lot more than me about it). One problem with that kind of feedback is that it incentives the AI to make us think it solved the problem when it didn’t, for example by hallucinating convincing information. That means it specifically learns how to lie to us so it doesn’t really help Also I guess giving feedback is sort of like babysitting, but I did interpret it the wrong way
  - wizzwizz4 3y ago
    > One problem with that kind of feedback is that it incentives the AI to make us think it solved the problem when it didn’t, Supervised learning is: "here's the task" … "here's the expected solution" *adjusts model parameters to bring it closer to the expected solution*. What you're describing is specification hacking, which only occurs in a different kind of AI system: https://vkrakovna.wordpress.com/2018/04/02/specification-gaming-examples-in-ai/ https://vkrakovna.wordpress.com/2018/04/02/specification-gam... In theory, it could occur with feedback-based fine-tuning, but I doubt it'd result in anything impressive happening.
    - circuit10 3y ago
      Oh, that seems less problematic (though not completely free of problems), but also less powerful because it can’t really exceed human performance

flagrant_taco 3y ago

Where is the real supervision of AI though? Even those that are developing and managing it make it clear that they really have no insights into how or what the AI has learned. If we can't peak behind the curtain to see what's really going on, how can we really supervise it? Ever since GPT 3.5 dropped and people really started talking again about whether these AI are sentient, I've wondered if researchers are leaving on quantum theory to write it off as "it can't be sentient until we look to see if it is sentient"

JonChesterfield 3y ago

Your line of reasoning would be more comforting if the military wasn't enthusiastically arming such machine intelligence as we've managed to construct so far. Enslave us all feels unlikely as the benefit to said machine is unclear, kill us all is totally plausible.

DirkH 3y ago

Nobody said the AI needs to be sentient to enslave us all. Highly supervised AI systems mess up all the time and we often have no idea why. It "enslaving" us or whatever the overly extreme paperclip analogy is that is the current hot thought-experiment of the week is usually just talking about a mismatch between what we want and what the AI does. This is already a huge problem and will simply become a bigger problem as AI gets more powerful and complex. Imagine you asked GPT-7 to execute your business plan. What it does is so complex you can't know whether its plan will violate any laws. Let's say you're not an evil corporation and want to do good, but that it is impossible to compete without also using AI. At some point these systems may well become so complex we have no idea what they're actually doing. But empirically they seem to do what we want most of the time so we don't care and use them anyway. The problem is not only people using this new tool for evil. The problem is also the tool itself. Because it might be defective.