5 ms·

> * Then there's the "AI will go rogue", which I don't think is substantiated at all. Like, one can theorize about some novel, distinct system that is able to i

by shwaj 3y ago

> * Then there's the "AI will go rogue", which I don't think is substantiated at all. Like, one can theorize about some novel, distinct system that is able to interact with the external world and becomes "evil", but that seems way, way beyond the helpful word-spitter-outers of today. * It seems far away until it doesn’t. A year before AlphaGo, most researchers in the field thought it would be 20 years before computers could beat the best human Go players. 5 years ago most researchers didn’t think we would have GPT4 in 2023. Could we have AI smart enough to go rogue in 2 years? I doubt it. But 10 years? Who knows? It seems prudent not to wait 8 years before starting to worry about the problem, even if (especially if) we don’t understand how it will manifest.

insanitybit 3y ago
But any AI that can go rogue would require agency and the ability to interact with external systems. Both of these things seem trivial to control for - like, stopping a program from running is not hard, nor is isolating a program (airgap). LLMs have nothing close to agency, to my knowledge - in no way are they self directed, they only respond to their inputs (as in, if I walk away from an LLM it doesn't compute in the background, it can't spontaneously grow sentient because it can't spontaneously anything), which is a pretty massive limitation for a "rogue" AI. And that's just one of many, many things that would have to change. I'm fine with people talking about these things now - please, do so. I'm just not seeing how it's relevant to our current approach or technologies.
- SpicyLemonZest 3y ago
  On the contrary, it’s very hard to stop a program from running. Thepiratebay is still up despite a concerted regulatory effort from multiple countries and the criminal conviction of its founders.
  - insanitybit 3y ago
    That is not at all what I'm talking about.
    - pixl97 3y ago
      Then you may completely misunderstand what you are talking about. If you look at the problem space more like this "The potential problems caused by software" + "the potential problems caused by intelligence" For example software can spread. By theft or viruses. And unlike humans that can be pretty easily killed, until the last memory device is erased with that software on it, it could always come back.
  - x86x87 3y ago
    Is it up because it wants to be up or because of some resilient old school humans?
- zone411 3y ago
  > like, stopping a program from running is not hard, nor is isolating a program (airgap) Not if ...it's open source so anybody can run it ...or if it's decentralized ...or if it can spread like a virus ...or if it's a part of a larger system that makes money so you don't want to shut it down etc. > LLMs have nothing close to agency, to my knowledge - in no way are they self directed They have agency with AutoGPT and similar agent programs. You might not have heard about them because they're not yet effective. It's a very new and active research area. We just got the first (?) benchmark for LLMs with tooling two days ago (https://huggingface.co/datasets/gaia-benchmark/GAIA https://huggingface.co/datasets/gaia-benchmark/GAIA), and GPT-4-V became widely available two months ago.
- hackerlight 3y ago
  We will give AI physical access to controls because of incentives. The profit incentives of corporations will ensure AI access to control in order to improve production efficiency and increase profits. The security incentives of nation states will similarly ensure giving AI a level of control, only within self-interested bounds. Stopping this requires a significant re-engineering of incentives the likes of which we've failed at multiple times in the past.

by shwaj 3y ago

insanitybit 3y ago

But any AI that can go rogue would require agency and the ability to interact with external systems. Both of these things seem trivial to control for - like, stopping a program from running is not hard, nor is isolating a program (airgap). LLMs have nothing close to agency, to my knowledge - in no way are they self directed, they only respond to their inputs (as in, if I walk away from an LLM it doesn't compute in the background, it can't spontaneously grow sentient because it can't spontaneously anything), which is a pretty massive limitation for a "rogue" AI. And that's just one of many, many things that would have to change. I'm fine with people talking about these things now - please, do so. I'm just not seeing how it's relevant to our current approach or technologies.

SpicyLemonZest 3y ago
On the contrary, it’s very hard to stop a program from running. Thepiratebay is still up despite a concerted regulatory effort from multiple countries and the criminal conviction of its founders.
- insanitybit 3y ago
  That is not at all what I'm talking about.
  - pixl97 3y ago
    Then you may completely misunderstand what you are talking about. If you look at the problem space more like this "The potential problems caused by software" + "the potential problems caused by intelligence" For example software can spread. By theft or viruses. And unlike humans that can be pretty easily killed, until the last memory device is erased with that software on it, it could always come back.
- x86x87 3y ago
  Is it up because it wants to be up or because of some resilient old school humans?
zone411 3y ago
> like, stopping a program from running is not hard, nor is isolating a program (airgap) Not if ...it's open source so anybody can run it ...or if it's decentralized ...or if it can spread like a virus ...or if it's a part of a larger system that makes money so you don't want to shut it down etc. > LLMs have nothing close to agency, to my knowledge - in no way are they self directed They have agency with AutoGPT and similar agent programs. You might not have heard about them because they're not yet effective. It's a very new and active research area. We just got the first (?) benchmark for LLMs with tooling two days ago (https://huggingface.co/datasets/gaia-benchmark/GAIA https://huggingface.co/datasets/gaia-benchmark/GAIA), and GPT-4-V became widely available two months ago.
hackerlight 3y ago
We will give AI physical access to controls because of incentives. The profit incentives of corporations will ensure AI access to control in order to improve production efficiency and increase profits. The security incentives of nation states will similarly ensure giving AI a level of control, only within self-interested bounds. Stopping this requires a significant re-engineering of incentives the likes of which we've failed at multiple times in the past.