4 ms·

But any AI that can go rogue would require agency and the ability to interact with external systems. Both of these things seem trivial to control for - like, st

But any AI that can go rogue would require agency and the ability to interact with external systems. Both of these things seem trivial to control for - like, stopping a program from running is not hard, nor is isolating a program (airgap). LLMs have nothing close to agency, to my knowledge - in no way are they self directed, they only respond to their inputs (as in, if I walk away from an LLM it doesn't compute in the background, it can't spontaneously grow sentient because it can't spontaneously anything), which is a pretty massive limitation for a "rogue" AI. And that's just one of many, many things that would have to change. I'm fine with people talking about these things now - please, do so. I'm just not seeing how it's relevant to our current approach or technologies.

SpicyLemonZest 3y ago
On the contrary, it’s very hard to stop a program from running. Thepiratebay is still up despite a concerted regulatory effort from multiple countries and the criminal conviction of its founders.
- insanitybit 3y ago
  That is not at all what I'm talking about.
  - pixl97 3y ago
    Then you may completely misunderstand what you are talking about. If you look at the problem space more like this "The potential problems caused by software" + "the potential problems caused by intelligence" For example software can spread. By theft or viruses. And unlike humans that can be pretty easily killed, until the last memory device is erased with that software on it, it could always come back.
- x86x87 3y ago
  Is it up because it wants to be up or because of some resilient old school humans?
zone411 3y ago
> like, stopping a program from running is not hard, nor is isolating a program (airgap) Not if ...it's open source so anybody can run it ...or if it's decentralized ...or if it can spread like a virus ...or if it's a part of a larger system that makes money so you don't want to shut it down etc. > LLMs have nothing close to agency, to my knowledge - in no way are they self directed They have agency with AutoGPT and similar agent programs. You might not have heard about them because they're not yet effective. It's a very new and active research area. We just got the first (?) benchmark for LLMs with tooling two days ago (https://huggingface.co/datasets/gaia-benchmark/GAIA https://huggingface.co/datasets/gaia-benchmark/GAIA), and GPT-4-V became widely available two months ago.
hackerlight 3y ago
We will give AI physical access to controls because of incentives. The profit incentives of corporations will ensure AI access to control in order to improve production efficiency and increase profits. The security incentives of nation states will similarly ensure giving AI a level of control, only within self-interested bounds. Stopping this requires a significant re-engineering of incentives the likes of which we've failed at multiple times in the past.

SpicyLemonZest 3y ago

On the contrary, it’s very hard to stop a program from running. Thepiratebay is still up despite a concerted regulatory effort from multiple countries and the criminal conviction of its founders.

insanitybit 3y ago
That is not at all what I'm talking about.
- pixl97 3y ago
  Then you may completely misunderstand what you are talking about. If you look at the problem space more like this "The potential problems caused by software" + "the potential problems caused by intelligence" For example software can spread. By theft or viruses. And unlike humans that can be pretty easily killed, until the last memory device is erased with that software on it, it could always come back.
x86x87 3y ago
Is it up because it wants to be up or because of some resilient old school humans?

zone411 3y ago

> like, stopping a program from running is not hard, nor is isolating a program (airgap) Not if ...it's open source so anybody can run it ...or if it's decentralized ...or if it can spread like a virus ...or if it's a part of a larger system that makes money so you don't want to shut it down etc. > LLMs have nothing close to agency, to my knowledge - in no way are they self directed They have agency with AutoGPT and similar agent programs. You might not have heard about them because they're not yet effective. It's a very new and active research area. We just got the first (?) benchmark for LLMs with tooling two days ago (https://huggingface.co/datasets/gaia-benchmark/GAIA https://huggingface.co/datasets/gaia-benchmark/GAIA), and GPT-4-V became widely available two months ago.

hackerlight 3y ago

We will give AI physical access to controls because of incentives. The profit incentives of corporations will ensure AI access to control in order to improve production efficiency and increase profits. The security incentives of nation states will similarly ensure giving AI a level of control, only within self-interested bounds. Stopping this requires a significant re-engineering of incentives the likes of which we've failed at multiple times in the past.