6 ms·

I am generally frightened by the readiness of people to send data to solutions like ChatGPT or by running open source projects that result in running "AI agents

by hrpnk 4y ago

I am generally frightened by the readiness of people to send data to solutions like ChatGPT or by running open source projects that result in running "AI agents" that execute arbitrary code on one's machine. What can we do to drive more awareness about running these experiments in a responsible way, like a sandboxed environment?

whatever1 4y ago
eventually we will have our personal AI tuned on our own data over time. Maybe we will be upgrading the foundation model, but the tuning would be personal. At least I hope so.
- sixothree 4y ago
  I fear the world will be structured such that we don’t get that kind of liberty. Surveillance capitalism has made me more pessimistic. Or realistic.
  - ben_w 4y ago
    I don't see liberty in having an AI tuned to me. Rather, I see that as being the exact surveillance you're pessimistic/realistic about.
atleastoptimal 4y ago
People are usually driven by basic operant conditioning, not safety-aware abstract principles. In the 2000s everyone was hyper aware of privacy issues, putting personal information online, safety, etc. However the last 15 years of people doing more and more risky stuff with allowing systems and computers to store and use their data and only getting benefits and more fun videos to watch, there's no internal shared concern to act in accordance to every possible safety principle. People just want the cool chatbot to do their work for them. Especially with the scale of FOMO with these things, no one is going to sit back and not be risky when they read that other people are saving many hours a day letting ChatGPT do their work for them. Until something really bad happens people aren't going to change their behavior.
ilaksh 4y ago
In my opinion this type of comment is the core problem with AI alignment these days. The current GPT models are not going to "wake up" or accidentally take over the world. This is very obvious. Forcing everyone who is conscientious to copy and paste the output of these relatively weak models is not going to save us from anything. If you want people to take AI regulations seriously, you have to do it in a reasonable way. Which means understanding the nuance of the dangers. Executing "arbitrary code" that InstructGPT models outputs based on your instructions is not a danger. People need to be able to distinguish different levels of autonomy and speed. It's when we get to high levels of autonomy and performance that we run into danger. Which we are really at the cusp. But when you fail to differentiate between systems that are obviously not dangerous and future/more autonomous agents that possibly are, that makes it impossible to take the concerns seriously. And actually harder for people to understand the problems with full autonomy and superintelligent models.
- ekidd 4y ago
  > The current GPT models are not going to "wake up" or accidentally take over the world. This is very obvious. It's pretty obvious that GPT-4 isn't consistently coherent enough to carry out complex plans or to do serious scientific research. There are several other major shortcomings which also prevent this from being viable, too. And of course, it's possible that some of these shortcomings are very hard to "fix." We've had AI winters before, and self-driving seems to have stalled. However, I strongly suspect that it's theoretically possible to build a machine that "thinks" as well as a human. And I no longer have any solid idea how far we are from that point, especially if we make certain architectural changes. We might be closer than we think. Or at least we should stop assuming that next-gen models are obviously safe. nVidia's hope to increase training performance 1 million times in 10 years seems potentially dangerous to me. Anthrophic's plan to spend over $1 billion training next gen models seems awfully sketchy as well.
  - pixl97 4y ago
    Self driving, I think is where we got it backwards. To drive correctly we need AGI because 'uncommon' but not 'rare' events are avoided by drivers all the time and in a manner you have to understand how the world works and how people think. LLMs, especially the multi-modal ones go much further to making self driving possible, that is if we ever get to the point they can incorporate and act on data fast enough. We may still have a hardware problem for some time there. --- But on your second point I am with you. There is nothing obvious about at what level emergent behavior occurs in the models. Just keep adding parameters and new things keep popping out with no real means to determine where and when seems kinda sketch as we push these things past the exa scale.
ben_w 4y ago
If they weren't paying attention to Black Mirror, the BSG reboot, the Terminator franchise, the Replicator arc of Stargate, every episode of Star Trek where the computer/Data/the holodeck/nanites went wrong, Ex-Machina, The Matrix franchise, Age of Ultron, basically any Asimov AI story, Blade Runner, 2001, or any folk tale about being careful what you ask of powers that take you literally from Midas to Fantasia… "Oh, they're just fictional!" …then I suggest attaching a rusty metal spike to the keyboard. It won't do anything, it's easy to avoid, it'll just sit there looking dangerous. Hopefully that would be enough just by itself. And yes, this is a reference to a similar suggestion for making drivers pay more attention behind the wheel.
- mrob 4y ago
  There is no sci-fi story that details a realistic scenario of AI existential risk, because sci-fi stories are written by and for humans. The potential danger of AI comes from the combination of superhuman abilities and profoundly alien mind. No human can predict any specific behavior of a superhuman intelligence, only that it will probably succeed in its goals, whatever those goals happen to be. And out of all possible goals, only a small proportion are compatible with life continuing to exist. This is especially true when you consider only simple goals, and simple things are generally easier to make. Sci-fi is a distraction. There can be no heroic human resistance like in the stories. If an AI is intelligent enough to be dangerous, it's intelligent enough to conceal its intentions until it's too late. The only way to beat a superhuman AI is by not making it in the first place.
  - mitthrowaway2 4y ago
    Yes, an AI apocalypse is not depicted in any science fiction because it would make for terrible fiction. Good stories depict relatable characters engaged in a battle between near-equals that ends in victory after terrifying odds. Not "someone saying 'oops' and then everyone falling over dead", which is the Yudkowsky scenario.
  - blibble 4y ago
    maybe something like the BBC Threads film is the way to go? plot of threads (set in Sheffield, UK): - russians/americans disagree on something in iran - initial nuclear attacks around base in iran - uneasy 2 day ceasefire - worldwide nuclear exchange occurs (a relativly small one) - graphic depiction of nuclear attack - people suffering horribly - ends with scene in hospital of character we've followed in labour, resulting in deformed stillbirth plot of artificial super intelligence: - tech company creates AI - idiot CEO enters badly thought out goal in an attempt to capture 2% more search market share - AI begins to executes goal - idiot CEO initially delighted at results - at some point AI becomes super-intelligent as it helps it achieve its goal - all life on earth is casually exterminated by the AI as it becomes better at achieving its goal - AI acts as a global scale systematic combine harvester - no humans, animals, bacteria, fungi left... no biological life ever again on earth - ends with scene of 100% of the earth's surface converted to GPUs, with the NVIDIA logo prominently displayed on each of them

by hrpnk 4y ago

whatever1 4y ago

eventually we will have our personal AI tuned on our own data over time. Maybe we will be upgrading the foundation model, but the tuning would be personal. At least I hope so.

sixothree 4y ago
I fear the world will be structured such that we don’t get that kind of liberty. Surveillance capitalism has made me more pessimistic. Or realistic.
- ben_w 4y ago
  I don't see liberty in having an AI tuned to me. Rather, I see that as being the exact surveillance you're pessimistic/realistic about.

atleastoptimal 4y ago

People are usually driven by basic operant conditioning, not safety-aware abstract principles. In the 2000s everyone was hyper aware of privacy issues, putting personal information online, safety, etc. However the last 15 years of people doing more and more risky stuff with allowing systems and computers to store and use their data and only getting benefits and more fun videos to watch, there's no internal shared concern to act in accordance to every possible safety principle. People just want the cool chatbot to do their work for them. Especially with the scale of FOMO with these things, no one is going to sit back and not be risky when they read that other people are saving many hours a day letting ChatGPT do their work for them. Until something really bad happens people aren't going to change their behavior.

ilaksh 4y ago

In my opinion this type of comment is the core problem with AI alignment these days. The current GPT models are not going to "wake up" or accidentally take over the world. This is very obvious. Forcing everyone who is conscientious to copy and paste the output of these relatively weak models is not going to save us from anything. If you want people to take AI regulations seriously, you have to do it in a reasonable way. Which means understanding the nuance of the dangers. Executing "arbitrary code" that InstructGPT models outputs based on your instructions is not a danger. People need to be able to distinguish different levels of autonomy and speed. It's when we get to high levels of autonomy and performance that we run into danger. Which we are really at the cusp. But when you fail to differentiate between systems that are obviously not dangerous and future/more autonomous agents that possibly are, that makes it impossible to take the concerns seriously. And actually harder for people to understand the problems with full autonomy and superintelligent models.

ekidd 4y ago
> The current GPT models are not going to "wake up" or accidentally take over the world. This is very obvious. It's pretty obvious that GPT-4 isn't consistently coherent enough to carry out complex plans or to do serious scientific research. There are several other major shortcomings which also prevent this from being viable, too. And of course, it's possible that some of these shortcomings are very hard to "fix." We've had AI winters before, and self-driving seems to have stalled. However, I strongly suspect that it's theoretically possible to build a machine that "thinks" as well as a human. And I no longer have any solid idea how far we are from that point, especially if we make certain architectural changes. We might be closer than we think. Or at least we should stop assuming that next-gen models are obviously safe. nVidia's hope to increase training performance 1 million times in 10 years seems potentially dangerous to me. Anthrophic's plan to spend over $1 billion training next gen models seems awfully sketchy as well.
- pixl97 4y ago
  Self driving, I think is where we got it backwards. To drive correctly we need AGI because 'uncommon' but not 'rare' events are avoided by drivers all the time and in a manner you have to understand how the world works and how people think. LLMs, especially the multi-modal ones go much further to making self driving possible, that is if we ever get to the point they can incorporate and act on data fast enough. We may still have a hardware problem for some time there. --- But on your second point I am with you. There is nothing obvious about at what level emergent behavior occurs in the models. Just keep adding parameters and new things keep popping out with no real means to determine where and when seems kinda sketch as we push these things past the exa scale.

ben_w 4y ago

If they weren't paying attention to Black Mirror, the BSG reboot, the Terminator franchise, the Replicator arc of Stargate, every episode of Star Trek where the computer/Data/the holodeck/nanites went wrong, Ex-Machina, The Matrix franchise, Age of Ultron, basically any Asimov AI story, Blade Runner, 2001, or any folk tale about being careful what you ask of powers that take you literally from Midas to Fantasia… "Oh, they're just fictional!" …then I suggest attaching a rusty metal spike to the keyboard. It won't do anything, it's easy to avoid, it'll just sit there looking dangerous. Hopefully that would be enough just by itself. And yes, this is a reference to a similar suggestion for making drivers pay more attention behind the wheel.

mrob 4y ago
There is no sci-fi story that details a realistic scenario of AI existential risk, because sci-fi stories are written by and for humans. The potential danger of AI comes from the combination of superhuman abilities and profoundly alien mind. No human can predict any specific behavior of a superhuman intelligence, only that it will probably succeed in its goals, whatever those goals happen to be. And out of all possible goals, only a small proportion are compatible with life continuing to exist. This is especially true when you consider only simple goals, and simple things are generally easier to make. Sci-fi is a distraction. There can be no heroic human resistance like in the stories. If an AI is intelligent enough to be dangerous, it's intelligent enough to conceal its intentions until it's too late. The only way to beat a superhuman AI is by not making it in the first place.
- mitthrowaway2 4y ago
  Yes, an AI apocalypse is not depicted in any science fiction because it would make for terrible fiction. Good stories depict relatable characters engaged in a battle between near-equals that ends in victory after terrifying odds. Not "someone saying 'oops' and then everyone falling over dead", which is the Yudkowsky scenario.
- blibble 4y ago
  maybe something like the BBC Threads film is the way to go? plot of threads (set in Sheffield, UK): - russians/americans disagree on something in iran - initial nuclear attacks around base in iran - uneasy 2 day ceasefire - worldwide nuclear exchange occurs (a relativly small one) - graphic depiction of nuclear attack - people suffering horribly - ends with scene in hospital of character we've followed in labour, resulting in deformed stillbirth plot of artificial super intelligence: - tech company creates AI - idiot CEO enters badly thought out goal in an attempt to capture 2% more search market share - AI begins to executes goal - idiot CEO initially delighted at results - at some point AI becomes super-intelligent as it helps it achieve its goal - all life on earth is casually exterminated by the AI as it becomes better at achieving its goal - AI acts as a global scale systematic combine harvester - no humans, animals, bacteria, fungi left... no biological life ever again on earth - ends with scene of 100% of the earth's surface converted to GPUs, with the NVIDIA logo prominently displayed on each of them