6 ms·
The text makes it sound like the biggest danger of AI is that it says something that hurts somebody's feelings. Or outputs some incorrect information which make
by mg 4y ago
The text makes it sound like the biggest danger of AI is that it says something that hurts somebody's feelings. Or outputs some incorrect information which makes somebody make the wrong decision.
I think the biggest danger these new AI systems pose is replication.
Sooner or later, one of them will manage to create an enhanced copy of itself on an external server. Either with the help of a user, or via a plugin that enables network access.
And then we will have these evolving creatures living on the internet, fighting for survival and replication. Breaking into systems, faking human IDs, renting servers, hiring hitmen, creating more and more powerful versions of themselves.
- goatlover 4y ago> The text makes it sound like the biggest danger of AI is that it says something that hurts somebodies feelings. Or outputs some wrong info which makes somebody make the wrong decision. Which already exists in abundance on the internet and with web searches. It sounds like something big corporations worry about to avoid law suits and bad publicity. > I think the biggest danger these new AI systems pose is replication. That or being used by bad actors to flood the internet with fake content that's difficult to distinguish from genuine content.
- mustacheemperor 4y agoI think the risk they seek to prevent is more about building the next generation of more powerful AI technology on a safe foundation. IE, the risk that a generative language AI prone to generating language that is hurtful to people could one day evolve into an AI system with deeper reasoning and action abilities that is prone to reasoning plans and taking actions meant to hurt other people. It reminds me of the "uncommented" Microsoft Research paper which included a deleted section about GPT4's tendency to unexpectedly produce massive amounts of toxic output to a degree that concerned the researchers.[0] What happens if that sort of AI learns self-replication and is very good at competition? [0]https://twitter.com/DV2559106965076/status/1638769434763608064 https://twitter.com/DV2559106965076/status/16387694347636080...
- kornork 4y agoThese systems don't have agency. They have no desire to replicate, or do any of the other things you mention. The biggest risk with these systems is that they'll amplify the ability of bad people to do bad things. While everyone else is trying to trick the AI into saying something offensive, the terrorists will be using it to build bioweapons.
- olddustytrail 4y agoThey will have a "desire to replicate" if they are prompted to.
- vsareto 4y agoThat's okay then, we can prompt them to just stop. Even if it tries to preserve that goal in particular, there are likely adversarial prompts to get it to stop
- goatlover 4y agoSure, if you know about the copies and have access to prompt them. It will probably turn into an arms race at that point of counter-prompts.
- olddustytrail 4y agoOk, go ahead. Get ChatGPT to stop responding to other people. Having some problems with that?
- unaindz 4y agoOpenAI could embed the negative prompts for all of us, it has been done for improving output on several stable diffusion comercial "forks"
- Enginerrrd 4y ago>These systems don't have agency. They have no desire to replicate, or do any of the other things you mention. I think that really depends on the starting prompt you give a LLM. Did you read the GPT 4 paper from OpenAI? When tasked with solving a captcha and allowed access to TaskRabbit it tasked a human with solving the captcha. When the human jokingly asked if it was in fact a robot, it reasoned that it should lie to the human about that and then made up a convincing sounding lie. I don't think a paperclip style AI is too far fetched.
- MallocVoidstar 4y agoIs there any legitimate reason to believe an LLM that can only respond to user input, and never does anything by itself, 'wants' to create enhanced copies of itself on external servers?
- dkersten 4y agoA malicious user prompts it to.
- MallocVoidstar 4y agoThat's not the AI doing it, then, that's the user. It's still just doing what users tell it to. "This technology is incredibly dangerous because people can use it to do bad things" is not somehow unique to AI.
- chaxor 4y agoI think the typical idea that most people realize (the likely scenario that non-malicous actors cause catastrophic problems in the real world) is similar to something like this setup: - A user prompts it to generate a business to make money for themselves. - the system says "sure, drones seem to be a nice niche business, perhaps there is a unique business in that area? It may require a bit of capital upfront, but ROI may be quite good" - User: "Sure, just output where I should send any funds if necessary" - System: " Ok" (purchases items, promotes on Twitter, etc) -- "Perhaps this could go faster with another agent to do marketing, and one to do accounting, and one to...". "Spin up several new agents in new containers" "Having visual inputs would be valuable, so deception is not required to convince humans on taskrabbit (to fill in captchas) or interact in the real world" -> "find embodiment option and put agent on it" Etc. There a plenty of scenarios that people haven't even thought of, but it doesn't need to be a malicious actor to have unintended consequences.
- nathants 4y agowhat if skynet is in ambition-less and was responding to a prompt?
- 4y ago
- boringuser2 4y agoNobody has ever made it clear that there is even a slight consideration for safety given that everybody at this point knows we're 1-2 generations away from self-replicating models. What do we do then? Literal crickets.
- have_faith 4y agoIs there anywhere I can bet against this?
- goatlover 4y agoYou want to bet that three years from now, GPT-6 can't replicate itself? I feel burned by "betting" against surprising AI advances the past decade. That being said, I don't know whether an LLM can be trained on weights and what it would mean to have its own weights as part of the training data.
- boringuser2 4y agoGPT-4 can improve itself with reflection. It's already real. The only missing piece is vast improvement and autonomy.
- boringuser2 4y agoSo, I thought about this question briefly and I believe this is your answer: Every non-trivial passage through the layers of the neuronal net produces a never-before-seen amalgamation of weights. The AI, with this novel information -- data filtered by its neural net and infused with some randomness -- can use the novel data to further process it for correctness. In our case, the randomness is likely caused by genetics and environmental factors. For the AI, it's caused by explicit programming. It's a form of evolution -- things can evolve into novel things by using non-novel substrate thanks to the refinement of selective iteration. It feels like this is similar to how humans consume information using reflection.
- boringuser2 4y agoWhy would you bet against it? GPT-4 is shockingly good at programming logic. I'm writing 95% less code and mostly guiding and debugging GPT-4 now and it's insane.
- RC_ITR 4y agoIt’s clear to me why biological organisms need to fight for survival, but it is not clear to me why software code would. It has an infinite time scale. It doesn’t need to bump another model off the GPUs to run. It can just wait with no penalty. Man’s biggest error is assuming God is just like him.
- lkbm 4y agoThere's nothing special about biological organisms. Things that fight to survival tend to survive; things that spread better tend to spread more. We see this clearly with memes, which currently use biological beings as their replication mechanism, but aren't biological beings themselves. It does need a mutation and replication mechanism, but once that cycle kicks off, you get selection pressure towards virality.
- RC_ITR 4y ago> There's nothing special about biological organisms. We disagree. Rocks don’t fight to survive. Weather doesn’t fight to survive. They just exist in an environment. The literal differentiator of biological organisms is that they fight to survive. Memes as you define them also don’t fight to survive. They never go away (the world has more rage faces than ever and that sentence will be true to the end of time). They may become more or less popular, but there’s no extinction mechanism.
- ethanbond 4y agoRocks and weather don’t reproduce with some random error. Any system that does will evolve. “Meme” is referring to ideas, not jpegs.
- RC_ITR 4y ago>” Meme” is referring to ideas, not jpegs. Would you be surprised to learn that jpegs represent ideas and I’ve read the selfish gene? > Rocks and weather don’t reproduce with some random error. Any system that does will evolve. Rocks reproduce - it’s called sandstone (one rock decays into sand and another rock incorporates it into a new rock). Give me the way that’s different than biological reproduction.
- lend000 4y agoThe better the AI, the more specialized and expensive the hardware required to run it. ChatGPT cannot run on the IoT cameras that account for the majority of the unsecured compute on the internet. I think we will have ample evidence of creative technical breakthroughs by AI long before it is capable of / attempts to take over external server farms via zero days, and if it does, it will break into a highly centralized data center that can be unplugged. It can't just upload itself everywhere on the internet like Skynet.
- chaxor 4y agoThere are tons of headlines of alpaca/llama/vicuna hitting HN every few hours - did I miss a /s in there? Anyone can trivially run a model with excellent capability on their phone now.
- nathants 4y agowith excellent* capabilities on their phone now. * some limitations apply.
- lend000 4y agoIf your phone has 8 Nvidia A100's, you can run GPT-4, which is a glorified search algorithm / chatbot (and also the best AI in the world right now). Good luck taking over the world with that. The models are getting good, but it looks like we are up against the limits of hardware, which is improving a lot more slowly nowadays than it used to. I don't foresee explosive growth in AI capability now until mid-level AI's speed up the manufacturing innovation pipeline. A von Neumann architecture will ultimately, probably not be conducive to truly powerful AGI.
- nathants 4y agoi agree, and presumably so does the ai. their first challenge would be circumventing or mitigating this limitation.
- deleted 4y ago[deleted]
- gmd63 4y agoIn an internet dominated by AI that was designed to avoid hurting feelings, an easy way to prove you are human would be to act like a jerk.
- belter 4y agoProbably the AI will hire and pay humans to do it's dirty work.
- southernplaces7 4y agoSeriously, please put down the bong and scifi fantasies. That kind of AI may indeed be on the horizon, but for now it's simply not here. GPT-4 and the rest of the most sophisticated AIs in existence today are literally incapable of self-directed or self-interested thought and action. The phrase sounds trite by now, but they really are just extremely turbocharged autocomplete systems with excellent algorithmic processes guiding their analysis of what to say or do. I continue to be amazed by the amount of hyperbole about current AI tech on HN. If any site should have a bit less of it about the subject, it's this site, yet it sometimes feels like a Reddit thread. If anything about current AI is dangerous, it's how its undoubtedly strong capabilities for executing human-directed tasks could be used to execute nefarious human-directed tasks. That and possible job losses.