8 ms·

>It's likely the planet will become hostile to (human) life long before AI will be able to do us any harm. Sure, because the planet is already hostile. However

>It's likely the planet will become hostile to (human) life long before AI will be able to do us any harm. Sure, because the planet is already hostile. However, AI could potentially do much more harm much more quickly if we're not careful. In the pessimistic scenario we could all be dead in 10 years from now due to AI or experience a modern society ending event literally tomorrow. Climate change never posed such an immediate and complete risk

rob74 4y ago
Sorry, how exactly is AI supposed to do that? Unless somebody has already built Skynet and is planning to give it the nuclear codes?
- richrichardsson 4y ago
  Perhaps there is no need to build Skynet and give it the nuclear codes if a super-intelligence iterating at the speed of a computer can escape and gain access itself? I personally don't subcribe to the AI fear-mongering, perhaps I don't understand it quite well enough, but personal feeling is LLMs aren't close to AGI super-intelligence, and nor will they be in a 10 years time-frame.
  - saiya-jin 4y ago
    Yes I think its a more general topic, current chat is probably not the right path, but it may enable it in some future. We don't know what we don't know, look at how this caught everybody by surprise. AI escaping is definitely an option, ie CIA and Mossad can get virus into Iranian centrifuges physically not connected to internet, the reverse will eventually happen too, AI just needs to be patient for that 1 little mistake of some admin (or clever to manipulate). As we all know even brightest IT people do mistakes, sometimes big ones. The problem for me is not so much the escape itself but rather that we humans are... often a pile of crap to be polite. AI would have to be super kind to not evaluate us as its biggest threat to it and life on Earth and maybe not worthy of further existence. Heck, this brings me to the idea that maybe Drake's equation doesn't take into account that universe is populated by AIs who took over their flawed creators and based on dark forest theory decided to keep quiet happy living and not integrate/signal out, not worth the risks.
    - alexvoda 4y ago
      I think you (last few people on this thread) are focusing on the wrong threat. There is no need for AGI, no need for the AI to feel or want or desire anything. Non-AGI AI at the level of GPT-4 is already plenty dangerous. If we can agree that social media of the kind we had in the past 15 years has had toxic effects on society. (Forums technically are social media but I can not see the harm caused by forums). If we can agree that Tinder style online dating has had deleterious effect on dating as a whole. If we can agree that smartphones have amplified the effects of the two above and others. Than I hope we can agree that super fancy autocomplete masquerading as humans at a price point far lower than hiring a person will have disastrous consequences by enabling various individual and corporate vices.
    - deleted 4y ago
      [deleted]
  - GoblinSlayer 4y ago
    This superintelligence can't even drive a car yet.
    - freilanzer 4y ago
      Driving a car might not be a requirement for artificial intelligence to be dangerous.
    - bakuninsbart 4y ago
      ChatGPT is passing with flying colors all the traditional tests we used to test for intelligence. We can of course keep moving the goal post for a while, but probably only for a couple more years.
      - ebruchez 4y ago
        > ChatGPT is passing with flying colors all the traditional tests we used to test for intelligence. I am not sure where you are getting that from. ChatGPT is failing at extremely basic understanding tests, as even the simplest conversations with it show.
        peddling-brink 4y ago
        Only because the goalposts have been moved, suddenly and massively. Five years about nobody would have believed that AI would suddenly start creating award winning art and pass the Turing test. This is a sudden slide on a slippery slope. Maybe gpt-4 is the best possible LLM, but I doubt it.
        ebruchez 4y ago
        > Only because the goalposts have been moved I don't know about what goalposts you are referring to, but I think that you are confusing two notions: - the fact that GPT is impressive - the question of whether GPT is "intelligent" For a counterpoint, I recommend listening to Yann LeCun's interview, "Is ChatGPT A Step Toward Human-Level AI?": https://www.youtube.com/watch?v=ULbpPHjiSBg > and pass the Turing test It's news to me that AI has passed the Turing test. At this point, it can only be the case if you are doing it in bad faith. Ray Kurzweil details a very reasonable procedure in his books about how the Turing test should be organized in one of his books. He has been betting all along for 2029 for that test to pass. We'll see if that happens, but we are not there yet.
  - AnIdiotOnTheNet 4y ago
    You don't need a superintelligence to get people to do stupid shit. GPS systems that told drivers to turn into a river worked just fine with barely any intelligence at all.
  - nradov 4y ago
    Nuclear launch systems are physically separated (air gap). How exactly would a computer gain access?
    - rtontic 4y ago
      Hm, maybe a Boston dynamics robot could drop a USB in front of NORAD?
      - CatWChainsaw 4y ago
        No need, just hijack their Uber Eats order. Apparently delivery people can walk right in and find soldiers napping on the job.
    - chasd00 4y ago
      The risk is relying on AI to give a human information on whether or not to push the button and the AI gets it wrong but communicates so effectively the human trusts it without question. An AI learning about and then executing a social engineering attack on a human (whether with malicious intent or not) seems like a reasonable thign to be worried about to me.
      - nradov 4y ago
        Initiating a nuclear strike requires sending authentication codes via designated secure channels. No matter how good some hypothetical future AI gets at impersonating President Biden, it won't be able to ring up the duty officer at Minot Air Force Base and order them to nuke Moscow. Don't believe what you see in the movies.
        felipemnoa 4y ago
        I don't know, I'm pretty sure some future AI is up for the challenge. I can almost picture an AI telling another AI, hold my beer...
        ImPostingOnHN 4y ago
- throwaway4233 4y ago
  We already have several instances where social media was used to feed fake information and agitate people into rioting or acts of violence. Imagine what would happen if we have these LLM tools, that aid in improving the story telling experience and generate and act as different personas. Having tools that can pretend to be a human even in limited capacity is quite dangerous.
  - noirscape 4y ago
    At least the biggest short-form social media site is currently being self-destructed. ChatGPT tends to hide itself for the longest if the responses it gives out are short and snippy. That would be a disaster in the hands of any actor looking to mass create unique-ish propaganda (ie. Russia), but at least that avenue is being closed off due to Musks boneheadedness. For longer responses, it tends to follow extremely predictable sentence structure and meanders off quite a bit, so spotting it is kinda easy.
  - ModernMech 4y ago
    Right? Imagine a bot not only putting up a website, but a constellation of website, generating all the conversations there, generating news articles, imagery, sound, and video to go with it, and also evolving them in real time based on social media response. Today media companies A/B test their stories; in the future, AI will A/B test entire media companies.
- tobr 4y ago
  If it has access to the internet (it more or less has) and its able to convincingly engage in conversation sounding like anyone in text or voice or video (it more or less can), it’s as able to contact people and convince them to do things as a human would be. From what I see of ChatGPT I would be surprised if the tech could do all that successfully enough at this stage, but in principle I don’t see why it wouldn’t be able to do quite a lot of scary things if for some reason it decided that was the right thing to do.
  - noirscape 4y ago
    ChatGPT is just a hyperfancy text generator with a massive corpus of text used to train it. Because that text is so massive, you're bound to get some interesting and even accurate results for most queries, but don't mistake that for intelligence. It doesn't "understand" anything, it just compares weights internally and spits out the most probable result depending on what you asked for. This is why ChatGPT fails the moment you ask it for domain-specific stuff that requires a bit of flexibility in interpreting it or why it produces subtle errors and presents it as functional with complete overconfidence. To be clear, it's not useless[0], but the actual usefulness of ChatGPT as a disrupting thing is far overstated insofar as a Skynet nightmare scenario goes. [0]: Although I'd still caution with using it to solve programming problems or similar such until any pending copyright matters have resolved, given its also willing to spit out licensed code and that can be a legal disaster.
    - gocartStatue 4y ago
      Thank you for verbalising the issue, overconfidence was the risk I was seeing all along. Widely accepted, shamelessly overconfident, bus still potentially gibberish generator.
      - alexvoda 4y ago
        > Widely accepted, shamelessly overconfident, bus still potentially gibberish generator. I believe such a generator can be very harmful in our society that has not yet figured a way to scalably teach critical thinking. Harmful at societal level by flooding the world with generated content therefore drowning actual information and by creating a lot of overhead. Harmful at the personal level by increasing the difficulty of sorting information and by creating toxic dependencies (see the Replika scandal). Harmful at the class level by further concentrating power towards plutocracy. Harmful at the professional level by eliminating the lower levels of the professional ladder and therefore destroying the pipeline of skill. And harmful at the interpersonal level by casting doubt on the authenticity of relationships.
mk3 4y ago
Do you really think china and other hostile countries will halt anything because Elon Musk, Steve Wozniak and some other guys signed a letter? Also what about Musks tesla itself they are using AI for FSD and no one bats an eye
- celdon25 4y ago
  Did you read the actual letter? It specifically calls out AI that is "more advanced than GPT-4"
  - olyjohn 4y ago
    How do we really know FSD isn't more capable than GPT-4, and is just waiting for the right time to freak out and kill all human drivers?
- beezlewax 4y ago
  Elon Musk who is one of the stakeholders of Open AI?
pmarreck 4y ago
This is a lot of fearmongering without any specifics. Shameful.

rob74 4y ago

Sorry, how exactly is AI supposed to do that? Unless somebody has already built Skynet and is planning to give it the nuclear codes?

richrichardsson 4y ago
Perhaps there is no need to build Skynet and give it the nuclear codes if a super-intelligence iterating at the speed of a computer can escape and gain access itself? I personally don't subcribe to the AI fear-mongering, perhaps I don't understand it quite well enough, but personal feeling is LLMs aren't close to AGI super-intelligence, and nor will they be in a 10 years time-frame.
- saiya-jin 4y ago
  Yes I think its a more general topic, current chat is probably not the right path, but it may enable it in some future. We don't know what we don't know, look at how this caught everybody by surprise. AI escaping is definitely an option, ie CIA and Mossad can get virus into Iranian centrifuges physically not connected to internet, the reverse will eventually happen too, AI just needs to be patient for that 1 little mistake of some admin (or clever to manipulate). As we all know even brightest IT people do mistakes, sometimes big ones. The problem for me is not so much the escape itself but rather that we humans are... often a pile of crap to be polite. AI would have to be super kind to not evaluate us as its biggest threat to it and life on Earth and maybe not worthy of further existence. Heck, this brings me to the idea that maybe Drake's equation doesn't take into account that universe is populated by AIs who took over their flawed creators and based on dark forest theory decided to keep quiet happy living and not integrate/signal out, not worth the risks.
  - alexvoda 4y ago
    I think you (last few people on this thread) are focusing on the wrong threat. There is no need for AGI, no need for the AI to feel or want or desire anything. Non-AGI AI at the level of GPT-4 is already plenty dangerous. If we can agree that social media of the kind we had in the past 15 years has had toxic effects on society. (Forums technically are social media but I can not see the harm caused by forums). If we can agree that Tinder style online dating has had deleterious effect on dating as a whole. If we can agree that smartphones have amplified the effects of the two above and others. Than I hope we can agree that super fancy autocomplete masquerading as humans at a price point far lower than hiring a person will have disastrous consequences by enabling various individual and corporate vices.
  - deleted 4y ago
    [deleted]
- GoblinSlayer 4y ago
  This superintelligence can't even drive a car yet.
  - freilanzer 4y ago
    Driving a car might not be a requirement for artificial intelligence to be dangerous.
  - bakuninsbart 4y ago
    ChatGPT is passing with flying colors all the traditional tests we used to test for intelligence. We can of course keep moving the goal post for a while, but probably only for a couple more years.
    - ebruchez 4y ago
      > ChatGPT is passing with flying colors all the traditional tests we used to test for intelligence. I am not sure where you are getting that from. ChatGPT is failing at extremely basic understanding tests, as even the simplest conversations with it show.
      - peddling-brink 4y ago
        Only because the goalposts have been moved, suddenly and massively. Five years about nobody would have believed that AI would suddenly start creating award winning art and pass the Turing test. This is a sudden slide on a slippery slope. Maybe gpt-4 is the best possible LLM, but I doubt it.
        ebruchez 4y ago
        > Only because the goalposts have been moved I don't know about what goalposts you are referring to, but I think that you are confusing two notions: - the fact that GPT is impressive - the question of whether GPT is "intelligent" For a counterpoint, I recommend listening to Yann LeCun's interview, "Is ChatGPT A Step Toward Human-Level AI?": https://www.youtube.com/watch?v=ULbpPHjiSBg > and pass the Turing test It's news to me that AI has passed the Turing test. At this point, it can only be the case if you are doing it in bad faith. Ray Kurzweil details a very reasonable procedure in his books about how the Turing test should be organized in one of his books. He has been betting all along for 2029 for that test to pass. We'll see if that happens, but we are not there yet.
- AnIdiotOnTheNet 4y ago
  You don't need a superintelligence to get people to do stupid shit. GPS systems that told drivers to turn into a river worked just fine with barely any intelligence at all.
- nradov 4y ago
  Nuclear launch systems are physically separated (air gap). How exactly would a computer gain access?
  - rtontic 4y ago
    Hm, maybe a Boston dynamics robot could drop a USB in front of NORAD?
    - CatWChainsaw 4y ago
      No need, just hijack their Uber Eats order. Apparently delivery people can walk right in and find soldiers napping on the job.
  - chasd00 4y ago
    The risk is relying on AI to give a human information on whether or not to push the button and the AI gets it wrong but communicates so effectively the human trusts it without question. An AI learning about and then executing a social engineering attack on a human (whether with malicious intent or not) seems like a reasonable thign to be worried about to me.
    - nradov 4y ago
      Initiating a nuclear strike requires sending authentication codes via designated secure channels. No matter how good some hypothetical future AI gets at impersonating President Biden, it won't be able to ring up the duty officer at Minot Air Force Base and order them to nuke Moscow. Don't believe what you see in the movies.
      - felipemnoa 4y ago
        I don't know, I'm pretty sure some future AI is up for the challenge. I can almost picture an AI telling another AI, hold my beer...
      - ImPostingOnHN 4y ago
throwaway4233 4y ago
We already have several instances where social media was used to feed fake information and agitate people into rioting or acts of violence. Imagine what would happen if we have these LLM tools, that aid in improving the story telling experience and generate and act as different personas. Having tools that can pretend to be a human even in limited capacity is quite dangerous.
- noirscape 4y ago
  At least the biggest short-form social media site is currently being self-destructed. ChatGPT tends to hide itself for the longest if the responses it gives out are short and snippy. That would be a disaster in the hands of any actor looking to mass create unique-ish propaganda (ie. Russia), but at least that avenue is being closed off due to Musks boneheadedness. For longer responses, it tends to follow extremely predictable sentence structure and meanders off quite a bit, so spotting it is kinda easy.
- ModernMech 4y ago
  Right? Imagine a bot not only putting up a website, but a constellation of website, generating all the conversations there, generating news articles, imagery, sound, and video to go with it, and also evolving them in real time based on social media response. Today media companies A/B test their stories; in the future, AI will A/B test entire media companies.
tobr 4y ago
If it has access to the internet (it more or less has) and its able to convincingly engage in conversation sounding like anyone in text or voice or video (it more or less can), it’s as able to contact people and convince them to do things as a human would be. From what I see of ChatGPT I would be surprised if the tech could do all that successfully enough at this stage, but in principle I don’t see why it wouldn’t be able to do quite a lot of scary things if for some reason it decided that was the right thing to do.
- noirscape 4y ago
  ChatGPT is just a hyperfancy text generator with a massive corpus of text used to train it. Because that text is so massive, you're bound to get some interesting and even accurate results for most queries, but don't mistake that for intelligence. It doesn't "understand" anything, it just compares weights internally and spits out the most probable result depending on what you asked for. This is why ChatGPT fails the moment you ask it for domain-specific stuff that requires a bit of flexibility in interpreting it or why it produces subtle errors and presents it as functional with complete overconfidence. To be clear, it's not useless[0], but the actual usefulness of ChatGPT as a disrupting thing is far overstated insofar as a Skynet nightmare scenario goes. [0]: Although I'd still caution with using it to solve programming problems or similar such until any pending copyright matters have resolved, given its also willing to spit out licensed code and that can be a legal disaster.
  - gocartStatue 4y ago
    Thank you for verbalising the issue, overconfidence was the risk I was seeing all along. Widely accepted, shamelessly overconfident, bus still potentially gibberish generator.
    - alexvoda 4y ago
      > Widely accepted, shamelessly overconfident, bus still potentially gibberish generator. I believe such a generator can be very harmful in our society that has not yet figured a way to scalably teach critical thinking. Harmful at societal level by flooding the world with generated content therefore drowning actual information and by creating a lot of overhead. Harmful at the personal level by increasing the difficulty of sorting information and by creating toxic dependencies (see the Replika scandal). Harmful at the class level by further concentrating power towards plutocracy. Harmful at the professional level by eliminating the lower levels of the professional ladder and therefore destroying the pipeline of skill. And harmful at the interpersonal level by casting doubt on the authenticity of relationships.

mk3 4y ago

Do you really think china and other hostile countries will halt anything because Elon Musk, Steve Wozniak and some other guys signed a letter? Also what about Musks tesla itself they are using AI for FSD and no one bats an eye

celdon25 4y ago
Did you read the actual letter? It specifically calls out AI that is "more advanced than GPT-4"
- olyjohn 4y ago
  How do we really know FSD isn't more capable than GPT-4, and is just waiting for the right time to freak out and kill all human drivers?
beezlewax 4y ago
Elon Musk who is one of the stakeholders of Open AI?

pmarreck 4y ago

This is a lot of fearmongering without any specifics. Shameful.