5 ms·

The danger of this shit can not be understated. Four years ago already there was a video where a deepfake of a president of the USA read a speech: In Event of M

by chx 2y ago

The danger of this shit can not be understated. Four years ago already there was a video where a deepfake of a president of the USA read a speech: In Event of Moon Disaster https://www.youtube.com/watch?v=LWLadJFI8Pk https://www.youtube.com/watch?v=LWLadJFI8Pk we of course know Nixon never gave this speech. What happens when this "AI" is used to sway an election? Last month a family got hospitalized after eating mushrooms they found and identified from a AI generated book. They didn't know it was AI generated. What happens when and this is not if, alas but when people die from this? This shit is a danger to democracy and human lives. Napster was not.

scotty79 2y ago
> What happens when this "AI" is used to sway an election? It already happened and the world went merrily on its way towards wherever the hell it goes.
- consteval 2y ago
  This doesn't mean it's good or even fine. The world also had a holocaust, and we moved on. But it would've been preferable to not have a holocaust.
  - scotty79 2y ago
    You are right. Even if I think elections don't matter there was one instance where it happened and the outcome actually mattered. The Brexit vote. But even there I don't think it's a huge difference if politicians are lying themselves or they do it with the use of AI. As long as it's an open technology the game is still fair.
    - consteval 2y ago
      > As long as it's an open technology the game is still fair. I don't necessarily agree because money has a lot to do with it. In theory, anyone can use AI for evil. In practice, only the richest can inflict damage. It's a matter of scale. That's why me killing someone is pretty bad, but me building a Nuclear Warhead is much worse. Luckily I don't have the power to build a Nuclear Warhead. Now anybody with enough money has the tech to just make one.
      - scotty79 2y ago
        > In practice, only the richest can inflict damage. It's a matter of scale. That's always been true for every technology ever. Even nuclear warheads you mentioned. They are not controlled by poor people. Even hacking that was seen as sort of equalizer is way more dangerous in hands of states that can hire whole armies of hackers.
emporas 2y ago
>Last month a family got hospitalized after eating mushrooms they found and identified from a AI generated book. Yeah, mushrooms are very dangerous, but the argument here is that the book could be written by a human. So what's the best way forward? Ban A.I. books, and human books? A.I. also, is the best way IMHO to identify mushrooms. Mushrooms are drastically different from one another by magnifying their spores using a microscope. That's not the case when looking at the fruit. Mushrooms totally unrelated to one another, may turn out to be very similar, depending on season, humidity, rain, elevation, tree hosts, temperature and soil. However when a human tries to examine the spores, he has to compare the spores to thousands of mushrooms to be sure. That's an amount of information, that only mechanically could be tackled effectively. A.I. may have a good chance to solve that problem.
- 8n4vidtmkvmk 2y ago
  That's kind of a different thing. Making the best AI possible for recognizing mushrooms and trying to produce the most factually correct answers is different than letting AIs run rampant, generating fiction. Also, I'd want some kind of verification before using the mushroom app. Who made the app? Do they have people on their team familiar with mushrooms? Botanists or whatever you call them. Some random dude with free time and access to maybe 10 species of mushrooms in his backyard.. even if his intent is good, is still dangerous.
  - chx 2y ago
    LLMs can not produce factual answers. That's not a property of them. What people call "hallucination" is the only thing they can do. Even if it occasionally happens to deliver a correct results that's just the broken clock being right twice a day. The typical ones sound authorative and so people believe it's authorative, this is well documented in Thinking Fast & Slow. As Aza Raskin put it, they hit a zeroday in human cognition. But that doesn't change the fundamentals of it, only the perception of the output which we need to consciously combat. https://hachyderm.io/@inthehands/112006855076082650 https://hachyderm.io/@inthehands/112006855076082650 > You might be surprised to learn that I actually think LLMs have the potential to be not only fun but genuinely useful. “Show me some bullshit that would be typical in this context” can be a genuinely helpful question to have answered, in code and in natural language — for brainstorming, for seeing common conventions in an unfamiliar context, for having something crappy to react to. > Alas, that does not remotely resemble how people are pitching this technology.
geerlingguy 2y ago
Hi chx, long time no see! And I agree; deepfakes and voice cloning is already past the point it's fooling relatives. Some are good enough I have to spend time double checking if it's real or not. There are very real implications to all that, and being able to verify true from false is going to get more challenging in many circumstances.
actualwitch 2y ago
> What happens when this "AI" is used to sway an election? What do you mean "when"? It's already happening. https://apnews.com/article/trump-taylor-swift-fake-endorsement-ai-fec99c412d960932839e3eab8d49fd5f https://apnews.com/article/trump-taylor-swift-fake-endorseme...

by chx 2y ago

scotty79 2y ago

> What happens when this "AI" is used to sway an election? It already happened and the world went merrily on its way towards wherever the hell it goes.

consteval 2y ago
This doesn't mean it's good or even fine. The world also had a holocaust, and we moved on. But it would've been preferable to not have a holocaust.
- scotty79 2y ago
  You are right. Even if I think elections don't matter there was one instance where it happened and the outcome actually mattered. The Brexit vote. But even there I don't think it's a huge difference if politicians are lying themselves or they do it with the use of AI. As long as it's an open technology the game is still fair.
  - consteval 2y ago
    > As long as it's an open technology the game is still fair. I don't necessarily agree because money has a lot to do with it. In theory, anyone can use AI for evil. In practice, only the richest can inflict damage. It's a matter of scale. That's why me killing someone is pretty bad, but me building a Nuclear Warhead is much worse. Luckily I don't have the power to build a Nuclear Warhead. Now anybody with enough money has the tech to just make one.
    - scotty79 2y ago
      > In practice, only the richest can inflict damage. It's a matter of scale. That's always been true for every technology ever. Even nuclear warheads you mentioned. They are not controlled by poor people. Even hacking that was seen as sort of equalizer is way more dangerous in hands of states that can hire whole armies of hackers.

emporas 2y ago

>Last month a family got hospitalized after eating mushrooms they found and identified from a AI generated book. Yeah, mushrooms are very dangerous, but the argument here is that the book could be written by a human. So what's the best way forward? Ban A.I. books, and human books? A.I. also, is the best way IMHO to identify mushrooms. Mushrooms are drastically different from one another by magnifying their spores using a microscope. That's not the case when looking at the fruit. Mushrooms totally unrelated to one another, may turn out to be very similar, depending on season, humidity, rain, elevation, tree hosts, temperature and soil. However when a human tries to examine the spores, he has to compare the spores to thousands of mushrooms to be sure. That's an amount of information, that only mechanically could be tackled effectively. A.I. may have a good chance to solve that problem.

8n4vidtmkvmk 2y ago
That's kind of a different thing. Making the best AI possible for recognizing mushrooms and trying to produce the most factually correct answers is different than letting AIs run rampant, generating fiction. Also, I'd want some kind of verification before using the mushroom app. Who made the app? Do they have people on their team familiar with mushrooms? Botanists or whatever you call them. Some random dude with free time and access to maybe 10 species of mushrooms in his backyard.. even if his intent is good, is still dangerous.
- chx 2y ago
  LLMs can not produce factual answers. That's not a property of them. What people call "hallucination" is the only thing they can do. Even if it occasionally happens to deliver a correct results that's just the broken clock being right twice a day. The typical ones sound authorative and so people believe it's authorative, this is well documented in Thinking Fast & Slow. As Aza Raskin put it, they hit a zeroday in human cognition. But that doesn't change the fundamentals of it, only the perception of the output which we need to consciously combat. https://hachyderm.io/@inthehands/112006855076082650 https://hachyderm.io/@inthehands/112006855076082650 > You might be surprised to learn that I actually think LLMs have the potential to be not only fun but genuinely useful. “Show me some bullshit that would be typical in this context” can be a genuinely helpful question to have answered, in code and in natural language — for brainstorming, for seeing common conventions in an unfamiliar context, for having something crappy to react to. > Alas, that does not remotely resemble how people are pitching this technology.

geerlingguy 2y ago

Hi chx, long time no see! And I agree; deepfakes and voice cloning is already past the point it's fooling relatives. Some are good enough I have to spend time double checking if it's real or not. There are very real implications to all that, and being able to verify true from false is going to get more challenging in many circumstances.

actualwitch 2y ago

> What happens when this "AI" is used to sway an election? What do you mean "when"? It's already happening. https://apnews.com/article/trump-taylor-swift-fake-endorsement-ai-fec99c412d960932839e3eab8d49fd5f https://apnews.com/article/trump-taylor-swift-fake-endorseme...