5 ms·

Imho is stunning, yet what is happening there is super dangerous. These videos will and may be too realistic. Our society is not prepared for this kind of rea

by 0xcb0 2y ago

Imho is stunning, yet what is happening there is super dangerous. These videos will and may be too realistic. Our society is not prepared for this kind of reality "bending" media. These hyperrealistic videos will be the reason for hate and murder. Evil actors will use it to influence elections on a global scale. Create cults around virtual characters. Deny the rules of physics and human reason. And yet, there is no way for a person to detect instantly that he is watching a generated video. Maybe now, but in 1 year, it will be indistinguishable from a real recorded video

ddalex 2y ago
The society voted with their money. Google refrained from launching their early chatbots and image generation tools due to perceived risks of unsafe and misleading content being generated, and got beaten to the punch in the market. Of course now they'll launch early and often, the market has spoken.
- Retr0id 2y ago
  We have constructed a society where market forces feel inevitable, but it doesn't have to be that way.
  - ddalex 2y ago
    Of course; but this is the current society, and attempts to reform it, e.g. communism, failed abjectly, so by evolution pressure, the capitalist society dominated by market forces is the best that we have
    - Retr0id 2y ago
      Right, but there are plenty of middle grounds between true communism and just letting markets freewheel.
      - amunozo 2y ago
        And places with these systems are those that achieved the best quality of life and peace.
      - xvector 2y ago
        There's no evidence that this fearmongering over safety is actually correct. The worst thing you can do is pummel an emerging technology into the grave because of misplaced fear. Just take a look at how many everyday things were "incredibly dangerous for society" - https://pessimistsarchive.org/ https://pessimistsarchive.org/
        Retr0id 2y ago
        Spend any amount of time on mainstream social media and you'll see AI-generated media being shared credulously. It's not a hypothetical risk, it's already happening. Even if you're not convinced that it's dangerous, at the very least it's incredibly annoying. If someone dumped a trailer full of trash in your garden, you're not going to say "oh well, market forces compelled them to do that".
        xvector 2y ago
        Eh, growing pains.
dtquad 2y ago
Instead of calling for regulations, the big tech companies should run big campaigns educating the public, especially boomers, that they no longer can trust images, videos, and audio on the Internet. Put paid articles and ads about this in local newspapers around the world so even the least online people gets educated about this.
- Retr0id 2y ago
  What would motivate "big tech" to warn people about their own products, if not regulations?
- WickyNilliams 2y ago
  Do we really want a world where we can't trust anything we see, hear, or read? Where people need to be educated to not trust their senses, the things we use to interpret reality and the world around us. I feel this kind of hypervigilance will be mentally exhausting, and not being able to trust your primary senses will have untold psychological effects
  - unglaublich 2y ago
    You can trust what you see and hear around you. You might be able to trust information from a party you trust. You certainly shouldn't trust digital information from unknown entities with unknown agendas. We're already in a world where "fake news" and "alt-facts" influence our daily lives and political outcomes.
    - WickyNilliams 2y ago
      What I see and hear around me is a miniscule fraction of the outside world. To have a shared understanding of reality, of what is happening in my town, my city, my state, my country, my continent, the world, requires much more than what is available in your immediate environment. In the grand scheme of understanding the world at large, our immediate senses are not particularly valuable. So we _have_ to rely on other streams of information. And the trend is towards more of those streams being digital. The existence of "fake news" and "alt facts", doesn't mean we should accept a further and dramatic worsening of our ability to have a shared reality. To accept that as an inevitability is defeatist and a kind of learned helplessness. Have you seen the Adam Curtis documentary "Hypernormalisation"? It deals with some similar themes, but on a much smaller scale (at least it is smaller in the context of current and near future tech)
krapp 2y ago
We already have hate and murder, evil actors influencing elections on a global scale, denial of physics and reason, and cults of personality. We also already have the ability to create realistic videos - not that it matters because for many people the bar of credulity isn't realism but simply confirming their priors. We already live in a world where TikTok memes and Facebook are the primary sources around which the masses base their reality, and that shit doesn't even take effort. The only thing this changes is not needing to pay human beings for work.
ks2048 2y ago
Are Apple and other phone/camera makers working on ways to "sign" a video to say it's an unedited video from a camera? Does this exist now? Is it possible? I'm thinking of simple cryptographic signing of a file, rather than embedding watermarks into the content, but that's another option. I don't think it will solve the fake video onslaught, but it could help.
- ttul 2y ago
  I think this will be a thing one day, where photos are digitally watermarked by the camera sensor in a non-repudiable manner.
  - eddd-ddde 2y ago
    This is a losing battle. You can always just record an AI video with your camera. Done, now you have a real video.
    - hbn 2y ago
      This is what I think every time I hear about AI watermarking. If anything, convincing people that AI watermarking is a real, reliable thing is just gonna cause more harm because bad actors that want to convince people something fake is real would obviously do the simple subversion tactics. Then you have a bunch of people seeing it passes the watermark check, and therefore is real.
    - 110 2y ago
      Potential solutions: 1. AI video watermarks that carry over even if a video of the AI video is taken 2. Cameras that can see AI video watermarks and put an AI video watermark on the videos of any AI videos they take
    - ks2048 2y ago
      I agree is probably a losing battle, but maybe worth fighting. If the metadata is also encrypted, you can also verify the time and place it was recorded. Of course, this requires closed/locked hardware and still possible to spoof. Not ideal, but some assurances are better than a future of can't trust anything.
tomp 2y ago
We've had realistic sci-fi and alternate history movies for a very long time.
- DonHopkins 2y ago
  [flagged]
- oldmanhorton 2y ago
  Which take millions of dollars and huge teams to make. These take one bored person, a sentence, and a few minutes to go from idea to posting on social media. That difference is the entire concern.
  - encoderer 2y ago
    If “evil actors” could really “manipulate elections” with fake video, would they really let a few million dollars stop them? That’s not that much money.
    - esafak 2y ago
      Who says they don't? Interference is being "democratized".
      - deleted 2y ago
        [deleted]
      - encoderer 2y ago
        Any examples of hoax videos that you can name? I’m having a hard time placing any. I really find the threat to be overhyped.
        esafak 2y ago
        You can find citations in https://en.wikipedia.org/wiki/Russian_interference_in_the_2024_United_States_elections https://en.wikipedia.org/wiki/Russian_interference_in_the_20...
golergka 2y ago
Photoshop has been a thing for over 30 years.
- EForEndeavour 2y ago
  Isn't the whole point of OP that we're currently watching the barrier to generating realistic assets go from "spend months grinding Photoshop tutorials" to "type what you want into this box and wait a few minutes"?
onel 2y ago
The same things could be said when everyone could print their own newspapers or books. How would people distinguish between fake and real news? I think we will need the same healthy media diet.
- dbbk 2y ago
  There wasn't even a healthy media diet before generative AI given the amount of 'fake news' in 2016 and 2020.
dbbk 2y ago
I still don't really know why we're doing this. What is the upside? Democratising Hollywood? At the expense of... enormous catastrophic disinformation and media manipulation.

by 0xcb0 2y ago

ddalex 2y ago

The society voted with their money. Google refrained from launching their early chatbots and image generation tools due to perceived risks of unsafe and misleading content being generated, and got beaten to the punch in the market. Of course now they'll launch early and often, the market has spoken.

Retr0id 2y ago
We have constructed a society where market forces feel inevitable, but it doesn't have to be that way.
- ddalex 2y ago
  Of course; but this is the current society, and attempts to reform it, e.g. communism, failed abjectly, so by evolution pressure, the capitalist society dominated by market forces is the best that we have
  - Retr0id 2y ago
    Right, but there are plenty of middle grounds between true communism and just letting markets freewheel.
    - amunozo 2y ago
      And places with these systems are those that achieved the best quality of life and peace.
    - xvector 2y ago
      There's no evidence that this fearmongering over safety is actually correct. The worst thing you can do is pummel an emerging technology into the grave because of misplaced fear. Just take a look at how many everyday things were "incredibly dangerous for society" - https://pessimistsarchive.org/ https://pessimistsarchive.org/
      - Retr0id 2y ago
        Spend any amount of time on mainstream social media and you'll see AI-generated media being shared credulously. It's not a hypothetical risk, it's already happening. Even if you're not convinced that it's dangerous, at the very least it's incredibly annoying. If someone dumped a trailer full of trash in your garden, you're not going to say "oh well, market forces compelled them to do that".
        xvector 2y ago
        Eh, growing pains.

dtquad 2y ago

Instead of calling for regulations, the big tech companies should run big campaigns educating the public, especially boomers, that they no longer can trust images, videos, and audio on the Internet. Put paid articles and ads about this in local newspapers around the world so even the least online people gets educated about this.

Retr0id 2y ago
What would motivate "big tech" to warn people about their own products, if not regulations?
WickyNilliams 2y ago
Do we really want a world where we can't trust anything we see, hear, or read? Where people need to be educated to not trust their senses, the things we use to interpret reality and the world around us. I feel this kind of hypervigilance will be mentally exhausting, and not being able to trust your primary senses will have untold psychological effects
- unglaublich 2y ago
  You can trust what you see and hear around you. You might be able to trust information from a party you trust. You certainly shouldn't trust digital information from unknown entities with unknown agendas. We're already in a world where "fake news" and "alt-facts" influence our daily lives and political outcomes.
  - WickyNilliams 2y ago
    What I see and hear around me is a miniscule fraction of the outside world. To have a shared understanding of reality, of what is happening in my town, my city, my state, my country, my continent, the world, requires much more than what is available in your immediate environment. In the grand scheme of understanding the world at large, our immediate senses are not particularly valuable. So we _have_ to rely on other streams of information. And the trend is towards more of those streams being digital. The existence of "fake news" and "alt facts", doesn't mean we should accept a further and dramatic worsening of our ability to have a shared reality. To accept that as an inevitability is defeatist and a kind of learned helplessness. Have you seen the Adam Curtis documentary "Hypernormalisation"? It deals with some similar themes, but on a much smaller scale (at least it is smaller in the context of current and near future tech)

krapp 2y ago

We already have hate and murder, evil actors influencing elections on a global scale, denial of physics and reason, and cults of personality. We also already have the ability to create realistic videos - not that it matters because for many people the bar of credulity isn't realism but simply confirming their priors. We already live in a world where TikTok memes and Facebook are the primary sources around which the masses base their reality, and that shit doesn't even take effort. The only thing this changes is not needing to pay human beings for work.

ks2048 2y ago

Are Apple and other phone/camera makers working on ways to "sign" a video to say it's an unedited video from a camera? Does this exist now? Is it possible? I'm thinking of simple cryptographic signing of a file, rather than embedding watermarks into the content, but that's another option. I don't think it will solve the fake video onslaught, but it could help.

ttul 2y ago
I think this will be a thing one day, where photos are digitally watermarked by the camera sensor in a non-repudiable manner.
- eddd-ddde 2y ago
  This is a losing battle. You can always just record an AI video with your camera. Done, now you have a real video.
  - hbn 2y ago
    This is what I think every time I hear about AI watermarking. If anything, convincing people that AI watermarking is a real, reliable thing is just gonna cause more harm because bad actors that want to convince people something fake is real would obviously do the simple subversion tactics. Then you have a bunch of people seeing it passes the watermark check, and therefore is real.
  - 110 2y ago
    Potential solutions: 1. AI video watermarks that carry over even if a video of the AI video is taken 2. Cameras that can see AI video watermarks and put an AI video watermark on the videos of any AI videos they take
  - ks2048 2y ago
    I agree is probably a losing battle, but maybe worth fighting. If the metadata is also encrypted, you can also verify the time and place it was recorded. Of course, this requires closed/locked hardware and still possible to spoof. Not ideal, but some assurances are better than a future of can't trust anything.

tomp 2y ago

We've had realistic sci-fi and alternate history movies for a very long time.

DonHopkins 2y ago
[flagged]
oldmanhorton 2y ago
Which take millions of dollars and huge teams to make. These take one bored person, a sentence, and a few minutes to go from idea to posting on social media. That difference is the entire concern.
- encoderer 2y ago
  If “evil actors” could really “manipulate elections” with fake video, would they really let a few million dollars stop them? That’s not that much money.
  - esafak 2y ago
    Who says they don't? Interference is being "democratized".
    - deleted 2y ago
      [deleted]
    - encoderer 2y ago
      Any examples of hoax videos that you can name? I’m having a hard time placing any. I really find the threat to be overhyped.
      - esafak 2y ago
        You can find citations in https://en.wikipedia.org/wiki/Russian_interference_in_the_2024_United_States_elections https://en.wikipedia.org/wiki/Russian_interference_in_the_20...

golergka 2y ago

Photoshop has been a thing for over 30 years.

EForEndeavour 2y ago
Isn't the whole point of OP that we're currently watching the barrier to generating realistic assets go from "spend months grinding Photoshop tutorials" to "type what you want into this box and wait a few minutes"?

onel 2y ago

The same things could be said when everyone could print their own newspapers or books. How would people distinguish between fake and real news? I think we will need the same healthy media diet.

dbbk 2y ago
There wasn't even a healthy media diet before generative AI given the amount of 'fake news' in 2016 and 2020.

dbbk 2y ago

I still don't really know why we're doing this. What is the upside? Democratising Hollywood? At the expense of... enormous catastrophic disinformation and media manipulation.