4 ms·

There was a brief window where photography and videos became widespread so events could be documented and evidenced. Generative AI is drawing that period to an

There was a brief window where photography and videos became widespread so events could be documented and evidenced. Generative AI is drawing that period to an end, and we're returning to "who said / posted this" and having to trust the source rather than the image/message itself. It's a big regression.

rightbyte 1y ago
Videos and photos have been faked for a long time. Nothing has changed in that regard but decreasing the effort required somewhat.
- jdiff 1y ago
  "Somewhat" is doing an awful lot of heavy lifting. Fakery has gone from handcrafted in small batches (or at worst, produced through much effort in sweatshops) to fully automated and mass produced.
  - rightbyte 1y ago
    Ye well "alot" might have been a better wording. However, the real breaking point in my view was when shills and semi-automated bots become so prevalent that they could fool people into believing some consensus had changed. Faking photos doesn't add much to that in my view.
    - jdiff 1y ago
      Why do you believe that is the case? Aren't standard, naturally-occurring social media echo chambers far more likely?
      - rightbyte 1y ago
        My gut says manipulation. I have no proofs what so ever.
- notTooFarGone 1y ago
  Only the price changed and that is everything.
  - zettapwn 1y ago
    The open question is who’s worse: one major tyrant three thousand miles away or three thousand minor tyrants a mile away. If we consent to live under capitalism, then we’re destined to live in a world of lies. We always have; it’s all we’ve ever known. What we don’t know is whether we’ll be worse or better off when the technology of forgery is available to random broke assholes as easily as it is to governments and companies. More slop seems bad, but if our immunity against bullshit improves, people might redevelop critical thinking skills and capitalism could end.
- EGreg 1y ago
  "Nothing has changed in that regard" really, nothing? Does anyone other than me notice this common tendency on HN: 1. Blockhain use case mentioned? Someone must say "blockchain doesn't solve any problems" no matter what, and always ignoring any actual substance of what's being done until pressed. 2. AI issue mentioned? Someone must say "nothing at all has changed, this problem has always existed" downplaying the vast leaps forward in scale, and quality of output, ignoring any details until pressed. It's like when people feel the need to preface "there is nothing wrong with capitalism, but" before critiquing capitalism. You will not criticize the profit. It's not really a shibboleth. What's the name for this type of thing, groupthink?
  - deleted 1y ago
    [deleted]
  - cb321 1y ago
    I would just call it a "pattern", but if you want to be more specific Re your 1/2, perhaps a "pattern of over-simplification". Over-simplification is, of course, basically human nature, not specific to HN, and something "scientists" of all stripes fight against in themselves. So, there may be a better one-worder. EDIT: while oversimplification is essentially always a problem, nuance and persuasion are usually at odds. So, it's especially noticeable in contexts where people are trying to persuade each other. The best parts of HN are not that, but rather where they try to inform each other.
- raincole 1y ago
  People had been able to send messages to each other for a very long time. However the internet still changed a lot of things.
  - EGreg 1y ago
    I'll admit there were a few people doing the "HN thing" and saying "nothing really changed" when the Internet came out... but it was as a joke: https://www.youtube.com/watch?v=fs-YpQj88ew&t=3m20s https://www.youtube.com/watch?v=fs-YpQj88ew&t=3m20s
- thomasahle 1y ago
  Things like the government MAHA report, full of fake references to papers that don't exist, probably didn't happen as blatantly before AI. Even if people could easily have lied back then too. The ease of with which AI lies (or "hallucinates") makes a qualitative difference.
  - tokai 1y ago
    Fake citations has always been common. Before they weren't just straight up fabrications. The cited papers usually exists, they just don't contain what is claimed by the referring paper. At least made up citations are quick and easy to denounce.
- SupremumLimit 1y ago
  Your comment follows two persistent HN tropes: (1) ignoring the article, which deals precisely with why the cost of production matters, and (2) steadfastly refusing to recognise that quantity has a quality of its own - in this case a monumental reduction in production cost clearly leads to a tectonic reshaping of the information landscape.
  - brookst 1y ago
    …but enough about the printing press.
    - _Algernon_ 1y ago
      Funny that you mention the printing press. One of the first books published using it was about how to identify witches which led to "witch" burnings in Europe. At some point society adapted to the misinformation (and we are better off for it), but a lot of innocent people suffered in the meantime. https://www.independent.co.uk/news/science/archaeology/europe-witch-trials-hunt-printing-press-b2629307.html https://www.independent.co.uk/news/science/archaeology/europ...
  - rightbyte 1y ago
    I did read the article and it does deal with fake photos being an old thing and name drops e.g. Stalin's former companions being removed one by one from photos. "There was a brief window where photography and videos became widespread so events could be documented and evidenced." Photos are video have in themself not been evidence. You need trust the photographer or publisher too, since the camera was invented. Moving the cost from big $$$ level to my neighbour level makes scams more personalized, sure.
- piva00 1y ago
  I get bored by myself repeating it but I always think it's worth to mention: the scale and degree matter, in almost all cases of an issue, if not we can reduce a lot of issues to "it has always happened before". Localised fires are common in nature, a massive wildfire is just a fire, at a different scale and degree. Lunatics raving about conspiracies were very common in public squares, in front of metro stations, anywhere there was a large-ish flow of people, now they are very common in social media, reaching millions in an instant, different scale and degree. Just sheer scale and degree makes an issue a completely other issue, decreasing the effort required to fake a video to the point where a naïve/layperson cannot distinguish it from a real one is a massive difference. Before you needed to be technically proficient with a number of tools, put a lot of work, and get a somewhat satisfying result to fake something, now you just throw a magical incantation of words and it spits out a video you can deceive someone, it's a completely different issue.
Vilian 1y ago
Would be very interesting to require generative AI companies to log their created images for that purpose, it isn't gonna eliminate self hosted alternative, but it can destroy any fake evidence faster
- jdiff 1y ago
  I don't think this would solve the problem, unfortunately. AI can certainly be coerced into reproducing an existing image, leaving plenty of room for probable deniability.
- A_D_E_P_T 1y ago
  Logging them would be rather cost prohibitive, but images can be hashed + (invisibly) watermarked and video can be hashed frame by frame, in such a way that each frame authenticates the one before it. Surely there's a way to durably mark generated content.
  - Iulioh 1y ago
    It works untill someone takes a screenshoot of the image
    - A_D_E_P_T 1y ago
      The watermark would still be there. You can take a photograph of the image with an old Polaroid and, if the resolution is high enough, the watermark would still be there.
      - brookst 1y ago
        If it can be detected, it can be removed. Asking that false things carry claims of falseness is a dead end.
        A_D_E_P_T 1y ago
        Let's say you want to release a deepfake image. Your AI image, as generated, is not watermarked or hashed. Difficulty: Story-mode. You can release your image in two minutes flat. Let's now let's say your AI deepfake image is watermarked/hashed upon generation. Difficulty: Intermediate to hard. You might be able to get it done, but not without real effort, and removing all traces of the watermark without leaving artefacts might be so difficult as to be nearly impossible. ...So it doesn't eliminate the possibility of fakes, but it hugely raises the cost and effort bar. Only the most motivated would bother.
thegreatpeter 1y ago
This has been a problem with the media for years as well. You’re forced to trust the source and “read between the lines” or you’re reading something politically motivated. Nothing new. I hope folks start trusting the source more.
neepi 1y ago
I await cryptographically signed photos coming out of cameras. Actually I’ve said that should be the case for the last 20 years. You should be able to follow a chain of evidence towards an unprocessed raw image potentially.
- thih9 1y ago
  There is ongoing work[1] and cameras that offer some support already[2]. [1]: https://en.m.wikipedia.org/wiki/Content_Authenticity_Initiative https://en.m.wikipedia.org/wiki/Content_Authenticity_Initiat... [2]: https://leica-camera.com/en-int/photography/content-credentials https://leica-camera.com/en-int/photography/content-credenti...
- k2enemy 1y ago
  There have been several initiatives in this direction. Here is one of the latest: https://contentauthenticity.org https://contentauthenticity.org
- holowoodman 1y ago
  There have also been calls for a mechanism like this to prevent doctored photos of controversial newsworthy events being spread by news agencies. But afaik the only thing that came off it was "we only pay for camera jpg, only allowed changes are brightness, contrast, color". Edit: sister comments have links.
- bayindirh 1y ago
  I think Nikon and Canon's past cameras had signed photos by default, and you could get the software for verification if you're a police dept or similar. Both manufacturer's keys got extracted from their cameras, rendering the feature moot. Now, a more robust iteration is probably coming, possibly based on secure enclaves. I'd love to have the feature and use it to verify that my images are not corrupted on disk.
  - neepi 1y ago
    Used to work on archival. You want to store your images in TIFF and use FEC if you really care. That withstands significant bitrot. Signing will just tell you that the image is broken but not allow you to recover it. You can do fine with SHA256 and 3-2-1 backup.
RcouF1uZ4gsC 1y ago
Remember the fairy photo hoax that fooled Lewis Carrol? There was never a time when authenticated photo and video could be trusted without knowing the source and circumstances.
- j-bos 1y ago
  It fooled Sir Arthur Conan Doyle, a motivated believer, not Lewis Carrol. People will believe what they want. Trust is the fundamental issue.
  - psychoslave 1y ago
    Lewis Carroll also was fooled by it, when Churchill showed to him. Abraham Lincoln who was there at the moment it happened confirmed that to me, I can show you the original email he sent me about it (bar the elements I'll have to hide due to top secret information being included in the rest of the message).
- const_cast 1y ago
  Sure, but scale matters. 99% of images being fake is a different situation than 1% being fake. We can't just ignore that in favor of a "this always happened" argument. Everything has always happened, so who cares? We need to go deeper than that. Many things that are perfectly a-okay today are only so because we do it on a small enough scale. Many perfectly fine things, if we scale them up, destroy the world. Literally. Even something as simple as pirating, which I support, would melt all world economies if everyone did it with everything.
ThinkBeat 1y ago
It is more that it is becoming available to the masses. Doctoring photos in all manner of ways has been going on for decades. What is happening now will raise awareness of it and of course make it a several magnitudes bigger problem. I am sure there are large efforts ongoing to train AI to spot AI photo, video, written production. A system like printer tracking dots¹ may already be in widespread use. Who would take the enormous amount of time to figure out if somesuch is hiding somewhere in an llm, or related code. ¹ https://en.wikipedia.org/wiki/Printer_tracking_dots https://en.wikipedia.org/wiki/Printer_tracking_dots
- perching_aix 1y ago
  A mate of mine told me that ChatGPT started injecting zero width spaces into its output. Never fact checked this, but even if it's not done now, I'm sure it's bound to happen. Same for other types of watermarks.
  - HPsquared 1y ago
    Quite easy to remove something like that. Although ChatGPT has a writing style that's quite recognisable.
- const_cast 1y ago
  Everything is a function of scale. IMO saying "this always happened" means nothing. Lying on a small scale is no big deal. Lying on a big scale burns the world down. Me pirating Super Mario 64 means nothing. Everyone pirating everything burns the economy down. Me shooting a glass coke bottle is not note worthy. Nuclear warheads threaten humanity's existence. Yes, AI fabrication is a huge problem that we have never experienced before, precisely because of how it can scale.
palmotea 1y ago
> Generative AI is drawing that period to an end, and we're returning to "who said / posted this" and having to trust the source rather than the image/message itself. It's a big regression. Which just goes to show that one of the core tenets of techno-optimism is a lie. The Amish (as I understand them) actually have the right idea: instead of wantonly adopting every technology that comes along, they assess how each one affects their values and culture, and pick the ones that help and reject the ones that harm.
- AndrewKemendo 1y ago
  Which implies that they have a foundational epistemological, teleological and “coherent” philosophy Not something that can be said of most people. Worse, the number of affinity groups with long term coherence collapses into niche religions and regional cults. There’s no free lunch If you want structural stability then you’re gonna have to give up individuality for the group vector, if you want individuality you’re not going be able to take advantage of group benefits. Humans are never going to be able to figure out this balance because we don’t have any kind of foundational coherent Universal epistemological grounding that can be universally accepted. Good luck trying to get the world to even agree on the age of the earth

rightbyte 1y ago

Videos and photos have been faked for a long time. Nothing has changed in that regard but decreasing the effort required somewhat.

jdiff 1y ago
"Somewhat" is doing an awful lot of heavy lifting. Fakery has gone from handcrafted in small batches (or at worst, produced through much effort in sweatshops) to fully automated and mass produced.
- rightbyte 1y ago
  Ye well "alot" might have been a better wording. However, the real breaking point in my view was when shills and semi-automated bots become so prevalent that they could fool people into believing some consensus had changed. Faking photos doesn't add much to that in my view.
  - jdiff 1y ago
    Why do you believe that is the case? Aren't standard, naturally-occurring social media echo chambers far more likely?
    - rightbyte 1y ago
      My gut says manipulation. I have no proofs what so ever.
notTooFarGone 1y ago
Only the price changed and that is everything.
- zettapwn 1y ago
  The open question is who’s worse: one major tyrant three thousand miles away or three thousand minor tyrants a mile away. If we consent to live under capitalism, then we’re destined to live in a world of lies. We always have; it’s all we’ve ever known. What we don’t know is whether we’ll be worse or better off when the technology of forgery is available to random broke assholes as easily as it is to governments and companies. More slop seems bad, but if our immunity against bullshit improves, people might redevelop critical thinking skills and capitalism could end.
EGreg 1y ago
"Nothing has changed in that regard" really, nothing? Does anyone other than me notice this common tendency on HN: 1. Blockhain use case mentioned? Someone must say "blockchain doesn't solve any problems" no matter what, and always ignoring any actual substance of what's being done until pressed. 2. AI issue mentioned? Someone must say "nothing at all has changed, this problem has always existed" downplaying the vast leaps forward in scale, and quality of output, ignoring any details until pressed. It's like when people feel the need to preface "there is nothing wrong with capitalism, but" before critiquing capitalism. You will not criticize the profit. It's not really a shibboleth. What's the name for this type of thing, groupthink?
- deleted 1y ago
  [deleted]
- cb321 1y ago
  I would just call it a "pattern", but if you want to be more specific Re your 1/2, perhaps a "pattern of over-simplification". Over-simplification is, of course, basically human nature, not specific to HN, and something "scientists" of all stripes fight against in themselves. So, there may be a better one-worder. EDIT: while oversimplification is essentially always a problem, nuance and persuasion are usually at odds. So, it's especially noticeable in contexts where people are trying to persuade each other. The best parts of HN are not that, but rather where they try to inform each other.
raincole 1y ago
People had been able to send messages to each other for a very long time. However the internet still changed a lot of things.
- EGreg 1y ago
  I'll admit there were a few people doing the "HN thing" and saying "nothing really changed" when the Internet came out... but it was as a joke: https://www.youtube.com/watch?v=fs-YpQj88ew&t=3m20s https://www.youtube.com/watch?v=fs-YpQj88ew&t=3m20s
thomasahle 1y ago
Things like the government MAHA report, full of fake references to papers that don't exist, probably didn't happen as blatantly before AI. Even if people could easily have lied back then too. The ease of with which AI lies (or "hallucinates") makes a qualitative difference.
- tokai 1y ago
  Fake citations has always been common. Before they weren't just straight up fabrications. The cited papers usually exists, they just don't contain what is claimed by the referring paper. At least made up citations are quick and easy to denounce.
SupremumLimit 1y ago
Your comment follows two persistent HN tropes: (1) ignoring the article, which deals precisely with why the cost of production matters, and (2) steadfastly refusing to recognise that quantity has a quality of its own - in this case a monumental reduction in production cost clearly leads to a tectonic reshaping of the information landscape.
- brookst 1y ago
  …but enough about the printing press.
  - _Algernon_ 1y ago
    Funny that you mention the printing press. One of the first books published using it was about how to identify witches which led to "witch" burnings in Europe. At some point society adapted to the misinformation (and we are better off for it), but a lot of innocent people suffered in the meantime. https://www.independent.co.uk/news/science/archaeology/europe-witch-trials-hunt-printing-press-b2629307.html https://www.independent.co.uk/news/science/archaeology/europ...
- rightbyte 1y ago
  I did read the article and it does deal with fake photos being an old thing and name drops e.g. Stalin's former companions being removed one by one from photos. "There was a brief window where photography and videos became widespread so events could be documented and evidenced." Photos are video have in themself not been evidence. You need trust the photographer or publisher too, since the camera was invented. Moving the cost from big $$$ level to my neighbour level makes scams more personalized, sure.
piva00 1y ago
I get bored by myself repeating it but I always think it's worth to mention: the scale and degree matter, in almost all cases of an issue, if not we can reduce a lot of issues to "it has always happened before". Localised fires are common in nature, a massive wildfire is just a fire, at a different scale and degree. Lunatics raving about conspiracies were very common in public squares, in front of metro stations, anywhere there was a large-ish flow of people, now they are very common in social media, reaching millions in an instant, different scale and degree. Just sheer scale and degree makes an issue a completely other issue, decreasing the effort required to fake a video to the point where a naïve/layperson cannot distinguish it from a real one is a massive difference. Before you needed to be technically proficient with a number of tools, put a lot of work, and get a somewhat satisfying result to fake something, now you just throw a magical incantation of words and it spits out a video you can deceive someone, it's a completely different issue.

Vilian 1y ago

Would be very interesting to require generative AI companies to log their created images for that purpose, it isn't gonna eliminate self hosted alternative, but it can destroy any fake evidence faster

jdiff 1y ago
I don't think this would solve the problem, unfortunately. AI can certainly be coerced into reproducing an existing image, leaving plenty of room for probable deniability.
A_D_E_P_T 1y ago
Logging them would be rather cost prohibitive, but images can be hashed + (invisibly) watermarked and video can be hashed frame by frame, in such a way that each frame authenticates the one before it. Surely there's a way to durably mark generated content.
- Iulioh 1y ago
  It works untill someone takes a screenshoot of the image
  - A_D_E_P_T 1y ago
    The watermark would still be there. You can take a photograph of the image with an old Polaroid and, if the resolution is high enough, the watermark would still be there.
    - brookst 1y ago
      If it can be detected, it can be removed. Asking that false things carry claims of falseness is a dead end.
      - A_D_E_P_T 1y ago
        Let's say you want to release a deepfake image. Your AI image, as generated, is not watermarked or hashed. Difficulty: Story-mode. You can release your image in two minutes flat. Let's now let's say your AI deepfake image is watermarked/hashed upon generation. Difficulty: Intermediate to hard. You might be able to get it done, but not without real effort, and removing all traces of the watermark without leaving artefacts might be so difficult as to be nearly impossible. ...So it doesn't eliminate the possibility of fakes, but it hugely raises the cost and effort bar. Only the most motivated would bother.

thegreatpeter 1y ago

This has been a problem with the media for years as well. You’re forced to trust the source and “read between the lines” or you’re reading something politically motivated. Nothing new. I hope folks start trusting the source more.

neepi 1y ago

I await cryptographically signed photos coming out of cameras. Actually I’ve said that should be the case for the last 20 years. You should be able to follow a chain of evidence towards an unprocessed raw image potentially.

thih9 1y ago
There is ongoing work[1] and cameras that offer some support already[2]. [1]: https://en.m.wikipedia.org/wiki/Content_Authenticity_Initiative https://en.m.wikipedia.org/wiki/Content_Authenticity_Initiat... [2]: https://leica-camera.com/en-int/photography/content-credentials https://leica-camera.com/en-int/photography/content-credenti...
k2enemy 1y ago
There have been several initiatives in this direction. Here is one of the latest: https://contentauthenticity.org https://contentauthenticity.org
holowoodman 1y ago
There have also been calls for a mechanism like this to prevent doctored photos of controversial newsworthy events being spread by news agencies. But afaik the only thing that came off it was "we only pay for camera jpg, only allowed changes are brightness, contrast, color". Edit: sister comments have links.
bayindirh 1y ago
I think Nikon and Canon's past cameras had signed photos by default, and you could get the software for verification if you're a police dept or similar. Both manufacturer's keys got extracted from their cameras, rendering the feature moot. Now, a more robust iteration is probably coming, possibly based on secure enclaves. I'd love to have the feature and use it to verify that my images are not corrupted on disk.
- neepi 1y ago
  Used to work on archival. You want to store your images in TIFF and use FEC if you really care. That withstands significant bitrot. Signing will just tell you that the image is broken but not allow you to recover it. You can do fine with SHA256 and 3-2-1 backup.

RcouF1uZ4gsC 1y ago

Remember the fairy photo hoax that fooled Lewis Carrol? There was never a time when authenticated photo and video could be trusted without knowing the source and circumstances.

j-bos 1y ago
It fooled Sir Arthur Conan Doyle, a motivated believer, not Lewis Carrol. People will believe what they want. Trust is the fundamental issue.
- psychoslave 1y ago
  Lewis Carroll also was fooled by it, when Churchill showed to him. Abraham Lincoln who was there at the moment it happened confirmed that to me, I can show you the original email he sent me about it (bar the elements I'll have to hide due to top secret information being included in the rest of the message).
const_cast 1y ago
Sure, but scale matters. 99% of images being fake is a different situation than 1% being fake. We can't just ignore that in favor of a "this always happened" argument. Everything has always happened, so who cares? We need to go deeper than that. Many things that are perfectly a-okay today are only so because we do it on a small enough scale. Many perfectly fine things, if we scale them up, destroy the world. Literally. Even something as simple as pirating, which I support, would melt all world economies if everyone did it with everything.

ThinkBeat 1y ago

It is more that it is becoming available to the masses. Doctoring photos in all manner of ways has been going on for decades. What is happening now will raise awareness of it and of course make it a several magnitudes bigger problem. I am sure there are large efforts ongoing to train AI to spot AI photo, video, written production. A system like printer tracking dots¹ may already be in widespread use. Who would take the enormous amount of time to figure out if somesuch is hiding somewhere in an llm, or related code. ¹ https://en.wikipedia.org/wiki/Printer_tracking_dots https://en.wikipedia.org/wiki/Printer_tracking_dots

perching_aix 1y ago
A mate of mine told me that ChatGPT started injecting zero width spaces into its output. Never fact checked this, but even if it's not done now, I'm sure it's bound to happen. Same for other types of watermarks.
- HPsquared 1y ago
  Quite easy to remove something like that. Although ChatGPT has a writing style that's quite recognisable.
const_cast 1y ago
Everything is a function of scale. IMO saying "this always happened" means nothing. Lying on a small scale is no big deal. Lying on a big scale burns the world down. Me pirating Super Mario 64 means nothing. Everyone pirating everything burns the economy down. Me shooting a glass coke bottle is not note worthy. Nuclear warheads threaten humanity's existence. Yes, AI fabrication is a huge problem that we have never experienced before, precisely because of how it can scale.

palmotea 1y ago

> Generative AI is drawing that period to an end, and we're returning to "who said / posted this" and having to trust the source rather than the image/message itself. It's a big regression. Which just goes to show that one of the core tenets of techno-optimism is a lie. The Amish (as I understand them) actually have the right idea: instead of wantonly adopting every technology that comes along, they assess how each one affects their values and culture, and pick the ones that help and reject the ones that harm.

AndrewKemendo 1y ago
Which implies that they have a foundational epistemological, teleological and “coherent” philosophy Not something that can be said of most people. Worse, the number of affinity groups with long term coherence collapses into niche religions and regional cults. There’s no free lunch If you want structural stability then you’re gonna have to give up individuality for the group vector, if you want individuality you’re not going be able to take advantage of group benefits. Humans are never going to be able to figure out this balance because we don’t have any kind of foundational coherent Universal epistemological grounding that can be universally accepted. Good luck trying to get the world to even agree on the age of the earth