7 ms·

We're really in an era were laws and their enforcement will have a lot of catching up to do very fast. Fake historical proof, fake leaks, fake endorsments, fak

by nolok 2y ago

We're really in an era were laws and their enforcement will have a lot of catching up to do very fast. Fake historical proof, fake leaks, fake endorsments, fake ads ... People couldn't be bothered to double check when it was mere random text article on facetok, it's going to be so much worse ...

throwthrowuknow 2y ago
A digital audio file is not even close to being proof of anything. Even without voice cloning you can easily edit, clip and compose audio into almost anything you want. It’s also not difficult to simply impersonate someone else’s manner of speaking with practice something that is commonly done by both amateurs and professional actors. The only thing that changes is the ease with which this can be done which should help everyone understand how unreliable such “proof” is.
- colecut 2y ago
  When dealing with social conflicts, there is very rarely "proof", just evidence In courts of law, digital tapes have been frequently used as evidence.
  - exe34 2y ago
    The thing about evidence is that it can be contested. I imagine that in the cases you refer to, one side presented them as evidence and the other side said oh bugger, you got me. If they had reason to believe the audio had been tempered with, or entirely fabricated, they would then bring expert witnesses and eventually the other side would have to offer more evidence that it was not tampered - often this sort of thing comes down to "under oath, I swear I recorded this at the date/time and the subject on the tape is $name". This might not be enough to convict or exonerate, but it'll then count as part of the case.
    - close04 2y ago
      > The thing about evidence is that it can be contested. The danger is in the court of public opinion where just having the conversation is damage enough. If you have to go out there and fight it then you give it even more visibility and it takes your focus and resources away from more useful activities. At the end of the day many people only remember the accusation, not what comes after.
      - exe34 2y ago
        > At the end of the day many people only remember the accusation, not what comes after. Never a better time to learn the dichotomy of control. What is in your control is your thoughts and your actions. Outside of your control are: you body, your reputation, your wealth, you health, etc. It matters what you choose to do, but the outcomes are out of your control.
  - bo1024 2y ago
    One thing that surprised me, as a juror, was that nobody ever simply submitted a piece of evidence. Each of the hundreds of exhibits was presented with a person on the stand attesting to its provenance, under penalty of perjury.
- croes 2y ago
  Easily for people who what to do. Now it's easy for everyone.
- nolok 2y ago
  I don't know the political situation in your country, but all I will say is that the putin-aligned far right wing in mine uses very much fake quote and deep fake videos and things like that to propagate ideas, and that their "followers" eat it up, and then you either have to let those non truth remain or you spend all your energy fighting them / defending yourself, making you look guilty. And in the past 3/4 years (and it exploded with covid), it's the followers themselves that now start those things. For AI, those AI move it to "someone with some dedication can do it" to "anyone with a computer can do it".
  - SoftTalker 2y ago
    They do it here too, especially obvious in political campaign ads but no doubt the established powers do it to maintain their own image as well.
- tyingq 2y ago
  Legal opinions don't seem to have caught up... https://www.justice.gov/archives/jm/criminal-resource-manual-258-admissibility-spectrograms-voice-prints https://www.justice.gov/archives/jm/criminal-resource-manual...
- telesilla 2y ago
  Sounds like a remake of Sneakers is needed--with a fresh take on impersonation and social engineering, to remind people what's possible and potentially dangerous.
- jilijeanlouis 2y ago
  There are actually a big opportunity for companies like loccus: https://www.loccus.ai/ https://www.loccus.ai/
jorvi 2y ago
From hypernormalisation to hyperreality. I’ve been telling my friends that 5-10 years from now, the only things that you’ll be able to ~100% trust is what happens in front of your eyes, in that very moment. You can elect to trust reliable news organizations to vet things for you, but even if you do, due to polarization a huge subset of the world will think you’ve been got and discard everything as fake. Look at stuff like Sora, or all the new voice models coming out. Just a few days ago there was a high school athletic coach (!) arrested for cloning the principal’s voice using that to say vile stuff. He only got caught because he used his own e-mail. Now combine that with the fact that Microsoft’s new Phi-mini model approaches GPT-3.5 performance using 3.8 billion parameters, whilst GPT-3.5 uses 175 billion. And we’re only ~5 years of optimization into this tech. I want to get off Mr Bones’ wild ride.
- dudefeliciano 2y ago
  > you’ll be able to ~100% trust is what happens in front of your eyes 5-10 years from now we may have vision pro gen 5, or whatever system achieves market dominance, between our eyes and what happens in front of us.
  - ben_w 2y ago
    Even without that, we have at least one crowd funded YouTuber growing mammalian cells in vats with the explicit intention of making a "meat robot". https://youtu.be/Z_ZGq8Tah0k?si=GuV1QC50jiv7yHP7 https://youtu.be/Z_ZGq8Tah0k?si=GuV1QC50jiv7yHP7 There's not a huge gap between that and a Westworld style infiltration-by-printed-robot.
    - justapassenger 2y ago
      The gap is as huge as between ancient Greeks discovering planets and us having colony on Mars.
      - ben_w 2y ago
        You think that the gap between "a meat robot" and "a meat robot of a specific shape" is that big? (If you're thinking of the robots' brains in the TV show, sure, that's a totally different challenge; but you don't need them for a replacement-takeover scenario, there's a lot of other ways to do that).
simion314 2y ago
>We're really in an era were laws and their enforcement will have a lot of catching up to do very fast. I mean there are existing laws that would apply, there are laws against scamming or whatever bad stuff you can do with this. The world might need a law though to force media to label AI generated content and IMO a law where media and users would take responsability for their content. What I mean if say as a random user I claim something in a media post like "eating shit cures you by covid" I should either be forced to add "btw I am not taking any respectability for my comment and I am not a medic/lawyer/expert" otherwise I should be forced to pay damages is someone sues me for my bullshit claims. So cloning voices should be legal, sscamming people is illegal.
andy99 2y ago
I've come around to the idea that the hype around criminal or bad actor uses of AI is the same as the hype around other uses. Some real uses will shake out but the delta between what's actually enabled by the tech and what was possible anyway is way smaller than people like to represent.
- nolok 2y ago
  I am not talking about criminals or similar, for that I agree with you. I am talking about political. There has been a MASSIVE explosion of fakeness since covid, coupled with people who completely lost their ability to actually check information, and anything that makes it easier for anyone to propagate is dangerous in my view. We've reached the point where "check by yourself / do your own research" moved from what the proponent of the truth used to tell people to go check, to what the other side is now using to give as much value to their facetok videos and whatnot (I am speaking in the general sense).
  - psychoslave 2y ago
    Or an other perspective is that politics and religions are just byproducts of propaganda mesmerizing people. So they are already many battle proofed efficient way to brain wash people, and probably many more furtive and cost effective than these new stuffs. Thankfully you still can rely on fellow HN commenters like myself to bring you thoughts free of any mind binding manipulations and exempt of sarcasm.
  - cruffle_duffle 2y ago
    > There has been a MASSIVE explosion of fakeness since covid, coupled with people who completely lost their ability to actually check information, and anything that makes it easier for anyone to propagate is dangerous in my view. The best part about this statement is it doesn’t even matter what “side” you are describing because all “sides” are subject to the same issue. Hell for more than a year every covid article from New York Times quoted an obscenely high 4% “death rate” despite it being off by multiple orders of magnitude. And if pressed they’d be able to easily “prove” their rate by citing some dodgy data model, using badly collected data, or just not understanding how to interpret the data. And that “truth” propagated by the NYT was cited all over causing all kinds of wildly over the top responses. We live in a post truth world. You can always find some authoritative sounding source to support whatever it is you want to believe. There is always a way to massage the data and “facts” in a way that supports your claim as the “true truth”. It does even matter what “team” you are on… you’ll always be able to craft “the truth” by selectively using whatever source material you want. I don’t even know how you’d solve this problem because it is a core issue with trust. It’s very, very hard to establish “knowledge trust” on something like the internet.
andsoitis 2y ago
Trust is a dependency for human existence. Not just for civilization, but also very small communities and basic exchange of ideas, goods, and services. I cannot foretell how the risk of trust destruction from GenAI will unfold, but I'm optimistic our creativity will win out.
- froh 2y ago
  yes. that depndency is why it's even in the ten commandments: don't give false testimony / don't slander, don't lie. https://en.wikipedia.org/wiki/Thou_shalt_not_bear_false_witness_against_thy_neighbour https://en.wikipedia.org/wiki/Thou_shalt_not_bear_false_witn... but instead they discuss who I've married. sigh. anyhow: I share your optimism.
- bloopernova 2y ago
  This is partly why I'm so fascinated and disgusted by trolls and astroturfers. They erode trust in a given forum, which degrades the quality of discourse because no one wants to invest time in untrustworthy discussions. Sometimes I wish I could get an honest answer from trolls about what they hope to achieve, but of course that will never happen.
  - tetraca 2y ago
    > Sometimes I wish I could get an honest answer from trolls about what they hope to achieve, but of course that will never happen. It's usually not that complicated: They enjoy provoking people, particularly people that can be reflexively upset by reading words. It's a game to them, against a party that they do not respect. The words they say might upset you, but the words ultimately mean nothing to them outside of provoking you, and the more chaotic they can make the situation the more amusing it is.
    - maksimur 2y ago
      2 possible additional explanations for trolling: 1. Spite/damage a community for perceived or effective injustice/discrimination. I remember this happening on Reddit and Stack Overflow circa 2010-2014. 2. Actually a neurodivergent person mistaken for a troll. I remember this happening on HN. Remember him being either schizophrenic or autistic. Another example might have been the creator of Temple OS.
corobo 2y ago
Honestly it couldn't hurt for people to re-learn not to trust things on the internet. I haven't heard someone snarkily say "it's on the internet so it must be true" in like 12 years. Let's bring that back.
- ranger_danger 2y ago
  Really? I hear it all the time. I mean, if it's on the Internet it must be true.
ActionHank 2y ago
These are big issues, but I would say that a bigger issue is the case where a spam caller has you on the line talking for ~10 seconds and then calls the bank or family member as you. Android and iOS should support real time voice changers as the norm with a quick switch button on the dialer to disable it and an option to have it off for known contacts.

by nolok 2y ago

throwthrowuknow 2y ago

A digital audio file is not even close to being proof of anything. Even without voice cloning you can easily edit, clip and compose audio into almost anything you want. It’s also not difficult to simply impersonate someone else’s manner of speaking with practice something that is commonly done by both amateurs and professional actors. The only thing that changes is the ease with which this can be done which should help everyone understand how unreliable such “proof” is.

colecut 2y ago
When dealing with social conflicts, there is very rarely "proof", just evidence In courts of law, digital tapes have been frequently used as evidence.
- exe34 2y ago
  The thing about evidence is that it can be contested. I imagine that in the cases you refer to, one side presented them as evidence and the other side said oh bugger, you got me. If they had reason to believe the audio had been tempered with, or entirely fabricated, they would then bring expert witnesses and eventually the other side would have to offer more evidence that it was not tampered - often this sort of thing comes down to "under oath, I swear I recorded this at the date/time and the subject on the tape is $name". This might not be enough to convict or exonerate, but it'll then count as part of the case.
  - close04 2y ago
    > The thing about evidence is that it can be contested. The danger is in the court of public opinion where just having the conversation is damage enough. If you have to go out there and fight it then you give it even more visibility and it takes your focus and resources away from more useful activities. At the end of the day many people only remember the accusation, not what comes after.
    - exe34 2y ago
      > At the end of the day many people only remember the accusation, not what comes after. Never a better time to learn the dichotomy of control. What is in your control is your thoughts and your actions. Outside of your control are: you body, your reputation, your wealth, you health, etc. It matters what you choose to do, but the outcomes are out of your control.
- bo1024 2y ago
  One thing that surprised me, as a juror, was that nobody ever simply submitted a piece of evidence. Each of the hundreds of exhibits was presented with a person on the stand attesting to its provenance, under penalty of perjury.
croes 2y ago
Easily for people who what to do. Now it's easy for everyone.
nolok 2y ago
I don't know the political situation in your country, but all I will say is that the putin-aligned far right wing in mine uses very much fake quote and deep fake videos and things like that to propagate ideas, and that their "followers" eat it up, and then you either have to let those non truth remain or you spend all your energy fighting them / defending yourself, making you look guilty. And in the past 3/4 years (and it exploded with covid), it's the followers themselves that now start those things. For AI, those AI move it to "someone with some dedication can do it" to "anyone with a computer can do it".
- SoftTalker 2y ago
  They do it here too, especially obvious in political campaign ads but no doubt the established powers do it to maintain their own image as well.
tyingq 2y ago
Legal opinions don't seem to have caught up... https://www.justice.gov/archives/jm/criminal-resource-manual-258-admissibility-spectrograms-voice-prints https://www.justice.gov/archives/jm/criminal-resource-manual...
telesilla 2y ago
Sounds like a remake of Sneakers is needed--with a fresh take on impersonation and social engineering, to remind people what's possible and potentially dangerous.
jilijeanlouis 2y ago
There are actually a big opportunity for companies like loccus: https://www.loccus.ai/ https://www.loccus.ai/

jorvi 2y ago

From hypernormalisation to hyperreality. I’ve been telling my friends that 5-10 years from now, the only things that you’ll be able to ~100% trust is what happens in front of your eyes, in that very moment. You can elect to trust reliable news organizations to vet things for you, but even if you do, due to polarization a huge subset of the world will think you’ve been got and discard everything as fake. Look at stuff like Sora, or all the new voice models coming out. Just a few days ago there was a high school athletic coach (!) arrested for cloning the principal’s voice using that to say vile stuff. He only got caught because he used his own e-mail. Now combine that with the fact that Microsoft’s new Phi-mini model approaches GPT-3.5 performance using 3.8 billion parameters, whilst GPT-3.5 uses 175 billion. And we’re only ~5 years of optimization into this tech. I want to get off Mr Bones’ wild ride.

dudefeliciano 2y ago
> you’ll be able to ~100% trust is what happens in front of your eyes 5-10 years from now we may have vision pro gen 5, or whatever system achieves market dominance, between our eyes and what happens in front of us.
- ben_w 2y ago
  Even without that, we have at least one crowd funded YouTuber growing mammalian cells in vats with the explicit intention of making a "meat robot". https://youtu.be/Z_ZGq8Tah0k?si=GuV1QC50jiv7yHP7 https://youtu.be/Z_ZGq8Tah0k?si=GuV1QC50jiv7yHP7 There's not a huge gap between that and a Westworld style infiltration-by-printed-robot.
  - justapassenger 2y ago
    The gap is as huge as between ancient Greeks discovering planets and us having colony on Mars.
    - ben_w 2y ago
      You think that the gap between "a meat robot" and "a meat robot of a specific shape" is that big? (If you're thinking of the robots' brains in the TV show, sure, that's a totally different challenge; but you don't need them for a replacement-takeover scenario, there's a lot of other ways to do that).

simion314 2y ago

>We're really in an era were laws and their enforcement will have a lot of catching up to do very fast. I mean there are existing laws that would apply, there are laws against scamming or whatever bad stuff you can do with this. The world might need a law though to force media to label AI generated content and IMO a law where media and users would take responsability for their content. What I mean if say as a random user I claim something in a media post like "eating shit cures you by covid" I should either be forced to add "btw I am not taking any respectability for my comment and I am not a medic/lawyer/expert" otherwise I should be forced to pay damages is someone sues me for my bullshit claims. So cloning voices should be legal, sscamming people is illegal.

andy99 2y ago

I've come around to the idea that the hype around criminal or bad actor uses of AI is the same as the hype around other uses. Some real uses will shake out but the delta between what's actually enabled by the tech and what was possible anyway is way smaller than people like to represent.

nolok 2y ago
I am not talking about criminals or similar, for that I agree with you. I am talking about political. There has been a MASSIVE explosion of fakeness since covid, coupled with people who completely lost their ability to actually check information, and anything that makes it easier for anyone to propagate is dangerous in my view. We've reached the point where "check by yourself / do your own research" moved from what the proponent of the truth used to tell people to go check, to what the other side is now using to give as much value to their facetok videos and whatnot (I am speaking in the general sense).
- psychoslave 2y ago
  Or an other perspective is that politics and religions are just byproducts of propaganda mesmerizing people. So they are already many battle proofed efficient way to brain wash people, and probably many more furtive and cost effective than these new stuffs. Thankfully you still can rely on fellow HN commenters like myself to bring you thoughts free of any mind binding manipulations and exempt of sarcasm.
- cruffle_duffle 2y ago
  > There has been a MASSIVE explosion of fakeness since covid, coupled with people who completely lost their ability to actually check information, and anything that makes it easier for anyone to propagate is dangerous in my view. The best part about this statement is it doesn’t even matter what “side” you are describing because all “sides” are subject to the same issue. Hell for more than a year every covid article from New York Times quoted an obscenely high 4% “death rate” despite it being off by multiple orders of magnitude. And if pressed they’d be able to easily “prove” their rate by citing some dodgy data model, using badly collected data, or just not understanding how to interpret the data. And that “truth” propagated by the NYT was cited all over causing all kinds of wildly over the top responses. We live in a post truth world. You can always find some authoritative sounding source to support whatever it is you want to believe. There is always a way to massage the data and “facts” in a way that supports your claim as the “true truth”. It does even matter what “team” you are on… you’ll always be able to craft “the truth” by selectively using whatever source material you want. I don’t even know how you’d solve this problem because it is a core issue with trust. It’s very, very hard to establish “knowledge trust” on something like the internet.

andsoitis 2y ago

Trust is a dependency for human existence. Not just for civilization, but also very small communities and basic exchange of ideas, goods, and services. I cannot foretell how the risk of trust destruction from GenAI will unfold, but I'm optimistic our creativity will win out.

froh 2y ago
yes. that depndency is why it's even in the ten commandments: don't give false testimony / don't slander, don't lie. https://en.wikipedia.org/wiki/Thou_shalt_not_bear_false_witness_against_thy_neighbour https://en.wikipedia.org/wiki/Thou_shalt_not_bear_false_witn... but instead they discuss who I've married. sigh. anyhow: I share your optimism.
bloopernova 2y ago
This is partly why I'm so fascinated and disgusted by trolls and astroturfers. They erode trust in a given forum, which degrades the quality of discourse because no one wants to invest time in untrustworthy discussions. Sometimes I wish I could get an honest answer from trolls about what they hope to achieve, but of course that will never happen.
- tetraca 2y ago
  > Sometimes I wish I could get an honest answer from trolls about what they hope to achieve, but of course that will never happen. It's usually not that complicated: They enjoy provoking people, particularly people that can be reflexively upset by reading words. It's a game to them, against a party that they do not respect. The words they say might upset you, but the words ultimately mean nothing to them outside of provoking you, and the more chaotic they can make the situation the more amusing it is.
  - maksimur 2y ago
    2 possible additional explanations for trolling: 1. Spite/damage a community for perceived or effective injustice/discrimination. I remember this happening on Reddit and Stack Overflow circa 2010-2014. 2. Actually a neurodivergent person mistaken for a troll. I remember this happening on HN. Remember him being either schizophrenic or autistic. Another example might have been the creator of Temple OS.

corobo 2y ago

Honestly it couldn't hurt for people to re-learn not to trust things on the internet. I haven't heard someone snarkily say "it's on the internet so it must be true" in like 12 years. Let's bring that back.

ranger_danger 2y ago
Really? I hear it all the time. I mean, if it's on the Internet it must be true.

ActionHank 2y ago

These are big issues, but I would say that a bigger issue is the case where a spam caller has you on the line talking for ~10 seconds and then calls the bank or family member as you. Android and iOS should support real time voice changers as the norm with a quick switch button on the dialer to disable it and an option to have it off for known contacts.