5 ms·

Can someone pleasee convince why i shouldn't be absolutely shit out of my mind cynical about this innovation? we are literally seeing the downfall of trust in s

Can someone pleasee convince why i shouldn't be absolutely shit out of my mind cynical about this innovation? we are literally seeing the downfall of trust in society. and no, i dont believe i am exaggerating

bcherny 2y ago
Luckily, the same person bringing this tech to market (Sam Altman) sells the solution to the trust problems his tech will create (WorldCoin).
- skybrian 2y ago
  If you mean voice cloning, they aren't bringing that tech to market. (Someone else will, though.) Similarly, Google doesn't let you use face matching in image search to find every photograph of you on the web, even though they could, and quite similar technology is built into Google Photos.
iaaan 2y ago
If everyone is too cynical, then companies like OpenAI won't be able to make any money on it. You wouldn't want that, right??
throwaway48540 2y ago
We are just finally seeing the mainstream get the idea they shouldn't trust anything on the internet. Long overdue.
- Panoramix 2y ago
  Before these developments it was impossible to fake a politician saying arbitrary stuff. This is a major shift, there's no denying that.
  - throwaway48540 2y ago
    That's the thing though, it wasn't impossible, just harder - but people believed whatever they saw, so it was done regardless of the costs. Now it's easier and people are finally getting sceptical, as they should've 20+ years ago.
  - zaptrem 2y ago
    Deep faking voices has been real and super easy for years via 11labs etc. This doesn’t change that at all.
  - dragonwriter 2y ago
    > Before these developments it was impossible to fake a politician saying arbitrary stuff. Voice impersonation has been possible forever, actually; and it use for misinformation (as well as less nefarious things like entertainment) is hardly novel. (Same with impersonation that goes beyond voice.)
aimazon 2y ago
Attacks on trust have been around for as long as we have had trust. Generative AI makes some attack vectors easier but it's nothing new: if your trust is earned using a voice you recognise then you model for trust has been broken since before most of us were born. https://en.wikipedia.org/wiki/Impersonator https://en.wikipedia.org/wiki/Impersonator
AmericanChopper 2y ago
The way I see it LLMs are just making it more obvious to see all the flaws with the existing levels of trust. Humans have never had access to universal truths, or universal ways of validating anything. Any claim anybody makes could be intentionally or unintentionally deceitful or untrue. The idea that there are sources you can trust to do your thinking for you is the more dangerous illusion in my opinion, and I’m not convinced that society will be harmed by poking some holes through it.
- tsimionescu 2y ago
  > The idea that there are sources you can trust to do your thinking for you is the more dangerous illusion in my opinion, and I’m not convinced that society will be harmed by poking some holes through it There is no alternative to this idea. It is completely impossible for an individual to possess all of the knowledge of everything that affects their lives. The only option for getting some of this information is going to trusted sources that compile it and present some conclusions. This applies just as much to scientific knowledge as it does to medicine or to politics. If you want to avoid trusting any authority, it's hard to even confirm that North Korea exists. Confirming that it is ruled by an authoritarian regime and that it possesses nuclear weapons is impossible. And yet it's a trivial bit of info that everyone agrees on - imagine what avoiding trusted authorities would do to knowledge about other more subtle or more controversial topics.
  - AmericanChopper 2y ago
    > There is no alternative to this idea. I said do your thinking for you, not do your information gathering for you. I would suggest that you do not trust any single source to only ever tell you things that are true. If there’s a topic you want to know something about it’s a much better course of action to look at multiple different sources, and do your own thinking to come to your own conclusions. There are no authorities who can reliably take on this role for you, and LLMs don’t change this. The same is true with science. Even prior to LLMs, the replication crisis should have shown that a single paper on any topic can’t be relied on to contain any truth (the same would be true even in the absence of a replication crisis for that matter).
dragonwriter 2y ago
Why would the downfall of the largely unjustified and frequently harmfully exploited kinds of trust that have previously existed be bad?
squigz 2y ago
We have adapted to monumental shifts in how we develop trust in society for as long as society has existed - from the printing press to photography to the Internet to CGI to ... I don't see this as any different. We will determine new ways of establishing trust. They'll certainly have flaws, as establishing trust in a society always has, but we'll learn to recognize those flaws and hopefully fix them. Beyond that, what's the alternative? Banning the technology? That doesn't seem feasible for various reasons, not least of which is it isn't going to stop bad actors. Another pretty good reason is it's just not really possible - anyone with enough compute can build LLMs now. As a bit of an aside, why hasn't society fallen yet? I mean, ChatGPT has been around for a couple years now, and I've been hearing about how LLMs are the single greatest threat to civilized society we've ever faced... yet they don't seem to have had a major impact.
- b112 2y ago
  The big difference is, there was a very high bar to forging photographs, and most news was gated (with the ability to easily find, and sue those guilty of slander/libel). Now it's utterly simplistic to forge, to libel, to slander, and there is no easy path in many cases to sue. While you can say "yes, but..." to the above, that's the reality that we've lived with for 150 years, less extremely rare edge cases. All this has changed over the course of a couple of decades, with most of that change in the last 10, and focuses on the last 2 years. Beyond that, it took significant effort and labour to create fake stories and images. People had to be experts, or be wordsmiths. Now, click click, and fake generated stories abound. In fact, they're literally everywhere. There's absolutely no comparison here. Now, in the time it used to take one person to generate one fake story, you can generate millions and trillions if you have the cash. Really, it's the same problem with spam phone calls, and with spam email. You didn't get 1000 spam letters in the mail in the 80s, because that cost money. Email was free, thus spam became plentiful. The same with spam phone calls, it cost hard cash for each call, now it's pennies per hundreds of automated calls, so spam phone calls abound. The same is happening with all content on the internet. Realistically the web is now dead. It's now gone. Even things such as wikipedia are going to die, as over the next 2 to 3 years LLM output will become utterly and completely indistinguishable in all aspects.
pizzafeelsright 2y ago
I think you're seeing the lack of trust exposed. Contracts allow for recompense if there is mistrust. We've been signing contracts forever. Every automated system needs someone to jail in the event of failure.
cdrini 2y ago
I think the other comments make a good argument about how other forms of technology have also degraded trust, but that we've found a way through. I'll also add that I think one potential way we could reinstate trust is through signed multimedia. Cameras/microphones/etc could sign the videos/audio they create in a way that can be used to verify that the media hasn't been doctored. Not sure if that's actually a feasible approach, but it's one possibility.
- mike_hearn 2y ago
  It's feasible with advanced enough tech. The hard part isn't getting cameras to sign the files they produce. The hard part is to preserve the chain of custody as images are cropped, rescaled, recompressed etc. You can do it with tech like Intel SGX. But you also need serious defense of the camera platforms against hacking, of the CPUs, of the software stacks. And there's no demand. News orgs feel they should be implicitly trusted due to their brands, so why would they use complicated tech to build trust?
  - cdrini 2y ago
    They might use complicated tech because things are changing due to AI in a way that could degrade their brands trust. By having camera-signed videos, when folks create eg deep fakes of their news anchors/brands, there's a way for consumers to verify what's real. It lets their brands become more trustworthy. Yeah, preserving the chain of custody is hard. I was thinking there are a few options: (1) the signature of the original video could be attached even after editing/compression, and then a news org would let you look up the signature. That way if someone copied the signature and stuck it on a fake video, you could see the original video that actually passes with that signature, and determine if something has been doctored. Or (2), you could have editing software add a signature verifying the edits made: eg compression, rescaling etc. And then a law to make it illegal to remove/tamper/valsify a signature, like we have for DVDs, to allow some form of prosecution. The hardware stack is a little easier to protect; the software stack less so. But if we can do it with things like eg browser DRM or http signatures, maybe we can with media editing software? But I'm not versed enough in Cryptography to really know. And cool will read up on intel SGX, thanks!

bcherny 2y ago

Luckily, the same person bringing this tech to market (Sam Altman) sells the solution to the trust problems his tech will create (WorldCoin).

skybrian 2y ago
If you mean voice cloning, they aren't bringing that tech to market. (Someone else will, though.) Similarly, Google doesn't let you use face matching in image search to find every photograph of you on the web, even though they could, and quite similar technology is built into Google Photos.

iaaan 2y ago

If everyone is too cynical, then companies like OpenAI won't be able to make any money on it. You wouldn't want that, right??

throwaway48540 2y ago

We are just finally seeing the mainstream get the idea they shouldn't trust anything on the internet. Long overdue.

Panoramix 2y ago
Before these developments it was impossible to fake a politician saying arbitrary stuff. This is a major shift, there's no denying that.
- throwaway48540 2y ago
  That's the thing though, it wasn't impossible, just harder - but people believed whatever they saw, so it was done regardless of the costs. Now it's easier and people are finally getting sceptical, as they should've 20+ years ago.
- zaptrem 2y ago
  Deep faking voices has been real and super easy for years via 11labs etc. This doesn’t change that at all.
- dragonwriter 2y ago
  > Before these developments it was impossible to fake a politician saying arbitrary stuff. Voice impersonation has been possible forever, actually; and it use for misinformation (as well as less nefarious things like entertainment) is hardly novel. (Same with impersonation that goes beyond voice.)

aimazon 2y ago

Attacks on trust have been around for as long as we have had trust. Generative AI makes some attack vectors easier but it's nothing new: if your trust is earned using a voice you recognise then you model for trust has been broken since before most of us were born. https://en.wikipedia.org/wiki/Impersonator https://en.wikipedia.org/wiki/Impersonator

AmericanChopper 2y ago

The way I see it LLMs are just making it more obvious to see all the flaws with the existing levels of trust. Humans have never had access to universal truths, or universal ways of validating anything. Any claim anybody makes could be intentionally or unintentionally deceitful or untrue. The idea that there are sources you can trust to do your thinking for you is the more dangerous illusion in my opinion, and I’m not convinced that society will be harmed by poking some holes through it.

tsimionescu 2y ago
> The idea that there are sources you can trust to do your thinking for you is the more dangerous illusion in my opinion, and I’m not convinced that society will be harmed by poking some holes through it There is no alternative to this idea. It is completely impossible for an individual to possess all of the knowledge of everything that affects their lives. The only option for getting some of this information is going to trusted sources that compile it and present some conclusions. This applies just as much to scientific knowledge as it does to medicine or to politics. If you want to avoid trusting any authority, it's hard to even confirm that North Korea exists. Confirming that it is ruled by an authoritarian regime and that it possesses nuclear weapons is impossible. And yet it's a trivial bit of info that everyone agrees on - imagine what avoiding trusted authorities would do to knowledge about other more subtle or more controversial topics.
- AmericanChopper 2y ago
  > There is no alternative to this idea. I said do your thinking for you, not do your information gathering for you. I would suggest that you do not trust any single source to only ever tell you things that are true. If there’s a topic you want to know something about it’s a much better course of action to look at multiple different sources, and do your own thinking to come to your own conclusions. There are no authorities who can reliably take on this role for you, and LLMs don’t change this. The same is true with science. Even prior to LLMs, the replication crisis should have shown that a single paper on any topic can’t be relied on to contain any truth (the same would be true even in the absence of a replication crisis for that matter).

dragonwriter 2y ago

Why would the downfall of the largely unjustified and frequently harmfully exploited kinds of trust that have previously existed be bad?

squigz 2y ago

We have adapted to monumental shifts in how we develop trust in society for as long as society has existed - from the printing press to photography to the Internet to CGI to ... I don't see this as any different. We will determine new ways of establishing trust. They'll certainly have flaws, as establishing trust in a society always has, but we'll learn to recognize those flaws and hopefully fix them. Beyond that, what's the alternative? Banning the technology? That doesn't seem feasible for various reasons, not least of which is it isn't going to stop bad actors. Another pretty good reason is it's just not really possible - anyone with enough compute can build LLMs now. As a bit of an aside, why hasn't society fallen yet? I mean, ChatGPT has been around for a couple years now, and I've been hearing about how LLMs are the single greatest threat to civilized society we've ever faced... yet they don't seem to have had a major impact.

b112 2y ago
The big difference is, there was a very high bar to forging photographs, and most news was gated (with the ability to easily find, and sue those guilty of slander/libel). Now it's utterly simplistic to forge, to libel, to slander, and there is no easy path in many cases to sue. While you can say "yes, but..." to the above, that's the reality that we've lived with for 150 years, less extremely rare edge cases. All this has changed over the course of a couple of decades, with most of that change in the last 10, and focuses on the last 2 years. Beyond that, it took significant effort and labour to create fake stories and images. People had to be experts, or be wordsmiths. Now, click click, and fake generated stories abound. In fact, they're literally everywhere. There's absolutely no comparison here. Now, in the time it used to take one person to generate one fake story, you can generate millions and trillions if you have the cash. Really, it's the same problem with spam phone calls, and with spam email. You didn't get 1000 spam letters in the mail in the 80s, because that cost money. Email was free, thus spam became plentiful. The same with spam phone calls, it cost hard cash for each call, now it's pennies per hundreds of automated calls, so spam phone calls abound. The same is happening with all content on the internet. Realistically the web is now dead. It's now gone. Even things such as wikipedia are going to die, as over the next 2 to 3 years LLM output will become utterly and completely indistinguishable in all aspects.

pizzafeelsright 2y ago

I think you're seeing the lack of trust exposed. Contracts allow for recompense if there is mistrust. We've been signing contracts forever. Every automated system needs someone to jail in the event of failure.

cdrini 2y ago

I think the other comments make a good argument about how other forms of technology have also degraded trust, but that we've found a way through. I'll also add that I think one potential way we could reinstate trust is through signed multimedia. Cameras/microphones/etc could sign the videos/audio they create in a way that can be used to verify that the media hasn't been doctored. Not sure if that's actually a feasible approach, but it's one possibility.

mike_hearn 2y ago
It's feasible with advanced enough tech. The hard part isn't getting cameras to sign the files they produce. The hard part is to preserve the chain of custody as images are cropped, rescaled, recompressed etc. You can do it with tech like Intel SGX. But you also need serious defense of the camera platforms against hacking, of the CPUs, of the software stacks. And there's no demand. News orgs feel they should be implicitly trusted due to their brands, so why would they use complicated tech to build trust?
- cdrini 2y ago
  They might use complicated tech because things are changing due to AI in a way that could degrade their brands trust. By having camera-signed videos, when folks create eg deep fakes of their news anchors/brands, there's a way for consumers to verify what's real. It lets their brands become more trustworthy. Yeah, preserving the chain of custody is hard. I was thinking there are a few options: (1) the signature of the original video could be attached even after editing/compression, and then a news org would let you look up the signature. That way if someone copied the signature and stuck it on a fake video, you could see the original video that actually passes with that signature, and determine if something has been doctored. Or (2), you could have editing software add a signature verifying the edits made: eg compression, rescaling etc. And then a law to make it illegal to remove/tamper/valsify a signature, like we have for DVDs, to allow some form of prosecution. The hardware stack is a little easier to protect; the software stack less so. But if we can do it with things like eg browser DRM or http signatures, maybe we can with media editing software? But I'm not versed enough in Cryptography to really know. And cool will read up on intel SGX, thanks!