6 ms·

The biggest risk of large language models is that they may bury us in lies

44 points by garymarcus 4y ago

garymarcus 4y ago
How models like GPT-3 could accelerate the production of misinformation, and what we might do about it
maria2 4y ago
We’ve been in a post-truth society for some time now. Anything that makes it easier to lie is a positive in my book. Society needs to accept that we’re well past the time where information can be trusted. Either we adapt or die. One thing that helps me is to understand that I really don’t need to know the truth. I wake up. I see my family. I watch my daughter smile. I enjoy the company of friends. I drink a nice glass of wine. What value does the “truth” have in my daily life? Most of the time, what is true and what is lie only comes up in a political discussion, and it almost always is unpleasant.
- mint2 4y ago
  One only has to look at the Russian invasion based on “de-nazifying” Ukraine or global warming denialism or big tobacco spending decades obfuscating cancer or or or four thousand other post truth things to see that good people ignoring and being okay with lies leads to bad things.
  - maria2 4y ago
    You’re fighting a pointless war.
    - mint2 4y ago
      It’s better than Surrendering to lies
      - maria2 4y ago
        That’s the thing. You’re not surrendering to lies. You’re just choosing not to play.
    - a_t48 4y ago
      So, what? Give up, grin as the world burns?
      - nisegami 4y ago
        No, we learn how to adapt to a world where truth is not relevant.
- tomatotime 4y ago
  If someone tells you the wine you had was poisoned, is it not important if that is a lie or the truth? I imagine it would.
  - maria2 4y ago
    Would it? If you tell me you poisoned my wine I wouldn’t give it a single thought. Or is there more to this analogy that you’ve neglected to mention?
- pjc50 4y ago
  > I wake up. I see my family. I watch my daughter smile. I enjoy the company of friends. I drink a nice glass of wine. What value does the “truth” have in my daily life? When people talk about privilege, this kind of bucolic normalness is what they mean; all of those things are great, but their ability to stay great is conditional on a huge, precise system that depends on trust (which in turn depends on truth) to run. You can go to the store and buy wine without ever once having to test it for antifreeze.
  - spritefs 4y ago
    > You can go to the store and buy wine without ever once having to test it for antifreeze. This doesn't say anything about truth, only past experience. Which isn't truth > all of those things are great, but their ability to stay great is conditional on a huge, precise system that depends on trust (which in turn depends on truth) to run. It depends on trust, yes. But trust doesn't depend on truth (because someone doesn't need metaphysical or religious beliefs for them to be trustworthy)
  - maria2 4y ago
    > all of those things are great, but their ability to stay great is conditional on a huge, precise system that depends on trust (which in turn depends on truth) to run. I 100% agree that the system is built on trust and that the system is where I derive my enjoyment of "bucolic normalness." However, trust in the system has absolutely nothing to do with truth. All systems are driven by those with power. The opinion of the masses, whether informed by the truth or otherwise, doesn't matter. For instance, another person mentioned the lie of "de-nazifying Ukraine." Russia has also plainly expressed its imperialist ambition and the ability to reduce NATO's influence in Eastern Europe. Both of those things are true. Does that mean Russia is justified in invading Ukraine because they told the truth about their intentions? The population in Ukraine is strongly against Russian influence. But if those in power wanted Russia's influence, the population would be powerless to stop the transition. To see evidence of this, look no further than Belarus. In Belarus' 2020 election, something like 90% of the population voted against the president. 5% - 10% of the population actively protested in the streets. Workers went on strike. And yet, the elites in Belarus aren't interested in regime change so nothing happened. Take gerrymandering as another example. The goal of gerrymandering is very plainly to disenfranchise certain voters. Everything knows that. It is the truth. And yet gerrymandering still happens. Another example: do you really think Trump, the senators, and representatives responsible for the Jan. 6th insurrection really believed that the election was stolen? Of course not. They wanted power and "stop the steal" was just a pretext. I'm sure many of the "stop the steal" supporters understand this too. The lie is just a pretext to get what they want. Again, the truth doesn't matter. You're not going to convince Jan 6 supporters of "the truth" and have them change their mind, because they're motivated by a selfish grab for power. They could care less about the truth. Ultimately, trust in the system comes from the shared belief that our representative form of government is the best for everyone. Sometimes, that means needing to sacrifice a personal objective, like a quest for power, for the good of the entire system. If the elites want a different form of government, like one where they consolidate power for themselves, then that is what will happen. That is what's happening. The truth isn't going to stop it.
schiffern 4y ago
> they may bury us in [the wrong people's] lies Fixed the headline.
simple-thoughts 4y ago
Truth itself is an imaginary construct. All animals are evolved to survive not to seek truth. “Truth seekers” are performing in a status game with the objective to prove genetic strength to prospective mates (however unsuccessful that quest might be). Even if we could know the truth, it would be a waste of valuable resources that could be better spent elsewhere.
- thewarrior 4y ago
  And we know the truth of this how ?
  - simple-thoughts 4y ago
    It’s not Truth because it’s not from a position of authority - I am just a random commenter. You’re asking the wrong question. The right question would be, “Would agreeing with this make my life better or worse?”
- wwweston 4y ago
  How do you know what a non-wasteful allocation of those valuable resources is without truth? An accurate understanding of reality is certainly not the only factor in achieving fitness & status, and I'd even believe at a certain point increased resolution when it comes to understanding reality has diminishing returns (and am aware of some research that supports this). But absent understanding of reality, there is no effective sustainable fitness or status (or effective allocation of resources). I don't know what you'd call that understanding but truth works for me.
  - simple-thoughts 4y ago
    You have to trust your own body and mind to be healthy enough to keep you going - and if it can’t trust others to support you. None of that requires Truth. But definitely to avoid jail time you better understand what Truth is in your place of residence. And that is to realize it’s bullshit - if you buy into the lies you’re going to be a mark.
Biologist123 4y ago
> Systems like ChatGPT are enormously entertaining and even mind-bogglingly human-sounding, but they are also unreliable and could create an avalanche of misinformation. It sounds like it could be used as a “flood the zone” machine in some ways, and led me to wonder whether this capacity was a feature or a bug.
vineyardmike 4y ago
I think this will rapidly accelerate a set of closely related trends that were rapidly growing in the internet: curation, trust, and reputation. In a world with the massive aggregating ability of google, amazon, Facebook, etc, we've become deprived already of truth. Whether its the alto-junk filling up search results, fake reviews on cheap knockoffs on amazon, or the latest mis-information campaign on Facebook, it becomes hard to sort through the endless options (curation), trust the reviews, and find reliable sources. People already search on reddit for some review, because they don't trust Amazon reviews, but what happens when reddit becomes GPT playgrounds? You'll see a further rise in organizations that either sell Curation (Target, Costco), reputation (Apple App Store) or sell Trust (Consumer Reports, Wirecutter). People flock to influencer YouTubers for product reviews, because its harder to fake, and people will build new business around such solutions.
- spritefs 4y ago
  > I think this will rapidly accelerate a set of closely related trends that were rapidly growing in the internet: curation, trust, and reputation. But will it scale?
  - vineyardmike 4y ago
    Who knows but that might not be the most important factor. > One of the most common types of advice we give at Y Combinator is to do things that don't scale I’m not going to explain someone else’s quote, but in my perspective, in a world of limitless abundance, perhaps trying to do everything isn’t the winning strategy. If Amazon has every product ever made available (and 75% of it is cheap drop shipped junk), there’s extra value in being the trustworthy brand that doesn’t have everything, but instead the right things. If Twitter is full of bots and misinformation, then there’s value in being a small mastadon instance of hand curated friends where you can trust everyone. If Google can index every website, and most of it ends up as GPT lies, then being a librarian that can vet sounded for you is immensely valuable. http://paulgraham.com/ds.html http://paulgraham.com/ds.html
    - spritefs 4y ago
mharig 4y ago
A cheap way to replace politicians.
janandonly 4y ago
3500 years ago, the Assyrians and Babylonians made it policy to always lie (propagandize) about their victories. They had no official losses, they only won closer and closer to home... There is nothing new under the sun.
spritefs 4y ago
Something to keep in mind with this article (and the myriad of other articles like it): the author has a psych background, not an engineering background. He's worked as a consultant, not an engineer. He has a lot of things to say about AI He calls for changes in AI, but never does he talk about concrete implementation (in this article or elsewhere). People like the author of this article, let's call them "thought leaders", what are they actually good for? What do they build? It seems like all they can do is make noise and spend money
naveen99 4y ago
Lies are just a special case of an uncountable infinity of impossibilities. It will only make humans who lie have an existential crisis. People who believe in a consistent world can carry on normally.

44 points by garymarcus 4y ago

How models like GPT-3 could accelerate the production of misinformation, and what we might do about it

We’ve been in a post-truth society for some time now. Anything that makes it easier to lie is a positive in my book. Society needs to accept that we’re well past the time where information can be trusted. Either we adapt or die. One thing that helps me is to understand that I really don’t need to know the truth. I wake up. I see my family. I watch my daughter smile. I enjoy the company of friends. I drink a nice glass of wine. What value does the “truth” have in my daily life? Most of the time, what is true and what is lie only comes up in a political discussion, and it almost always is unpleasant.

mint2 4y ago
One only has to look at the Russian invasion based on “de-nazifying” Ukraine or global warming denialism or big tobacco spending decades obfuscating cancer or or or four thousand other post truth things to see that good people ignoring and being okay with lies leads to bad things.
- maria2 4y ago
  You’re fighting a pointless war.
  - mint2 4y ago
    It’s better than Surrendering to lies
    - maria2 4y ago
      That’s the thing. You’re not surrendering to lies. You’re just choosing not to play.
  - a_t48 4y ago
    So, what? Give up, grin as the world burns?
    - nisegami 4y ago
      No, we learn how to adapt to a world where truth is not relevant.
tomatotime 4y ago
If someone tells you the wine you had was poisoned, is it not important if that is a lie or the truth? I imagine it would.
- maria2 4y ago
  Would it? If you tell me you poisoned my wine I wouldn’t give it a single thought. Or is there more to this analogy that you’ve neglected to mention?
pjc50 4y ago
> I wake up. I see my family. I watch my daughter smile. I enjoy the company of friends. I drink a nice glass of wine. What value does the “truth” have in my daily life? When people talk about privilege, this kind of bucolic normalness is what they mean; all of those things are great, but their ability to stay great is conditional on a huge, precise system that depends on trust (which in turn depends on truth) to run. You can go to the store and buy wine without ever once having to test it for antifreeze.
- spritefs 4y ago
  > You can go to the store and buy wine without ever once having to test it for antifreeze. This doesn't say anything about truth, only past experience. Which isn't truth > all of those things are great, but their ability to stay great is conditional on a huge, precise system that depends on trust (which in turn depends on truth) to run. It depends on trust, yes. But trust doesn't depend on truth (because someone doesn't need metaphysical or religious beliefs for them to be trustworthy)
- maria2 4y ago
  > all of those things are great, but their ability to stay great is conditional on a huge, precise system that depends on trust (which in turn depends on truth) to run. I 100% agree that the system is built on trust and that the system is where I derive my enjoyment of "bucolic normalness." However, trust in the system has absolutely nothing to do with truth. All systems are driven by those with power. The opinion of the masses, whether informed by the truth or otherwise, doesn't matter. For instance, another person mentioned the lie of "de-nazifying Ukraine." Russia has also plainly expressed its imperialist ambition and the ability to reduce NATO's influence in Eastern Europe. Both of those things are true. Does that mean Russia is justified in invading Ukraine because they told the truth about their intentions? The population in Ukraine is strongly against Russian influence. But if those in power wanted Russia's influence, the population would be powerless to stop the transition. To see evidence of this, look no further than Belarus. In Belarus' 2020 election, something like 90% of the population voted against the president. 5% - 10% of the population actively protested in the streets. Workers went on strike. And yet, the elites in Belarus aren't interested in regime change so nothing happened. Take gerrymandering as another example. The goal of gerrymandering is very plainly to disenfranchise certain voters. Everything knows that. It is the truth. And yet gerrymandering still happens. Another example: do you really think Trump, the senators, and representatives responsible for the Jan. 6th insurrection really believed that the election was stolen? Of course not. They wanted power and "stop the steal" was just a pretext. I'm sure many of the "stop the steal" supporters understand this too. The lie is just a pretext to get what they want. Again, the truth doesn't matter. You're not going to convince Jan 6 supporters of "the truth" and have them change their mind, because they're motivated by a selfish grab for power. They could care less about the truth. Ultimately, trust in the system comes from the shared belief that our representative form of government is the best for everyone. Sometimes, that means needing to sacrifice a personal objective, like a quest for power, for the good of the entire system. If the elites want a different form of government, like one where they consolidate power for themselves, then that is what will happen. That is what's happening. The truth isn't going to stop it.

schiffern 4y ago

> they may bury us in [the wrong people's] lies Fixed the headline.

simple-thoughts 4y ago

Truth itself is an imaginary construct. All animals are evolved to survive not to seek truth. “Truth seekers” are performing in a status game with the objective to prove genetic strength to prospective mates (however unsuccessful that quest might be). Even if we could know the truth, it would be a waste of valuable resources that could be better spent elsewhere.

thewarrior 4y ago
And we know the truth of this how ?
- simple-thoughts 4y ago
  It’s not Truth because it’s not from a position of authority - I am just a random commenter. You’re asking the wrong question. The right question would be, “Would agreeing with this make my life better or worse?”
wwweston 4y ago
How do you know what a non-wasteful allocation of those valuable resources is without truth? An accurate understanding of reality is certainly not the only factor in achieving fitness & status, and I'd even believe at a certain point increased resolution when it comes to understanding reality has diminishing returns (and am aware of some research that supports this). But absent understanding of reality, there is no effective sustainable fitness or status (or effective allocation of resources). I don't know what you'd call that understanding but truth works for me.
- simple-thoughts 4y ago
  You have to trust your own body and mind to be healthy enough to keep you going - and if it can’t trust others to support you. None of that requires Truth. But definitely to avoid jail time you better understand what Truth is in your place of residence. And that is to realize it’s bullshit - if you buy into the lies you’re going to be a mark.

Biologist123 4y ago

> Systems like ChatGPT are enormously entertaining and even mind-bogglingly human-sounding, but they are also unreliable and could create an avalanche of misinformation. It sounds like it could be used as a “flood the zone” machine in some ways, and led me to wonder whether this capacity was a feature or a bug.

vineyardmike 4y ago

I think this will rapidly accelerate a set of closely related trends that were rapidly growing in the internet: curation, trust, and reputation. In a world with the massive aggregating ability of google, amazon, Facebook, etc, we've become deprived already of truth. Whether its the alto-junk filling up search results, fake reviews on cheap knockoffs on amazon, or the latest mis-information campaign on Facebook, it becomes hard to sort through the endless options (curation), trust the reviews, and find reliable sources. People already search on reddit for some review, because they don't trust Amazon reviews, but what happens when reddit becomes GPT playgrounds? You'll see a further rise in organizations that either sell Curation (Target, Costco), reputation (Apple App Store) or sell Trust (Consumer Reports, Wirecutter). People flock to influencer YouTubers for product reviews, because its harder to fake, and people will build new business around such solutions.

spritefs 4y ago
> I think this will rapidly accelerate a set of closely related trends that were rapidly growing in the internet: curation, trust, and reputation. But will it scale?
- vineyardmike 4y ago
  Who knows but that might not be the most important factor. > One of the most common types of advice we give at Y Combinator is to do things that don't scale I’m not going to explain someone else’s quote, but in my perspective, in a world of limitless abundance, perhaps trying to do everything isn’t the winning strategy. If Amazon has every product ever made available (and 75% of it is cheap drop shipped junk), there’s extra value in being the trustworthy brand that doesn’t have everything, but instead the right things. If Twitter is full of bots and misinformation, then there’s value in being a small mastadon instance of hand curated friends where you can trust everyone. If Google can index every website, and most of it ends up as GPT lies, then being a librarian that can vet sounded for you is immensely valuable. http://paulgraham.com/ds.html http://paulgraham.com/ds.html
  - spritefs 4y ago

mharig 4y ago

A cheap way to replace politicians.

janandonly 4y ago

3500 years ago, the Assyrians and Babylonians made it policy to always lie (propagandize) about their victories. They had no official losses, they only won closer and closer to home... There is nothing new under the sun.

spritefs 4y ago

Something to keep in mind with this article (and the myriad of other articles like it): the author has a psych background, not an engineering background. He's worked as a consultant, not an engineer. He has a lot of things to say about AI He calls for changes in AI, but never does he talk about concrete implementation (in this article or elsewhere). People like the author of this article, let's call them "thought leaders", what are they actually good for? What do they build? It seems like all they can do is make noise and spend money

naveen99 4y ago

Lies are just a special case of an uncountable infinity of impossibilities. It will only make humans who lie have an existential crisis. People who believe in a consistent world can carry on normally.