6 ms·

Misinformation Has Created a New World Disorder

47 points by kaisix 7y ago

hos234 7y ago
This is a good list of different type of issues being produced. I think there is a need of tracking it the same way we track software bugs. I thought Tristan Harris's Ledger of Harm's was a good attempt. Looks like this author is doing a similar thing. Should all be merged into some kind master list that software architects can stay aware of. The idea of a "9/11" Test, getting software devs to think about how their tools and features will be used during a 9/11 type event is a good suggestion. I had a boss once who would ask the team, 'how would the most shadiest character you personally know, use this proposed feature?'.
- Theizestooke 7y ago
  > I had a boss once who would ask the team, 'how would the most shadiest character you personally know, use this proposed feature?'. I mean society is based on trust and faith, with that kind of reasoning you'll only come up with the most paranoical and authoritarian design schemes.
  - pfisch 7y ago
    Human's always game every system we ever develop. It is reasonable to recognize this when you design systems. I have a f2p game and it is unbelievable how clever some of the free players are at figuring out ways to get everything for free.
    - deogeo 7y ago
      > unbelievable how clever some of the free players are You have my curiosity.
  - pjc50 7y ago
    Non-paranoid software design gets hacked. Sometimes horrifyingly quickly.
  - carapace 7y ago
    Society works by trust and faith, it's based on vicious talking apes. In this case, you should be paranoid because we know darn well there are shady people out there trying to get you. How many times a day does your phone ring and it's some scammer?
- radicalbyte 7y ago
  "What would Hitler do?" is the version of the test that I use.
growlist 7y ago
Not a single example of leftist/liberal misinformation in the piece. Either the left are as pure and innocent as a newborn kitten, or this article is itself misinformation.
- kuzehanka 7y ago
  I'll tell you something interesting. Yesterday a replicated version of GPT2 was published to the wild[1]. I've been playing with the model quite a bit since then and found something unexpected. If you give it a right-wing US politics prompt, it performs so well that most of the output could pass for a coherent human without any editing. An example prompt would be > The only way to save America is to vote for Donald trump. The democrats have failed us If you give it an inverse left-wing variant of the same prompt, it mostly returns incoherent output and sometimes actually flips back to the right-wing narrative. An example would be > The only way to save America is to impeach Trump. The republicans have failed us How well this model performs depends on how much training data it had access to. And this model was mostly trained on Reddit comments. So even this early on, OpenGPT is clearly highlighting biases in Reddit comments. Reddit is traditionally known as the bastion of the left, so the fact that OpenGPT is much more effective at generating right-wing propaganda is indicative of something. I'll leave it as an exercise to the reader to guess what. [1] https://medium.com/@vanya_cohen/opengpt-2-we-replicated-gpt-2-because-you-can-too-45e34e6d36dc https://medium.com/@vanya_cohen/opengpt-2-we-replicated-gpt-...
  - deogeo 7y ago
    It could simply be a consequence of the poor inversion - I don't recall seeing many left-wing news that start with "How to save America". You're effectively giving it conflicting input.
    - kuzehanka 7y ago
      You're welcome to propose a better inversion and I'll check the output and report back. It does need to be an inversion though, not a complete change of prompt. It's a general theme with this model though. When you try to get it to do left-wing propaganda, it has a tendency to flip back to right-wing because of the bias in the training data.
mirimir 7y ago
Back in the early 90s, I was very optimistic about the potential for Internet-mediated "citizen journalism" to counter misinformation delivered by the mass media. I was such a bloody idealist. The distinction between democratic populism and mob violence is iffy in practice.
Mikeb85 7y ago
It's not as though the mainstream press are blameless. Maybe they wouldn't have lost their position if they weren't blatantly peddling propaganda. In places like the UK and Canada, the top news agencies are literally government owned. CNN is hilariously biased. Western news outlets have been caught time and time again misleading people. Alternative news sources are far from perfect as well, but in an age where everyone is trying to mislead, it forces people who want to be informed to actually take in events from multiple outlets and try to discern where truth lies for themselves. Maybe it's my Slavic upbringing, but I was taught to be sceptical of everyone and everything. Also, information can't be weaponized cheaply. If one side has access to it, so does the other. They counteract each other. In the end, people still have to discern news for themselves. Finally, it's quite hilarious to hear news outlets trying to convince us that today we're misinformed, and if only we do a few things to 'fix' the system, we can get to a point where they're our only source of news again.
- mieseratte 7y ago
  As a child, it was always “Fox is just riling up the old people.” I’m not sure whether CNN was once reasonable, but it seems most, of not all, mainstream press has devolved into that Foxian shit-show for one side of the aisle or another trying to capture eye-balls and add dollars. My inner conspiracy-theorist wonders if this is divide-and-conquer, or just the death throes of dying media.
  - scruple 7y ago
    I noticed it with CNN back around 2008. I don't remember the specifics these days, but I do remember Glenn Greenwald really making a lot of salient points about the direction that CNN had taken. My inner conspiracy theorist has also spent a lot of time wondering if it's just divide-and-conquer in action.
- pjc50 7y ago
  The BBC used to be respectable, but they lost their nerve after the Hutton enquiry and have also fallen to the "clickbait" problem of magnifying extreme views. But they're nowhere near as bad as the rightwing press: Times, Mail, Telegraph, Sun, Express. The Telegraph is almost the reverse of state-owned media. Since they pay Boris Johnson more than the office of Prime Minister does, in some ways we're a media-owned state.
chris5745 7y ago
“Whenever the people are well informed, they can be trusted with their own government; that whenever things get so far wrong as to attract their notice, they may be relied on to set them to rights.” -Thomas Jefferson Everyone seems to talk a lot about governments, but at some point we may need to explore whether being informed about non-government entities may be just as important in this context. I would argue that being informed about government is important for preserving a democracy, and being informed about industry is important for preserving a free market. Information inequality threatens the integrity of any system that relies on a group to make decisions, whether it’s a town hall meeting or a shareholder event. By the vote or by the dollar. It creates inefficiencies that are exploited by the information-wealthy. As regrettable as it is to think this way, I am not sure there is a technological solution to the problem of misinformation. In the very near future we may only want to connect to the internet for entertainment, irrespective of truth. Critical thinking will still be alive, but anyone who engages in it truthfully will probably limit their thoughts to people, things, and events in their immediate surroundings. Later, when we’re honest with ourselves, we’ll have another chance to be honest with each other.
carapace 7y ago
Fidelity to reality. I think that Wendell Berry's "Standing by Words" is relevant > In his keynote essay, Berry argues that “we have seen, for perhaps a hundred and fifty years, a gradual increase in language that is either meaningless or destructive of meaning.” Furthermore, he continues, “this increasing unreliability of language parallels the increasing disintegration, over the same period, of persons and communities.” ~https://www.enotes.com/topics/standing-by-words https://www.enotes.com/topics/standing-by-words Mom talks about how TVs weren't even a thing when she was a girl. She remembers seeing the first one in the store. Back then they only broadcast for a few hours in the evenings. Now you can't ride a bus without someone watching TV on their phone. Nothing on those screens is real. Not even HD surround sound Nature shows are actually real. We are living in a hyper-mediated world. Align yourself with reality, it's nice out here.
- xtiansimon 7y ago
  Hmm. In the world you envision I see lower employment because there’s just less work. Hard to infinity divide and recombine labor in the ‘real world’ of butchers, bakers, construction, grocery stores, mechanics, retail, bus drivers. Whereas movies, television, streaming, toys, video game tie-ins need a lot of busy people. I see much more work in software and hardware and security and seo and e-commerce and recommendation engines and social media. Not to mention JavaScript, Go, Rust, Python, agile-long live agile; down with agile. Haha. You had me going for a minute.

47 points by kaisix 7y ago

This is a good list of different type of issues being produced. I think there is a need of tracking it the same way we track software bugs. I thought Tristan Harris's Ledger of Harm's was a good attempt. Looks like this author is doing a similar thing. Should all be merged into some kind master list that software architects can stay aware of. The idea of a "9/11" Test, getting software devs to think about how their tools and features will be used during a 9/11 type event is a good suggestion. I had a boss once who would ask the team, 'how would the most shadiest character you personally know, use this proposed feature?'.

Theizestooke 7y ago
> I had a boss once who would ask the team, 'how would the most shadiest character you personally know, use this proposed feature?'. I mean society is based on trust and faith, with that kind of reasoning you'll only come up with the most paranoical and authoritarian design schemes.
- pfisch 7y ago
  Human's always game every system we ever develop. It is reasonable to recognize this when you design systems. I have a f2p game and it is unbelievable how clever some of the free players are at figuring out ways to get everything for free.
  - deogeo 7y ago
    > unbelievable how clever some of the free players are You have my curiosity.
- pjc50 7y ago
  Non-paranoid software design gets hacked. Sometimes horrifyingly quickly.
- carapace 7y ago
  Society works by trust and faith, it's based on vicious talking apes. In this case, you should be paranoid because we know darn well there are shady people out there trying to get you. How many times a day does your phone ring and it's some scammer?
radicalbyte 7y ago
"What would Hitler do?" is the version of the test that I use.

growlist 7y ago

Not a single example of leftist/liberal misinformation in the piece. Either the left are as pure and innocent as a newborn kitten, or this article is itself misinformation.

kuzehanka 7y ago
I'll tell you something interesting. Yesterday a replicated version of GPT2 was published to the wild[1]. I've been playing with the model quite a bit since then and found something unexpected. If you give it a right-wing US politics prompt, it performs so well that most of the output could pass for a coherent human without any editing. An example prompt would be > The only way to save America is to vote for Donald trump. The democrats have failed us If you give it an inverse left-wing variant of the same prompt, it mostly returns incoherent output and sometimes actually flips back to the right-wing narrative. An example would be > The only way to save America is to impeach Trump. The republicans have failed us How well this model performs depends on how much training data it had access to. And this model was mostly trained on Reddit comments. So even this early on, OpenGPT is clearly highlighting biases in Reddit comments. Reddit is traditionally known as the bastion of the left, so the fact that OpenGPT is much more effective at generating right-wing propaganda is indicative of something. I'll leave it as an exercise to the reader to guess what. [1] https://medium.com/@vanya_cohen/opengpt-2-we-replicated-gpt-2-because-you-can-too-45e34e6d36dc https://medium.com/@vanya_cohen/opengpt-2-we-replicated-gpt-...
- deogeo 7y ago
  It could simply be a consequence of the poor inversion - I don't recall seeing many left-wing news that start with "How to save America". You're effectively giving it conflicting input.
  - kuzehanka 7y ago
    You're welcome to propose a better inversion and I'll check the output and report back. It does need to be an inversion though, not a complete change of prompt. It's a general theme with this model though. When you try to get it to do left-wing propaganda, it has a tendency to flip back to right-wing because of the bias in the training data.

mirimir 7y ago

Back in the early 90s, I was very optimistic about the potential for Internet-mediated "citizen journalism" to counter misinformation delivered by the mass media. I was such a bloody idealist. The distinction between democratic populism and mob violence is iffy in practice.

Mikeb85 7y ago

It's not as though the mainstream press are blameless. Maybe they wouldn't have lost their position if they weren't blatantly peddling propaganda. In places like the UK and Canada, the top news agencies are literally government owned. CNN is hilariously biased. Western news outlets have been caught time and time again misleading people. Alternative news sources are far from perfect as well, but in an age where everyone is trying to mislead, it forces people who want to be informed to actually take in events from multiple outlets and try to discern where truth lies for themselves. Maybe it's my Slavic upbringing, but I was taught to be sceptical of everyone and everything. Also, information can't be weaponized cheaply. If one side has access to it, so does the other. They counteract each other. In the end, people still have to discern news for themselves. Finally, it's quite hilarious to hear news outlets trying to convince us that today we're misinformed, and if only we do a few things to 'fix' the system, we can get to a point where they're our only source of news again.

mieseratte 7y ago
As a child, it was always “Fox is just riling up the old people.” I’m not sure whether CNN was once reasonable, but it seems most, of not all, mainstream press has devolved into that Foxian shit-show for one side of the aisle or another trying to capture eye-balls and add dollars. My inner conspiracy-theorist wonders if this is divide-and-conquer, or just the death throes of dying media.
- scruple 7y ago
  I noticed it with CNN back around 2008. I don't remember the specifics these days, but I do remember Glenn Greenwald really making a lot of salient points about the direction that CNN had taken. My inner conspiracy theorist has also spent a lot of time wondering if it's just divide-and-conquer in action.
pjc50 7y ago
The BBC used to be respectable, but they lost their nerve after the Hutton enquiry and have also fallen to the "clickbait" problem of magnifying extreme views. But they're nowhere near as bad as the rightwing press: Times, Mail, Telegraph, Sun, Express. The Telegraph is almost the reverse of state-owned media. Since they pay Boris Johnson more than the office of Prime Minister does, in some ways we're a media-owned state.

chris5745 7y ago

“Whenever the people are well informed, they can be trusted with their own government; that whenever things get so far wrong as to attract their notice, they may be relied on to set them to rights.” -Thomas Jefferson Everyone seems to talk a lot about governments, but at some point we may need to explore whether being informed about non-government entities may be just as important in this context. I would argue that being informed about government is important for preserving a democracy, and being informed about industry is important for preserving a free market. Information inequality threatens the integrity of any system that relies on a group to make decisions, whether it’s a town hall meeting or a shareholder event. By the vote or by the dollar. It creates inefficiencies that are exploited by the information-wealthy. As regrettable as it is to think this way, I am not sure there is a technological solution to the problem of misinformation. In the very near future we may only want to connect to the internet for entertainment, irrespective of truth. Critical thinking will still be alive, but anyone who engages in it truthfully will probably limit their thoughts to people, things, and events in their immediate surroundings. Later, when we’re honest with ourselves, we’ll have another chance to be honest with each other.

carapace 7y ago

Fidelity to reality. I think that Wendell Berry's "Standing by Words" is relevant > In his keynote essay, Berry argues that “we have seen, for perhaps a hundred and fifty years, a gradual increase in language that is either meaningless or destructive of meaning.” Furthermore, he continues, “this increasing unreliability of language parallels the increasing disintegration, over the same period, of persons and communities.” ~https://www.enotes.com/topics/standing-by-words https://www.enotes.com/topics/standing-by-words Mom talks about how TVs weren't even a thing when she was a girl. She remembers seeing the first one in the store. Back then they only broadcast for a few hours in the evenings. Now you can't ride a bus without someone watching TV on their phone. Nothing on those screens is real. Not even HD surround sound Nature shows are actually real. We are living in a hyper-mediated world. Align yourself with reality, it's nice out here.

xtiansimon 7y ago
Hmm. In the world you envision I see lower employment because there’s just less work. Hard to infinity divide and recombine labor in the ‘real world’ of butchers, bakers, construction, grocery stores, mechanics, retail, bus drivers. Whereas movies, television, streaming, toys, video game tie-ins need a lot of busy people. I see much more work in software and hardware and security and seo and e-commerce and recommendation engines and social media. Not to mention JavaScript, Go, Rust, Python, agile-long live agile; down with agile. Haha. You had me going for a minute.