23 ms·

Common-Knowledge Attacks on Democracy

207 points by yungchin 8y ago

jedharris 8y ago
Great model! Simple enough to understand, fits a lot of recent phenomena. Scary implications, but we should be able to respond effectively. Unexpected but good to see Schneier doing this kind of work.
nbp234 8y ago
Link to discussion on Bruce Schneier's blog: https://www.schneier.com/blog/archives/2018/11/propaganda_and_.html https://www.schneier.com/blog/archives/2018/11/propaganda_an...
- dfabulich 8y ago
  The comments on that blog post are examples of the problem. They're diving straight into disagreements about matters of fact (like whether a party cheated in a particular election, and whether both sides cheat). The point is that we're not (no longer) disagreeing about values, but about the facts, and that authoritarians are deliberately attempting to muddy the waters here. They're clearly succeeding, and the commenters seem to be distracted from that.
  - commandlinefan 8y ago
    > examples of the problem on the other hand, they both seem to come to the same conclusion: that we need our election machinery to be verifiably tamper-proof.
darawk 8y ago
Ya, i've been saying something like this for a while. China, for instance, is uniquely invulnerable to this sort of attack. Because the state is already in charge of censoring information, it is much more difficult to flood them with misinformation. It's one of the many reasons i'm concerned that models like Chinas will actually flourish more than they otherwise would in the coming years.
- forapurpose 8y ago
  They may be less vulnerable to attacks from outsiders, but they are much more vulnerable to attacks from their own government. It's similar to totalitarian governments; they might keep organized crime low, but they expose people to far more crime from the government itself. I would expect people in China and in the Chinese government to also be more vulnerable to misleading and suboptimal information, because there is a lack of competing ideas.
  - hammock 8y ago
    >They may be less vulnerable to attacks from outsiders, but they are much more vulnerable to attacks from their own government. It's similar to totalitarian governments Interesting that you say that about China, because that was exactly Bruce Schneier's conclusion about the United States: >Our research implies that insider attacks from within American politics can be more pernicious than attacks from other countries. https://www.schneier.com/blog/archives/2018/11/propaganda_and_.html https://www.schneier.com/blog/archives/2018/11/propaganda_an...
    - commandlinefan 8y ago
      > Without any evidence whatsoever, he said that Democrats were trying to steal the election through "FRAUD." Well, wait a minute - both of those races had already been called in favor of the Republican candidate on the night of the election until boxes and boxes of mysteriously Democrat-leaning ballots were suddenly found the following day... I think that constitutes evidence, even if it's not conclusive.
      - orf 8y ago
        Evidence of a poorly managed election system split into tens of thousands of individual parts across 50 states in dire need of reform (but will never be, because one party benefits from the status quo), or evidence of a Democratic conspiracy to fraudulently seize power by creating 'mysterious' ballot boxes in the most hamfisted and public way possible? Sure.
baby 8y ago
It'd be interesting to see a formal threat model of a Democracy like we write such documents for our applications and protocols.
- Balgair 8y ago
  If you have a lot of know-how about these threat models, I'd highly encourage you to write one for democracy as an idea. I'd love to read it!
  - baby 8y ago
    https://github.com/mimoo/democracy_threat_model https://github.com/mimoo/democracy_threat_model
peterwwillis 8y ago
Democracy is doomed, but not because of advanced attacks. Autocracy/nationalism/monarchy/etc are just the natural order of human societies. We've had a good run, but so did the Greeks and Romans.
- deleted 8y ago
  [deleted]
- Balgair 8y ago
  To be fair, Italy and Greece are republics (for the umpteenth time) as of today.
- deogeo 8y ago
  Isn't nationalism orthogonal to democracy? How are they opposed?
  - peterwwillis 8y ago
    They're not opposed. Nationalism just has the tendency of galvanizing a population's support, which can easily bolster an autocracy that represents the "national identity". It's sort of the supporting character for autocracy and related political systems. (I'm just basing this on opinion, not political theory)
  - wahern 8y ago
    It's sort of like gimbal lock: when a particular social movement becomes so pervasive and powerful the mechanisms of government become dangerously aligned and you lose the ability of the political system to perform an important function, namely defend itself and minorities from the tyranny of the majority. Remember, part of the design of separation of powers is that institutions must retain the ability and motivation to compete for power. If the motivation is gone the ability becomes at risk--either by atrophy or by one institution making power grabs that go unchallenged by others institutions or by the people. Discord is, to an extent, a feature, not a bug. Hugo Chavez was a nationalist that was popularly and fairly elected multiple times until, eventually, the elections weren't actually fair anymore and the institutions of government were irreparably corrupt. Nationalist movements aren't the only the way this can naturally occur, but it's perhaps the most common in the past two centuries.
tareqak 8y ago
I don't know why nbp234's comment with the link to Bruce Schneier's blog is dead, but I think it is pretty valuable, so I'll post it again: https://www.schneier.com/blog/archives/2018/11/propaganda_and_.html https://www.schneier.com/blog/archives/2018/11/propaganda_an...
- dexen 8y ago
  Meta: if a post is dead due to flagging / banning, but you find it valuable, you can "vouch" for it. With several (perhaps one?) users vouching, the post is back from dead. Click on the "30 minutes ago" link next to a dead post's author name to view this individual post, and now a "vouch" link should be available. Click it to perform vouching, done. This function not available for some users, depending on karma threshold and perhaps other factors.
  - pc86 8y ago
    I've never vouched for a post and had it not come back so I think when the account is shadow-banned at least it only takes one vote. I also very rarely vouch for it's possible there are additional factors at play.
johnchristopher 8y ago
The confidence attack has a patch: drop electronic voting.
tareqak 8y ago
I'm glad that someone of Schneier's reputation and ability is taking this issue seriously enough to at least co-author a paper on it. I know that the focus of this forum is technology news and whatever hackers find interesting, and I know (and have seen here) that the more political a particular post is, the more inevitable it becomes for the discourse to devolve into something non-constructive and toxic. At the same time, I strongly feel that as part of the community people who have designed and implemented the technologies that have had such an major impact on civil discourse and politics today, I find the reluctance to discuss and own up to the fact disappointing. Even the scientists behind the Manhattan Project felt and expressed some responsibility and remorse for the nuclear weapons that came about from their work.
- matt4077 8y ago
  I fear you will be disappointed... Large parts of the tech community seem to not just be blind to the consequences of their work, but to openly embrace and nurture the destruction of the fabric of society. This used to find voice in utopian visions of a sort of libertarian, meritocratic revival of democracy: bloggers replacing journalists, "makers", liquid democracy, etc. There are two successful examples of this spirit I can think of: Wikipedia, and OSS. Unfortunately, this movement also had/has a destructive streak. Partly because these new ideas had existing competitors that needed to be cut down to make room, and partly because they experienced opposition from existing players (sometimes only tangentially related) that quickly became branded as enemies. Two sides of the same philosophy. Guess which one had more staying power? Just look at the fate of The Pirate Bay vs The Pirate Party to get an idea. Or take this quiz: (a) Name a website distributing scientific papers with no concern for copyright. (b) Name an Open Access journal. With regard to the specific topic of the paper, namely information (and political news specifically) those ideas of the citizen-blogger have actually disappeared so thoroughly, you are likely to have no idea what I'm referencing if you are under 30 years of age. And while those ideas were initially coupled with a disdain for established institutions and the press because it was a storyline in need of a villain, the ideas died yet the rot feasting on our sources of shared truth survived. The target of all this destructive energy is, as a first approximation, the very concept of trust. Trust cannot be trusted is a sort-of mantra, that not only gives sense to what would otherwise just be existential dread aimlessly seeking escape in vandalism (4chan). It also makes you appear cool & in the know: "I wonder who paid for this article", "everybody knows a study with n=20000 is underpowered", "<X> wouldn't do <Y> unless <convoluted way to reduce all human activity to a profit motive>". On rare occasions, this destructive mindset still has the spark of creativity: Bitcoin, for all its flaws, is (was?) somewhat impressive. Yet it was always rooted in this sort of cynicism that distrusts institutions and the power of humans to have any positive impact with anything but the tools of physics and math: to wit, the endless conspiracy theories around the FED, the infatuation with Gold and land, etc. In the realm of politics, the destruction is just about total. Nothing of value was created. Meanwhile, the community gleefully watches the destruction of the free press, fine-tuning their adblockers because "information wants to be free", or because that newspaper whose articles they desperately want to read nonetheless made the fateful error of using the wrong JS framework, or something, but in any case, it's their fault if they can't survive. Plus they are just part of Soros' campaign anyway. Everybody knows that.
forapurpose 8y ago
We are strongly concerned with accuracy (correctness, completeness, consistency) on the network layers but I've been thinking that we need to apply the same emphasis to accuracy on the data layer. We'd never accept the data layer's error rate on the network layer. Another way of looking at it: We do emphasize accuracy in data in business databases - again, we'd never accept the error rate common in 'general' information. Why don't we put the same emphasis on data elsewhere?
- posterboy 8y ago
  We totally do, that's why most peoples opinion is irrelevant, and voting a rather limited direct influence, not to say useless.
- robinsloan 8y ago
  I think it depends on which "we" you're talking about! In many cases, "we" do value accurate information. I'm thinking not just of high-quality news organizations, but fact-checking sites like Snopes or Politifact -- both real treasures, in my opinion. However, if you cite either On The Internet, you'll often receive the response: "Pshh please. [Fact-checking site] is totally biased." This response is, in my opinion, (a) wrong, and (b) very often offered in bad faith -- an example of the very tactic discussed in the paper linked above -- but those features don't prevent it from short-circuiting the "error correction" you're seeking. So, if enterprises like Snopes and Politifact don't work... how DO you establish some kind of shared understanding of the world for a very broad "we"? It's really frustrating!
  - PavlovsCat 8y ago
    > how DO you establish some kind of shared understanding of the world for a very broad "we"? I think one can't undo decades of living in a completely separate social millieu, even in the same city in the same generation, by pointing at a website. How much less so when people come from different parts of the country, or even neighbouring countries, or even are from different continents. It takes time, and patience, and a lot of getting it wrong and still not giving up. It also helps to not drift apart in the first place. How many people are ignored or even mocked for being not so intelligent or other issues -- until they "become a problem", or rant at people on the bus, and then the other people can't figure out why they have no way to reach them, a person who they didn't even see until they became a problem. At that point it may or may not be possible anymore, but it didn't begin at the point where it became too problematic for us to ignore. Heck, how often do "stupid" people get gleefully blamed for being gullible and getting exploited? A lot of things that are shrugged off with "makes business sense" do hurt real people, pile on real bitterness onto a life that may already be hard enough. The bitterness doesn't disappear just because it's ignored or even censored, to the contrary. I would say that we didn't lose goodwill and solidarity towards each other because of language, differing sets of agreed on facts, but we somehow shed our goodwill and solidarity, and then used language to try and cover it up. Now language is losing its power, facts cease to matter, wise or knowledgeable people command no respect -- but that may be more a symptom than a cause. To put a sharper point on it: when clever slogans were still useful to get people in line and to exploit them systematically, the intelligentsia didn't exactly scream bloody murder and went on strike, it for the most part went along. So now factuality doesn't matter anymore, tribes and slogans do. Predictably, understandably so. This cannot be repaired without acknowledging intellectual dishonesty, and misdeeds against "the common man", something neither government or corporate or intellectual elites seem to be prepared to do in earnest. > They who have put out the people's eyes reproach them of their blindness. -- John Milton Yet I can talk with a person and not be able to follow their argument at all, or flat out disagree with that they consider to be fact, but still know from their body language and past interactions that they have goodwill towards me. If I also feel goodwill towards them and they know that, we may not "learn" something from each other that day, but even then that conversations can be a pleasant experience that increases our familiarity and sympathy. If I just act standoff-ish until I'm convinced the other person is thinking all the right things, I'll never get anywhere. Think how even animals of different species sometimes can form friendships and arrangements without exchanging a single abstract concept. Or how small kids that don't speak the same language might still play and build together just fine (the awkwardness and getting hung up on pointless stuff seems to come later in life), and how we learn language in the first place just based on some basic trust and being allowed to experiment. If you have the mutual goodwill, it's not complex, it happens "by itself". Without it, it gets complicated or even impossible. And I think it stands to reason that the stronger, wealthtier, more knowledgeable etc. party should always be expected to be more generous and more patient, and offer goodwill by default, even when its not reciprocated yet. My 2 cents, etc.
liftbigweights 8y ago
Ironic considering SSRN is owned by Elsevier, which is renown for trying to limit/control information for profit. It even published fake journals to spread "fake news" ( aka ads ) for the pharmaceutical industry. "The company has fought legislation designed to open up academic research, offered scholars money to file positive reviews, sued libraries for oversharing, and allegedly published fake journals on behalf of the pharmaceuticals industry." https://www.newyorker.com/tech/annals-of-technology/when-the-rebel-alliance-sells-out https://www.newyorker.com/tech/annals-of-technology/when-the...
zyxzevn 8y ago
I disagree, but I see the problems totally different. That is because I don't see the US as a democracy but as an oligarchy. The whole problem with US implementation of democracy is that the actors are centralized around a system that encourages corruption. The corruption is due to the power and money that is governing politics. And this again causes secrecy to hide the differences between the advertised politics and real practices. And to hide problems and responsibility within the political decisions. The corruption and secrecy again causes the media to publish on actors in a very biased way. It is even seen as bad to be positive or unbiased on the policies or events. If something goes wrong it is always the other's fault. The media earns now money by presenting the stories to support certain opinions and ideas. Not by presenting the complex and multifaceted reality. This is emphasized by the two party system. And sometimes also by the CIA propaganda system that is still active. This goes so far that there is a lot of staging of the presentation of events. Which makes the news far more dramatic. But it also makes it fake, whether the story really happened or not. The media also tends to emphasize minor problems, just to trigger emotional reactions. And then there is the problem of the over-militarization of the US government and its foreign policies. This is visible in the excessive amount of money going into this. And the huge amount of money lost in it. This gives the problem that the military and their corrupt sponsors rule the politics, instead of the people that are part of the democracy. It is also the reason why the military (&CIA) controls the media narrative on foreign politics so strongly. The first step would be to get the money out of the politics. http://represent.us http://represent.us has a good way of doing that. The media circus can be stopped by allowing more factions/parties and different viewpoints simultaneously. But that means stepping away from the two-party idea. That way different opinions become less hostile against each other. Separate military propaganda from the news. Now it is completely mixed up, because the military don't want to be unmasked. That is opposite of what the paper seems to be stating. "Stable autocracies will have common knowledge over who is in charge and their associated ideological or policy goals". In Europe the democratic parties have clear ideologies and goals. In the US this is not so much. In the US it is clear that democrats or republicans are in charge, but neither represent the people. Both represent the companies and organizations that pay them. If you want people to get more informed and involved with democracy, you need to decentralize the democracy. Make local people's votes count. And give people more autocracy/ self responsibility. That works in countries like Switzerland. In the US there is also a lot corruption on local levels. Give the people power and knowledge to stop that! That is democracy. A major problem in the US is that many want to control how others live. They want to control what drugs they use, or how they reproduce (or not). Who pays for who. Often mixed with people paying money to certain companies and monopolies. This is politics directly against self-responsibility. Instead the politics should be directed towards cooperation and build-up in a way that encourages self-responsibility. This is also represented in the information. Spread information that is self-responsible and cooperative, instead of information that is controlling, biased and/or emotional. For example: Wikipedia (even with its errors) helps the people around the world understand most of the world. But certain biased information sources also give information of how other people may be thinking. That way we understand each other, by learning more. This is also complex and multifaceted. But it is something that people are very much interested in dealing with themselves. I do not have a good idea, how the excessive military expenses and its influences can be reduced.
rrggrr 8y ago
The only counter-measure that works against polluting the polity with influence operations is earnestly educating people to be critical consumers of information. Why? Because one cannot empower government to counter-message influence operations without risking an abuse of power. My son's middle school appears to be tackling this problem to some degree by teaching students to: (a) not accept 'facts' without multiple sources of information; (b) consider the perspective/bias of the story-teller in the sources you consume; and (c) understand that most messages are persuasion.
- zackmorris 8y ago
  For the reasons you gave: as a litmus test for whether a politician is a statesman or a shill, I look at whether he/she supports public education or not.
  - wowzap 8y ago
    You should do some reading on pre-public education America, there actually wasn't much demand for it and private education served communities very well.
    - vkou 8y ago
      Exactly how well did private education serve black communities in America? That demographic had a >50% illiteracy rate, in 1900. [1] [1] https://nces.ed.gov/naal/lit_history.asp https://nces.ed.gov/naal/lit_history.asp
      - NeoBasilisk 8y ago
        You won't get an answer to this.
        excalibur 8y ago
        It was a rhetorical question.
    - 8y ago
blackholesRhot 8y ago
I respect Schneier but this is basically 10x-ing the word count on Terry Tao’s 2016 blog post: https://www.google.com/amp/s/terrytao.wordpress.com/2016/06/04/it-ought-to-be-common-knowledge-that-donald-trump-is-not-fit-for-the-presidency-of-the-united-states-of-america/amp/ https://www.google.com/amp/s/terrytao.wordpress.com/2016/06/... Unacceptable to me that it wasn’t referenced as it certainly made the rounds in security circles.
- r00fus 8y ago
  When I click through I get commentary about the 2016 election. Please update your link.
tareqak 8y ago
There was one idea that came to mind when I read The Second Half of Watergate Was Bigger, Worse, and Forgotten by the Public posted here 8 days ago [0][1]. The first part of the Foreign Corrupt Practices Act (FCPA) requires corporations to keep accurate financial records in order to not be able to hide bribes [2]. If a corporation can be compelled to be truthful in their financial records, then why can't a news organization be compelled to be truthful when proclaiming to disseminate news? If the news organization still wanted to publish something inaccurate, then they could label it as satire, but at least they would have surrender the appearance of being truthful. I understand that there would be freedom-of-speech / first amendment implications, but isn't a public financial record a kind of speech and corporations are effectively being told how to say (be truthful) what they want to say (their finances)? My idea definitely sounds a little far-fetched even to me, but I'd appreciate any input, additions, or criticism that anyone might have. [0] https://longreads.com/2018/11/20/the-second-half-of-watergate-was-bigger-worse-and-forgotten-by-the-public/ https://longreads.com/2018/11/20/the-second-half-of-watergat... [1] https://news.ycombinator.com/item?id=18498796 https://news.ycombinator.com/item?id=18498796 [2] https://en.wikipedia.org/wiki/Foreign_Corrupt_Practices_Act https://en.wikipedia.org/wiki/Foreign_Corrupt_Practices_Act
- JediWing 8y ago
  Lots of risk to free speech, with little benefit. Who decides the truth? All the same problems with automatic/AI approaches to fact-checking are repeated but in the legal arena. Also, I don't think it's really comparable to say "hey make sure what you say about the money receive is true", vs. "hey make sure everything you say is true"
  - wahern 8y ago
    > Who decides the truth? A jury? In most Western societies the final arbiter of "truth" in a legal dispute is the court, usually the judge but (especially in the U.S.) the jury. If the court's fact finder says the sky isn't blue then the sky isn't blue, period, at least in the context of the dispute. This is one of the principle roles of the courts, and it's why the judiciary is supposed to be independent of the executive organs. The U.S. is unique in holding journalists and politicians to a lower standard of truth. The U.K. has tough defamation laws that are arguably abused to quiet newspapers, but even so it's a stretch to equivocate the state of free speech in the U.K. to authoritarian societies. That is to say, few people would argue that U.K. citizens don't as a general matter enjoy free speech rights commensurate with American citizens. A legal principle like defamation, however, requires a cognizable injury to a particular person. It's difficult to pin point the injuries caused by any particular piece of fakes news, so you may have to loosen that standard if you want to encompass fake news. Recognizing, e.g., injuries to the public can be dangerous. OTOH the requirement of intent can help to reign that in.
gumby 8y ago
Are words like "epistemology" (or "philosophy") scary for American students (or teachers?). French and German kids have philosophie or Ethik from an early age.
pessimizer 8y ago
It's a huge flaw that this paper keeps referring to the "US" and its intentions, even explicitly distinguished from the FCC or its president in the blog entry linking the paper[1]. Baked into the examples used to illustrate its premise is a United States that is never specifically located or identified; it just makes appearances as "some Arizona Republicans" and "national security officials" and only intends to spread its "liberal and democratic values" with its "pro-freedom" bias. The idea that Americans are now soaked in more or more vulnerable to intentional misdirection due to ideas flooding the internet from twitter and facebook is completely ahistorical. If anything, American "consensus" beliefs have always been dictated and enforced from above, and have never been a consensus. I hope that he reads some Walter Lippmann before he continues to treat politics like a computer program. We are and have always been constantly under attacks from people who want to define the facts that we base our decisions on, including all parts of government. Additionally, those attackers do not always have bad intentions, and may be using deceptive simplifications in order to trick us into doing what they think is best for us. The evaluation of a flood of information coming from actors with a full range of motivations to manipulate that information is the basic dilemma of democracy. This paper itself is soaked in and re-enforces a bunch of questionable common knowledge, especially as it seems to be addressed to an American flag when only people are available to read it. [1] https://www.schneier.com/blog/archives/2018/11/propaganda_and_.html https://www.schneier.com/blog/archives/2018/11/propaganda_an...

207 points by yungchin 8y ago

Great model! Simple enough to understand, fits a lot of recent phenomena. Scary implications, but we should be able to respond effectively. Unexpected but good to see Schneier doing this kind of work.

nbp234 8y ago

Link to discussion on Bruce Schneier's blog: https://www.schneier.com/blog/archives/2018/11/propaganda_and_.html https://www.schneier.com/blog/archives/2018/11/propaganda_an...

dfabulich 8y ago
The comments on that blog post are examples of the problem. They're diving straight into disagreements about matters of fact (like whether a party cheated in a particular election, and whether both sides cheat). The point is that we're not (no longer) disagreeing about values, but about the facts, and that authoritarians are deliberately attempting to muddy the waters here. They're clearly succeeding, and the commenters seem to be distracted from that.
- commandlinefan 8y ago
  > examples of the problem on the other hand, they both seem to come to the same conclusion: that we need our election machinery to be verifiably tamper-proof.

darawk 8y ago

Ya, i've been saying something like this for a while. China, for instance, is uniquely invulnerable to this sort of attack. Because the state is already in charge of censoring information, it is much more difficult to flood them with misinformation. It's one of the many reasons i'm concerned that models like Chinas will actually flourish more than they otherwise would in the coming years.

forapurpose 8y ago
They may be less vulnerable to attacks from outsiders, but they are much more vulnerable to attacks from their own government. It's similar to totalitarian governments; they might keep organized crime low, but they expose people to far more crime from the government itself. I would expect people in China and in the Chinese government to also be more vulnerable to misleading and suboptimal information, because there is a lack of competing ideas.
- hammock 8y ago
  >They may be less vulnerable to attacks from outsiders, but they are much more vulnerable to attacks from their own government. It's similar to totalitarian governments Interesting that you say that about China, because that was exactly Bruce Schneier's conclusion about the United States: >Our research implies that insider attacks from within American politics can be more pernicious than attacks from other countries. https://www.schneier.com/blog/archives/2018/11/propaganda_and_.html https://www.schneier.com/blog/archives/2018/11/propaganda_an...
  - commandlinefan 8y ago
    > Without any evidence whatsoever, he said that Democrats were trying to steal the election through "FRAUD." Well, wait a minute - both of those races had already been called in favor of the Republican candidate on the night of the election until boxes and boxes of mysteriously Democrat-leaning ballots were suddenly found the following day... I think that constitutes evidence, even if it's not conclusive.
    - orf 8y ago
      Evidence of a poorly managed election system split into tens of thousands of individual parts across 50 states in dire need of reform (but will never be, because one party benefits from the status quo), or evidence of a Democratic conspiracy to fraudulently seize power by creating 'mysterious' ballot boxes in the most hamfisted and public way possible? Sure.

baby 8y ago

It'd be interesting to see a formal threat model of a Democracy like we write such documents for our applications and protocols.

Balgair 8y ago
If you have a lot of know-how about these threat models, I'd highly encourage you to write one for democracy as an idea. I'd love to read it!
- baby 8y ago
  https://github.com/mimoo/democracy_threat_model https://github.com/mimoo/democracy_threat_model

peterwwillis 8y ago

Democracy is doomed, but not because of advanced attacks. Autocracy/nationalism/monarchy/etc are just the natural order of human societies. We've had a good run, but so did the Greeks and Romans.

deleted 8y ago
[deleted]
Balgair 8y ago
To be fair, Italy and Greece are republics (for the umpteenth time) as of today.
deogeo 8y ago
Isn't nationalism orthogonal to democracy? How are they opposed?
- peterwwillis 8y ago
  They're not opposed. Nationalism just has the tendency of galvanizing a population's support, which can easily bolster an autocracy that represents the "national identity". It's sort of the supporting character for autocracy and related political systems. (I'm just basing this on opinion, not political theory)
- wahern 8y ago
  It's sort of like gimbal lock: when a particular social movement becomes so pervasive and powerful the mechanisms of government become dangerously aligned and you lose the ability of the political system to perform an important function, namely defend itself and minorities from the tyranny of the majority. Remember, part of the design of separation of powers is that institutions must retain the ability and motivation to compete for power. If the motivation is gone the ability becomes at risk--either by atrophy or by one institution making power grabs that go unchallenged by others institutions or by the people. Discord is, to an extent, a feature, not a bug. Hugo Chavez was a nationalist that was popularly and fairly elected multiple times until, eventually, the elections weren't actually fair anymore and the institutions of government were irreparably corrupt. Nationalist movements aren't the only the way this can naturally occur, but it's perhaps the most common in the past two centuries.

tareqak 8y ago

I don't know why nbp234's comment with the link to Bruce Schneier's blog is dead, but I think it is pretty valuable, so I'll post it again: https://www.schneier.com/blog/archives/2018/11/propaganda_and_.html https://www.schneier.com/blog/archives/2018/11/propaganda_an...

dexen 8y ago
Meta: if a post is dead due to flagging / banning, but you find it valuable, you can "vouch" for it. With several (perhaps one?) users vouching, the post is back from dead. Click on the "30 minutes ago" link next to a dead post's author name to view this individual post, and now a "vouch" link should be available. Click it to perform vouching, done. This function not available for some users, depending on karma threshold and perhaps other factors.
- pc86 8y ago
  I've never vouched for a post and had it not come back so I think when the account is shadow-banned at least it only takes one vote. I also very rarely vouch for it's possible there are additional factors at play.

johnchristopher 8y ago

The confidence attack has a patch: drop electronic voting.

tareqak 8y ago

I'm glad that someone of Schneier's reputation and ability is taking this issue seriously enough to at least co-author a paper on it. I know that the focus of this forum is technology news and whatever hackers find interesting, and I know (and have seen here) that the more political a particular post is, the more inevitable it becomes for the discourse to devolve into something non-constructive and toxic. At the same time, I strongly feel that as part of the community people who have designed and implemented the technologies that have had such an major impact on civil discourse and politics today, I find the reluctance to discuss and own up to the fact disappointing. Even the scientists behind the Manhattan Project felt and expressed some responsibility and remorse for the nuclear weapons that came about from their work.

matt4077 8y ago
I fear you will be disappointed... Large parts of the tech community seem to not just be blind to the consequences of their work, but to openly embrace and nurture the destruction of the fabric of society. This used to find voice in utopian visions of a sort of libertarian, meritocratic revival of democracy: bloggers replacing journalists, "makers", liquid democracy, etc. There are two successful examples of this spirit I can think of: Wikipedia, and OSS. Unfortunately, this movement also had/has a destructive streak. Partly because these new ideas had existing competitors that needed to be cut down to make room, and partly because they experienced opposition from existing players (sometimes only tangentially related) that quickly became branded as enemies. Two sides of the same philosophy. Guess which one had more staying power? Just look at the fate of The Pirate Bay vs The Pirate Party to get an idea. Or take this quiz: (a) Name a website distributing scientific papers with no concern for copyright. (b) Name an Open Access journal. With regard to the specific topic of the paper, namely information (and political news specifically) those ideas of the citizen-blogger have actually disappeared so thoroughly, you are likely to have no idea what I'm referencing if you are under 30 years of age. And while those ideas were initially coupled with a disdain for established institutions and the press because it was a storyline in need of a villain, the ideas died yet the rot feasting on our sources of shared truth survived. The target of all this destructive energy is, as a first approximation, the very concept of trust. Trust cannot be trusted is a sort-of mantra, that not only gives sense to what would otherwise just be existential dread aimlessly seeking escape in vandalism (4chan). It also makes you appear cool & in the know: "I wonder who paid for this article", "everybody knows a study with n=20000 is underpowered", "<X> wouldn't do <Y> unless <convoluted way to reduce all human activity to a profit motive>". On rare occasions, this destructive mindset still has the spark of creativity: Bitcoin, for all its flaws, is (was?) somewhat impressive. Yet it was always rooted in this sort of cynicism that distrusts institutions and the power of humans to have any positive impact with anything but the tools of physics and math: to wit, the endless conspiracy theories around the FED, the infatuation with Gold and land, etc. In the realm of politics, the destruction is just about total. Nothing of value was created. Meanwhile, the community gleefully watches the destruction of the free press, fine-tuning their adblockers because "information wants to be free", or because that newspaper whose articles they desperately want to read nonetheless made the fateful error of using the wrong JS framework, or something, but in any case, it's their fault if they can't survive. Plus they are just part of Soros' campaign anyway. Everybody knows that.

forapurpose 8y ago

We are strongly concerned with accuracy (correctness, completeness, consistency) on the network layers but I've been thinking that we need to apply the same emphasis to accuracy on the data layer. We'd never accept the data layer's error rate on the network layer. Another way of looking at it: We do emphasize accuracy in data in business databases - again, we'd never accept the error rate common in 'general' information. Why don't we put the same emphasis on data elsewhere?

posterboy 8y ago
We totally do, that's why most peoples opinion is irrelevant, and voting a rather limited direct influence, not to say useless.
robinsloan 8y ago
I think it depends on which "we" you're talking about! In many cases, "we" do value accurate information. I'm thinking not just of high-quality news organizations, but fact-checking sites like Snopes or Politifact -- both real treasures, in my opinion. However, if you cite either On The Internet, you'll often receive the response: "Pshh please. [Fact-checking site] is totally biased." This response is, in my opinion, (a) wrong, and (b) very often offered in bad faith -- an example of the very tactic discussed in the paper linked above -- but those features don't prevent it from short-circuiting the "error correction" you're seeking. So, if enterprises like Snopes and Politifact don't work... how DO you establish some kind of shared understanding of the world for a very broad "we"? It's really frustrating!
- PavlovsCat 8y ago
  > how DO you establish some kind of shared understanding of the world for a very broad "we"? I think one can't undo decades of living in a completely separate social millieu, even in the same city in the same generation, by pointing at a website. How much less so when people come from different parts of the country, or even neighbouring countries, or even are from different continents. It takes time, and patience, and a lot of getting it wrong and still not giving up. It also helps to not drift apart in the first place. How many people are ignored or even mocked for being not so intelligent or other issues -- until they "become a problem", or rant at people on the bus, and then the other people can't figure out why they have no way to reach them, a person who they didn't even see until they became a problem. At that point it may or may not be possible anymore, but it didn't begin at the point where it became too problematic for us to ignore. Heck, how often do "stupid" people get gleefully blamed for being gullible and getting exploited? A lot of things that are shrugged off with "makes business sense" do hurt real people, pile on real bitterness onto a life that may already be hard enough. The bitterness doesn't disappear just because it's ignored or even censored, to the contrary. I would say that we didn't lose goodwill and solidarity towards each other because of language, differing sets of agreed on facts, but we somehow shed our goodwill and solidarity, and then used language to try and cover it up. Now language is losing its power, facts cease to matter, wise or knowledgeable people command no respect -- but that may be more a symptom than a cause. To put a sharper point on it: when clever slogans were still useful to get people in line and to exploit them systematically, the intelligentsia didn't exactly scream bloody murder and went on strike, it for the most part went along. So now factuality doesn't matter anymore, tribes and slogans do. Predictably, understandably so. This cannot be repaired without acknowledging intellectual dishonesty, and misdeeds against "the common man", something neither government or corporate or intellectual elites seem to be prepared to do in earnest. > They who have put out the people's eyes reproach them of their blindness. -- John Milton Yet I can talk with a person and not be able to follow their argument at all, or flat out disagree with that they consider to be fact, but still know from their body language and past interactions that they have goodwill towards me. If I also feel goodwill towards them and they know that, we may not "learn" something from each other that day, but even then that conversations can be a pleasant experience that increases our familiarity and sympathy. If I just act standoff-ish until I'm convinced the other person is thinking all the right things, I'll never get anywhere. Think how even animals of different species sometimes can form friendships and arrangements without exchanging a single abstract concept. Or how small kids that don't speak the same language might still play and build together just fine (the awkwardness and getting hung up on pointless stuff seems to come later in life), and how we learn language in the first place just based on some basic trust and being allowed to experiment. If you have the mutual goodwill, it's not complex, it happens "by itself". Without it, it gets complicated or even impossible. And I think it stands to reason that the stronger, wealthtier, more knowledgeable etc. party should always be expected to be more generous and more patient, and offer goodwill by default, even when its not reciprocated yet. My 2 cents, etc.

liftbigweights 8y ago

Ironic considering SSRN is owned by Elsevier, which is renown for trying to limit/control information for profit. It even published fake journals to spread "fake news" ( aka ads ) for the pharmaceutical industry. "The company has fought legislation designed to open up academic research, offered scholars money to file positive reviews, sued libraries for oversharing, and allegedly published fake journals on behalf of the pharmaceuticals industry." https://www.newyorker.com/tech/annals-of-technology/when-the-rebel-alliance-sells-out https://www.newyorker.com/tech/annals-of-technology/when-the...

zyxzevn 8y ago

I disagree, but I see the problems totally different. That is because I don't see the US as a democracy but as an oligarchy. The whole problem with US implementation of democracy is that the actors are centralized around a system that encourages corruption. The corruption is due to the power and money that is governing politics. And this again causes secrecy to hide the differences between the advertised politics and real practices. And to hide problems and responsibility within the political decisions. The corruption and secrecy again causes the media to publish on actors in a very biased way. It is even seen as bad to be positive or unbiased on the policies or events. If something goes wrong it is always the other's fault. The media earns now money by presenting the stories to support certain opinions and ideas. Not by presenting the complex and multifaceted reality. This is emphasized by the two party system. And sometimes also by the CIA propaganda system that is still active. This goes so far that there is a lot of staging of the presentation of events. Which makes the news far more dramatic. But it also makes it fake, whether the story really happened or not. The media also tends to emphasize minor problems, just to trigger emotional reactions. And then there is the problem of the over-militarization of the US government and its foreign policies. This is visible in the excessive amount of money going into this. And the huge amount of money lost in it. This gives the problem that the military and their corrupt sponsors rule the politics, instead of the people that are part of the democracy. It is also the reason why the military (&CIA) controls the media narrative on foreign politics so strongly. The first step would be to get the money out of the politics. http://represent.us http://represent.us has a good way of doing that. The media circus can be stopped by allowing more factions/parties and different viewpoints simultaneously. But that means stepping away from the two-party idea. That way different opinions become less hostile against each other. Separate military propaganda from the news. Now it is completely mixed up, because the military don't want to be unmasked. That is opposite of what the paper seems to be stating. "Stable autocracies will have common knowledge over who is in charge and their associated ideological or policy goals". In Europe the democratic parties have clear ideologies and goals. In the US this is not so much. In the US it is clear that democrats or republicans are in charge, but neither represent the people. Both represent the companies and organizations that pay them. If you want people to get more informed and involved with democracy, you need to decentralize the democracy. Make local people's votes count. And give people more autocracy/ self responsibility. That works in countries like Switzerland. In the US there is also a lot corruption on local levels. Give the people power and knowledge to stop that! That is democracy. A major problem in the US is that many want to control how others live. They want to control what drugs they use, or how they reproduce (or not). Who pays for who. Often mixed with people paying money to certain companies and monopolies. This is politics directly against self-responsibility. Instead the politics should be directed towards cooperation and build-up in a way that encourages self-responsibility. This is also represented in the information. Spread information that is self-responsible and cooperative, instead of information that is controlling, biased and/or emotional. For example: Wikipedia (even with its errors) helps the people around the world understand most of the world. But certain biased information sources also give information of how other people may be thinking. That way we understand each other, by learning more. This is also complex and multifaceted. But it is something that people are very much interested in dealing with themselves. I do not have a good idea, how the excessive military expenses and its influences can be reduced.

rrggrr 8y ago

The only counter-measure that works against polluting the polity with influence operations is earnestly educating people to be critical consumers of information. Why? Because one cannot empower government to counter-message influence operations without risking an abuse of power. My son's middle school appears to be tackling this problem to some degree by teaching students to: (a) not accept 'facts' without multiple sources of information; (b) consider the perspective/bias of the story-teller in the sources you consume; and (c) understand that most messages are persuasion.

zackmorris 8y ago
For the reasons you gave: as a litmus test for whether a politician is a statesman or a shill, I look at whether he/she supports public education or not.
- wowzap 8y ago
  You should do some reading on pre-public education America, there actually wasn't much demand for it and private education served communities very well.
  - vkou 8y ago
    Exactly how well did private education serve black communities in America? That demographic had a >50% illiteracy rate, in 1900. [1] [1] https://nces.ed.gov/naal/lit_history.asp https://nces.ed.gov/naal/lit_history.asp
    - NeoBasilisk 8y ago
      You won't get an answer to this.
      - excalibur 8y ago
        It was a rhetorical question.
  - 8y ago

blackholesRhot 8y ago

I respect Schneier but this is basically 10x-ing the word count on Terry Tao’s 2016 blog post: https://www.google.com/amp/s/terrytao.wordpress.com/2016/06/04/it-ought-to-be-common-knowledge-that-donald-trump-is-not-fit-for-the-presidency-of-the-united-states-of-america/amp/ https://www.google.com/amp/s/terrytao.wordpress.com/2016/06/... Unacceptable to me that it wasn’t referenced as it certainly made the rounds in security circles.

r00fus 8y ago
When I click through I get commentary about the 2016 election. Please update your link.

tareqak 8y ago

There was one idea that came to mind when I read The Second Half of Watergate Was Bigger, Worse, and Forgotten by the Public posted here 8 days ago [0][1]. The first part of the Foreign Corrupt Practices Act (FCPA) requires corporations to keep accurate financial records in order to not be able to hide bribes [2]. If a corporation can be compelled to be truthful in their financial records, then why can't a news organization be compelled to be truthful when proclaiming to disseminate news? If the news organization still wanted to publish something inaccurate, then they could label it as satire, but at least they would have surrender the appearance of being truthful. I understand that there would be freedom-of-speech / first amendment implications, but isn't a public financial record a kind of speech and corporations are effectively being told how to say (be truthful) what they want to say (their finances)? My idea definitely sounds a little far-fetched even to me, but I'd appreciate any input, additions, or criticism that anyone might have. [0] https://longreads.com/2018/11/20/the-second-half-of-watergate-was-bigger-worse-and-forgotten-by-the-public/ https://longreads.com/2018/11/20/the-second-half-of-watergat... [1] https://news.ycombinator.com/item?id=18498796 https://news.ycombinator.com/item?id=18498796 [2] https://en.wikipedia.org/wiki/Foreign_Corrupt_Practices_Act https://en.wikipedia.org/wiki/Foreign_Corrupt_Practices_Act

JediWing 8y ago
Lots of risk to free speech, with little benefit. Who decides the truth? All the same problems with automatic/AI approaches to fact-checking are repeated but in the legal arena. Also, I don't think it's really comparable to say "hey make sure what you say about the money receive is true", vs. "hey make sure everything you say is true"
- wahern 8y ago
  > Who decides the truth? A jury? In most Western societies the final arbiter of "truth" in a legal dispute is the court, usually the judge but (especially in the U.S.) the jury. If the court's fact finder says the sky isn't blue then the sky isn't blue, period, at least in the context of the dispute. This is one of the principle roles of the courts, and it's why the judiciary is supposed to be independent of the executive organs. The U.S. is unique in holding journalists and politicians to a lower standard of truth. The U.K. has tough defamation laws that are arguably abused to quiet newspapers, but even so it's a stretch to equivocate the state of free speech in the U.K. to authoritarian societies. That is to say, few people would argue that U.K. citizens don't as a general matter enjoy free speech rights commensurate with American citizens. A legal principle like defamation, however, requires a cognizable injury to a particular person. It's difficult to pin point the injuries caused by any particular piece of fakes news, so you may have to loosen that standard if you want to encompass fake news. Recognizing, e.g., injuries to the public can be dangerous. OTOH the requirement of intent can help to reign that in.

gumby 8y ago

Are words like "epistemology" (or "philosophy") scary for American students (or teachers?). French and German kids have philosophie or Ethik from an early age.

pessimizer 8y ago

It's a huge flaw that this paper keeps referring to the "US" and its intentions, even explicitly distinguished from the FCC or its president in the blog entry linking the paper[1]. Baked into the examples used to illustrate its premise is a United States that is never specifically located or identified; it just makes appearances as "some Arizona Republicans" and "national security officials" and only intends to spread its "liberal and democratic values" with its "pro-freedom" bias. The idea that Americans are now soaked in more or more vulnerable to intentional misdirection due to ideas flooding the internet from twitter and facebook is completely ahistorical. If anything, American "consensus" beliefs have always been dictated and enforced from above, and have never been a consensus. I hope that he reads some Walter Lippmann before he continues to treat politics like a computer program. We are and have always been constantly under attacks from people who want to define the facts that we base our decisions on, including all parts of government. Additionally, those attackers do not always have bad intentions, and may be using deceptive simplifications in order to trick us into doing what they think is best for us. The evaluation of a flood of information coming from actors with a full range of motivations to manipulate that information is the basic dilemma of democracy. This paper itself is soaked in and re-enforces a bunch of questionable common knowledge, especially as it seems to be addressed to an American flag when only people are available to read it. [1] https://www.schneier.com/blog/archives/2018/11/propaganda_and_.html https://www.schneier.com/blog/archives/2018/11/propaganda_an...