18 ms·
Using Metadata to Find Paul Revere
- msandford 13y agoThis is a really great counterpoint to the "if you have nothing to hide..." crowd.
- mseebach 13y agoWell, Paul Revere did have something to hide.
- msandford 13y agoYes he did. The point is that having something to hide doesn't necessarily mean you're a criminal or a terrorist. Or that in some situations what our government today would classify as "criminal" or "terrorist" is in fact an act of patriotism. We all know that history is written by the victors so I won't suggest that what Paul Revere and the other revolutionaries did was an unmitigated good. But I will argue that needing to expend the resources it takes (salary, benefits, management, etc) to have one human being monitor another does put a pretty good check on the ability of those in power to stifle dissent. It also provides some kind of a critical threshold over which the revolutionaries can't reasonably be stopped. In many ways that's a good thing provided it's a high enough level. Once the public opinion swings the best thing to happen is for those in power to give up on suppression and start packing their bags.
- deleted 13y ago[deleted]
- quchen 13y agoWhile being an interesting read, the "old world" scenario was kind confusing me every couple of lines. (Could be because I'm not a native speaker.)
- aidenn0 13y agoIf you're not a native speaker, you're probably also not as well versed in the folklore of the US revolution. Paul Revere is a popular hero of the US revolution, so using him as the target terrorist invokes "one mans terrorist is another mans patriot" argument at the same time as it demonstrates the usage of metadata.
- jroseattle 13y agoMakes me think of the ramifications of false positives.
- danso 13y agoI don't know if David Simon's (creator of "The Wire" and "Treme") defense of the Verizon phone records request ever made it onto HN's page: Here it is, if you hadn't seen it: http://davidsimon.com/we-are-shocked-shocked/ http://davidsimon.com/we-are-shocked-shocked/ Apparently it was so controversial that his site crashed from the traffic, and he had to tell everyone to chill out: http://davidsimon.com/nsa-and-fisa-commentary-calling-it/ http://davidsimon.com/nsa-and-fisa-commentary-calling-it/ Anyway, why I thought of that in relation to the OP was, that I think some defenders of the NSA and general government surveillance policies are just unaware of how technology can fundamentally change things...As Google leaders have been known to say, "Speed is a feature"...and so it's not the finding of information that makes the establishment of Google time in human civilization, but how fast Google allows us to do it. So that said, Simon is one of the journalists I have the absolute highest regard for...I'll be one of the many who think "The Wire" is the best TV drama ever, both for its artistic take and for its illustration of how institutions -- the police, the schools, the drug trade -- corrupt even the best of individuals. "The Wire" is heavily based off of the year that Simon embedded himself in the Baltimore homicide department...the book (which spawned a network TV show) is the best book about the practice of journalism I've ever read. After a year following the detectives, you'd think Simon would be pretty much in cahoots with the police...but he followed up "Homicide" with "The Corner", in which he spent a year embedded with drug dealers and their customers...apparently most of the friendsships he made in the Baltimore Police department evaporated after he published a book bringing sympathy to Baltimore's downtrodden. Anyway, I don't think Simon has a love for government or authority. But I do think he's a little naive when it comes to advances in technology and their consequences. When "The Wire" started, the police were focused mostly on tapping pay phones. By the time "The Wire" ended, the police were surprised at the advent of camera phones. So when Simon says he thinks the NSA and other law agencies won't abuse their wiretap authority, I believe him...because in much of his experience, the practical obstacles (such as, having to have an officer watch a payphone all day) made it basically impossible for blanket surveillance. But technology is different...I think Simon's -- and others who I respect -- mistake is to think that the game is being played the same as it always is. It may be the case that the NSA is staffed with as people as good and conscientious as anywhere else...but it's naive to think (as was the primary lesson in "The Wire") that the power they have will lead them astray...and to those of us affected by it, it makes no difference if the violations were intentional or accidental. Anyway, back to Paul Revere and the OP...I think it's a great example. But of course, what makes that educational scenario feasible is technology and the ability to record information (metadata or whatever you want to call it) in an organized way. Frankly, I kind of thought anyone who read 1984 would understand how technology changes everything. But yeah, I do think there are some well-meaning people don't grasp the technology, and if they did, they'd have a different opinion about the dangers of unchecked surveillance. * edit: misspelled conscientious as 'contentious' * edit: as an example of how much Simon continues to challenge the police as a citizen, here's an essay he wrote after the success of the Wire, in which he tried, as a citizen, to get the basic details of a cop-involved shooting, something that has always been public record. He eventually succeeded, and the revelations about the officer involved ended up jeopardizing the prosecution: http://davidsimon.com/in-baltimore-no-one-left-to-press-the-police/ http://davidsimon.com/in-baltimore-no-one-left-to-press-the-... But as you can see if you read the piece, Simon is not to thrilled with how the Internet has displaced newspaper journalism
- csears 13y agoSo could we spoof bogus metadata to hide Paul Revere? In other words, if a large enough number of people started start calling and texting random Verizon customers, tweeting with random people from the middle east, inviting random people with Muslim-sounding names to Google+ Hangouts, and commenting on every Facebook like with "This ____ is the bomb!"... could that tip the signal to noise ratio enough to defeat this type of analysis?
- maxerickson 13y agoI think if you had enough people engaged that you could do anything to affect the analysis that you might as well work to eliminate the injustice (rather than just smearing it out across more innocent people).
- blktiger 13y agoIf you haven't read it yet, check out Cory Doctorow's book Little Brother (http://craphound.com/littlebrother/about/ http://craphound.com/littlebrother/about/, http://craphound.com/littlebrother/download/ http://craphound.com/littlebrother/download/). He explores ways that average citizens could fight back against technology that watches our every move.
- crgt 13y agoMaybe a simple mail client that adds 'extra' metadata to everything you send? Extra recipients, etc? Would add more hay to the haystack.
- antoko 13y agoThis is very clever and all, but from the British perspective - this is exactly what they would have been looking for and Revere would be considered a revolutionary and a real threat, so this is the system working as it is supposed to. This is the equivalent of saying that the NSA can use such metadata to find muslim extremists that want to kill your children and implement sharia law. (hyperbole courtesy of American media) Changing the timeline such that "americans" are the underdogs rather than the establishment isn't really helpful... is it? Or is this attempt to stoke revolutionary sentiment?
- ant512 13y agoIt spins the story around in such a way that Americans can relate to the dangers of a surveillance state. You're suggesting that the NSA is just looking for bad guys and that they should be able to capture evil muslim extremists. But now the US government decides to bring back alcohol prohibition. Drinking, they say, is immoral, and so are the people who drink. Do you still have nothing to fear? Abortion gets outlawed, and its supporters are branded as immoral. Still got nothing to fear? Creationism gets accepted as a viable, scientific alternative to evolution and gets taught in all schools, and people who believe otherwise are dangerous subversives. Are you now one of the bad guys? Are the NSA now interested in you? I find it interesting that Americans can believe that the right to bear arms will protect them from the world's largest military, should it ever be run by tyrants, but at the same time believe that the government will always share the same moral code that they themselves hold.
- euroclydon 13y agoIt's often been said during during debates about government waste and inefficiency, that citizens do not actually want an efficient government. The typical example is: would you want a boot to appear on your wheel, in your driveway, the minute your license plate expires? Because that would be efficient. The average person feels that they are getting more benefit out of Google searches, Gmail, Facebook, cell phones, and credit cards for that matter, then they are being harmed by volunteering so much information to the corporations who run these businesses. If you think the U.S. is headed in the direction of outlawing alcohol or teaching creationism is all schools, I respectfully suggest you find something different to worry about -- maybe pot... wait, we are becoming more lenient on it as well. I find it a bit silly, that when the government adopts new technology to fight crime, everyone brings up their pet boogey-man to try to strike fear in their like-minded citizens. I honestly can't understand if you are in support of, for instance, using web surveillance technology to capture and prosecute abortion clinic bombers or not.
- lettergram 13y agoThis would mean the larger your network, the larger your likely hood of being a terrorist
- pseut 13y agoFrom a predictive standpoint, that's not necessarily an error. Think of it as "the larger your network, the higher priority to investigate for terrorism."
- rickyconnolly 13y agoYou ʃhould have uʃed the long ʃ in this article
- surlyadopter 13y agoThe purfuit of happineff?
- deadairspace 13y agoThe character is ſ, and it doesn't go at the ends of words, so it would be: The purſuit of happineſs.
- LiamMcCalloway 13y agoAcademic version: http://db.tt/6UA1bXZq http://db.tt/6UA1bXZq DIVIDE AND CONQUER: DISTORTED COMMUNICATION IN NETWORKS, POWER, AND WEALTH DISTRIBUTION Wilson Perez-Oviedo Cornell University and Banco Central del Ecuador Abstract In a society composed of a dictator and its citizens, what are the determinants of the political equilibrium between these two? What are the conditions for a successful citizens’ revolt? What kind of strategies do governments follow to prevent such revolts? The situation of these types of societies can be understood as a game played between a leader, who has to decide the distribution of the aggregate income, and a group of citizens who have the opportunity to revolt if they are unhappy with the distribution. Coordinated action by citizens is possible because they form nodes in a communication network. However, communication through the network is distorted, which could preclude the emergence of collective action among citizens. The network structure and the distortion level are determinants of the political equilibrium and wealth distribution. The model explains how the dictator could use propaganda, cooptation, and repression to increase his expected utility. Finally, the model is illustrated by applying it to cases in Nigeria and Zaire/Congo.
- dfc 13y agoLink to version that does not require signing up for and signing into drop box: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=522362 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=522362 Big brother's little sister is just as bad. Why not give the ssrn link?
- LiamMcCalloway 13y agoI didn't check whether the paper was publicly available and not on gated publishers' websites, my apologies.
- dmix 13y agoThis paper is excellent. Thanks for sharing.
- LiamMcCalloway 13y ago
- startupfounder 13y agoPaul Revere, "in the Fall of 1774 and winter of 1775, I was one of upwards of thirty, chiefly mechanics, who formed ourselves into a Committee for the purpose of watching British soldiers and gaining every intelligence on the movements of the Tories."[1] Paul Revere was essentially one of the founding members of this country's counter intelligence program against his oppressive government. He was the first in a long line of Mark Felts, Bradley Mannings and Edward Snowdens of this country. If we have a system where these whistleblowers are stopped before they can leak information on the system that catches them before they can whistleblow there is no turning back. This would not be the UNITED STATES OF AMERICA without Paul Revere and Edward Snowden. [1]http://en.wikipedia.org/wiki/Intelligence_in_the_American_Revolutionary_War#Paul_Revere_and_the_Mechanics http://en.wikipedia.org/wiki/Intelligence_in_the_American_Re...
- mpyne 13y agoYou do realize that Paul Revere would by that logic be considered a founding member of the country's counter intelligence program against enemies of the U.S. as well, right? I.e. the NSA. As far as Revere was concerned he was way past fighting against his government; he was fighting to support his country against Britain's government.
- flyinRyan 13y agoWhat are you talking about? Britain was his government/country. By today's definitions the man was a terrorist aid. The founding fathers were all terrorists doing things like the Boston Tea Party and shooting only officers when rules of the day prohibited that kind of behavior.
- mpyne 13y agoHave you ever read the Declaration of Independence? It didn't just declare that a new nation exists. It declared why that action was taken, and made clear that the action wasn't being taken lightly: http://www.archives.gov/exhibits/charters/declaration_transcript.html http://www.archives.gov/exhibits/charters/declaration_transc... > Prudence, indeed, will dictate that Governments long established should not be changed for light and transient causes; and accordingly all experience hath shewn, that mankind are more disposed to suffer, while evils are sufferable, than to right themselves by abolishing the forms to which they are accustomed. And later, > The history of the present King of Great Britain is a history of repeated injuries and usurpations, all having in direct object the establishment of an absolute Tyranny over these States. Seriously, read the whole thing. It is a wonderful document which stands on its own, and explains why these Americans no longer considered themselves British, and also why it was not some mere disagreement with their government. Now, are you going to claim that government operation of a legally-instituted (and reviewed!) program that the majority of the population supports is "absolute Tyranny" with the sole purpose of oppression of the American populace? I'm just saying, I'd prefer we at least attempt to give the press, the EFF, the ACLU, and the concerned citizenry such as myself and many of my own friends a shot at fixing it before we go saying that violence and terror is the answer.
- kevinpet 13y agoThis is a lot of fancy math and graphics that don't differ all that much from what a very simple analysis can give you. Only Revere belonged to five of the groups of interest. Of the three who belonged to four groups, two of them are in "top scorers" on centrality, and the third, with one of the two in centrality, is in the final table.
- rollo_tommasi 13y agoIt's an exceedingly simple example used to illustrate the technique. The methods included in the python library are robust enough to work just as well on much larger and more complicated data-sets but that would be too difficult to follow for someone who isn't familiar with the subject.
- kevinpet 13y agoI think it's hard to understand what the point is without a result that differs from the naive solution. That is, my naive "who's an important terrorist" is "who belongs to the most terrorist groups".
- sesqu 13y agoWhile belonging to the most terrorist groups will usually give the highest score, it's not quite the same as what betweenness actually measures - belonging to the most terrorist groups that are each large but do not share many members.
- dfc 13y agokjh the author of this link has a greatintroduction to emacs: http://kieranhealy.org/resources/emacs-starter-kit.html http://kieranhealy.org/resources/emacs-starter-kit.html He also has a lot of good reference material on latex/org-mode/pandoc. Definitely a good resource to have handy if you ever have a less-techie friend who wants to get away from MS word and its ilk.
- deleted 13y ago[deleted]
- deleted 13y ago[deleted]