5 ms·
Who Wrote the Anti-Trump New York Times Op-Ed? Using Tidytext
- ada1981 8y agoI really wish some news outlet was circulating the headline “text analsysis shows Trump to be most likely writer of anonymous memo!”
- mc32 8y agoIs it also not somewhat equally likely someone who wanted to remain unknown to mask their own lexicon so as to make it harder to detect who the real author(s) were? If someone were to do something like this, would it not be in their interest to assume someone else's lexicon or, at least, coopt the lexicon of others in general to throw of the lexicographic 'scent'? In other words we can't only make the presumption that this is the author(s)' natural language.
- platz 8y agofails Occam's razor
- yathern 8y agoThis is another abuse of pre-packaged logic which I see regularly. Occam's razor dictates that when there are competing hypotheses - the one that makes the least number of assumptions tends to be true. This doesn't mean "automatically assume the simplest explanation is gospel truth, and if someone suggests otherwise, simply say the magic words 'Occams Razor' to them". Maybe this occurred. Maybe it didn't. But dismissing it with what's essentially a prepackaged argument adds very little to the conversation. If I were to leak info, I certainly would try to hide my way of writing. Not sure how to do it effectively. Possibly run it through Google translate and back a few times. Then patch it up to make it readable?
- tomjakubowski 8y agoIt seems easier, although maybe riskier, to have a trusted friend (or even your attorney) ghostwrite it for you. That would also avoid reintroducing your idiosyncrasies during the "patch it up" step.
- learc83 8y agoIf it were me, I'd make an outline and have my spouse write it (assuming I had a good relationship with my spouse).
- contravariant 8y agoAnother obvious choice would be someone at the New York Times. Even if we could figure out who then all we'd have discovered is dubious proof on someone who might know the identity of the source, which isn't terribly useful.
- andrewflnr 8y agoOccam's razor does not necessarily apply in cases where there is an intelligent agent actively trying to deceive you, by means including but not limited to taking advantage of your application of Occam's razor.
- seth_tr 8y agoAt the end of the article the author discusses this, quoting this tweet: https://twitter.com/dmimno/status/1037490608015925248 https://twitter.com/dmimno/status/1037490608015925248 "Now might be a good time to remind everyone that “distinctive phrases” and rare words (high TF-IDF) are not as good for stylometry as subtle differences like “and” vs “the” ratios. If you can easily notice it, someone can easily spoof it."
- emmelaich 8y agoThen again stylometry would be the thing you'd expect to change most between a tweet and an article published with style checkers and editors.
- ggm 8y agohere's a shout-out for stylometry: https://books.google.com/books/about/To_Couple_is_the_Custom.html?id=z8GzNQAACAAJ https://books.google.com/books/about/To_Couple_is_the_Custom... (obviously, biassed since I'm related)
- dickfickling 8y agoIs "'and' vs 'the'" ratios an actual thing? I tried searching for it, but as you can imagine "and vs the" does not return any useful results on google or ddg.
- minimaxir 8y agoThe trick behind NLP that's surprisingly under discussed is that you must use apples-to-apples comparisons between datasets. Using simple tweets as a proxy for style against a professionally-edited NYT op-ed is silly, but this analysis (along with the previous analyses) is transparent about that fact.
- resters 8y agoThis is obvious. So irritating that someone went to the trouble of doing the analysis on an inappropriate data set.
- jmkni 8y agoNot really, they are just showing off their library in a novel way. The author doesn’t even pretend that it’s going to produce an accurate result.
- colanderman 8y agoDo you mean "opaque"? "Transparent" usually means that someone is forthcoming, but I don't see your sentiment reflected in the article.
- everdev 8y agoThe OpEd reads more like a speech. I'd assume there are some publicly available speeches (or videos with subtitles) that can be analysed instead. Some articles mention that anonymous sources specifically take note of the speech patterns of other people so they can use those to throw off the trail. I'd be very surprised if the author didn't at least try to do that, or agree to have the NYT rewrite each sentence while preserving the meaning. It's an interesting exercise and can and might be able to rule out some authors, but it seems like someone harboring such opinions and wishing to express them to some extent (and has with others -- in the resistance) will be discovered in short order. Outing this person might be an even bigger story than the OpEd itself. But I put my money on old fashion journalism rather than text analysis.
- alex_anglin 8y ago
- whatshisface 8y agoIt's worth noting that Sec. Mike Pompeo used to be the director of the CIA and would probably know someone who could tell him about NLP and how to avoid it. The difference between identifying Shakespeare and identifying present-day politicians is that the politicians alive today have a chance of knowing about technology used today! https://en.wikipedia.org/wiki/Mike_Pompeo https://en.wikipedia.org/wiki/Mike_Pompeo
- resters 8y agoRight, a useful tool would be one that allowed an author to modify verbiage to mimic another author such that analysis like this would identify the wrong person.
- threeseed 8y agoNot sure if you've ever worked at a large company but that isn't how it works. CEOs don't know most of the people at the lower levels other than at company functions e.g. roadshow. They spend 99% of the time with their direct reports and the board. And even if Pompeo did know a Data Scientist who could tell him about NLP why would he ask them how to avoid being detected when writing a scathing op-ed piece ? You don't think that person would potentially leak ?
- williamscales 8y agoI'm not sure, perhaps if he had been presented with briefings containing evidence derived from these techniques he might be aware of their existence. At any rate, I would expect him to be familiar with the idea of analyzing a text to find a leaker. It just seems like he in particular would be more careful. But who knows? Speculating about this stuff is fun. You have some great points, remember how incompetent Manafort was?
- chx 8y agoYeah so a linguist already looked at this https://www.thedailybeast.com/this-linguist-helped-catch-the-unabomber-heres-his-take-on-the-new-york-times-resistance-op-ed https://www.thedailybeast.com/this-linguist-helped-catch-the... > “What’s been done so far is not scientific and simply silly,” he said. People. Don't get too drunk on your own kool-aid. Do I need to write an op ed that there are adults in the room :P ?
- mattnewton 8y agoThe author’s purpose is to demonstrate how to do this in R, and is upfront on how this is purely for academic exercise since he is comparing an op-ed to tweets. Much of the article is dedicated to analyzing how the analysis failed (it ends up recommending that Trump wrote it largely because of the presence of Russia in both that twitter account and the op-Ed) I still recommend giving the article a good scan.
- girvo 8y agoThis is a very quick dismissal, when the article itself mentions this! It's entire purpose is to show off their neat library, not to give a substantive prediction on who the author is.
- mcguire 8y agoI wonder if there are any ethical issues in outing someone using these techniques.
- minimaxir 8y agoAnalyzing writing patterns to determine the author has been a tool around for decades and predates statistical techniques like TF-IDF.
- deleted 8y ago[deleted]
- TheCoelacanth 8y agoHow is that relevant? Are old things automatically ethical?
- BurningFrog 8y agoWell, they're common knowledge. If I refuse to do something millions of others can easily do, it won't change anything.
- contravariant 8y agoIt's a bit like a 0-day if the proof is convincing enough. But publishing many dubious proofs that all point to different people isn't really much of an ethical issue.
- dingaling 8y agoNo more so than a 'news'-paper filling its pages with non-news. Op-ed is cheap, bulky and quick - but it's still not news.
- aphextron 8y agoThe op-ed was a plant, personally approved by Trump. This has literally been his standard MO for decades: plant fake stories and weaponize the press for his own use. And it works. He effectively ended the entire Manafort/Cohen news cycle overnight. Think about the plausibility of this story for even a moment. There's a list of less than 10 people that could have possibly written the op-ed, and what it amounts to legally is a written admission of high treason, a felony punishable by death. Do you seriously think any reasonable person would ever do that and assume they would get away with it? The entire administration has been one massive psy-ops campaign against the American public. Don't listen to a single word they say. Just watch their actions, and react with your own judgement accordingly.
- blazespin 8y agoIts a good theory .. it completely supports the deep state argumment and allows for Trump an excuse to root out moles. I can even see nyt editor falling for it to sell newspapers. Everyone wins, right. Amusingly, the analysis concluded that Trump was actually behind it. The whole thing is pretty stupid
- blazespin 8y agoFor the downvoters, you have to appreciate how stupid the whole is. Nothing was revealed we didn't already know. Republicans widely panned the entire thing. Trump has already talked about taking on the establishment, and that's who this is. He now has an excuse to fire if anyone doesn't do exactly what he asks.
- gamblor956 8y agoTrump could already fire people. This changes nothing.
- learc83 8y ago>and what it amounts to legally is a written admission of high treason, a felony punishable by death. Here is the legal definition of treason in the US (at the federal level). "Treason against the United States, shall consist only in levying War against them, or in adhering to their Enemies, giving them Aid and Comfort. No Person shall be convicted of Treason unless on the Testimony of two Witnesses to the same overt Act, or on Confession in open Court." Nothing in that letter comes close to fitting that definition. > He effectively ended the entire Manafort/Cohen news cycle overnight. The Manafort Cohen news cycle was already ended by the Woodward book preview. All this letter did was make the accusations in the book look even more plausible.
- dboreham 8y agoUnless "gutless anonymous" is a fool, they got someone else (e.g. NYT staffers) to write the actual text. Better to look at individuals who had recent physical contact with NYT senior level employees since they would certainly avoid telephone calls and email. Fairly likely the manifesto was written on a Remington or Underwood.
- willart4food 8y agoStep 1: write the Op-Ed Step 2: using google translate, translate the article from English to a different language, let's say German. Step 3: still using google translate, translate the German version into a different language, let's say French Step 4: translate from French back to English Step 5: have someone else re-edit it back into English
- pfarnsworth 8y agoOr, just use the same analysis that these bloggers used and then change your text to reshape the data. Anyone who is dealing with governments would know to watch out for exactly this, it's the first thing that people would watch out for.
- spacehome 8y agoIf you're that paranoid, why would you trust Google?
- badrabbit 8y agoBetter yet,analyze emotions and truthfulness using ML against video statements made by officials who deny they're the author.
- twblalock 8y agoJim Dao, who edits the op-ed page at the NYT, says that they did not do anything to alter the writing style: > He said there was no special effort to disguise the person's writing style, for example by rewriting the piece in some fashion. https://money.cnn.com/2018/09/05/media/new-york-times-resistance-oped-jim-dao/index.html https://money.cnn.com/2018/09/05/media/new-york-times-resist...
- vipulved 8y agoOne would assume that the author, who decided to publish a time-insensitive piece anonymously, was smart enough to employ language to intentionally misdirect analytical methods.
- jcmi 8y agoOne can assume, but I don't know if you watched the Twitter/Facebook congressional hearings last week but it doesn't seem like a high level of technical competency is required to work in government. Text attribution probably seems like magic to most folks without a stats background.
- coldcode 8y agoI think the editors likely worked the content over as well to mix multiple styles into it, which would make analysis like the OP difficult or impossible. If you know people will analyze content using common tools, you can too and keep mixing until its too fuzzy to be id'd.
- coldcode 8y agoThey say they didn't, but that doesn't mean the author didn't do it themselves.
- williamstein 8y agoThis situation reminds me a little of the Unabomber, who was finally caught entirely because somebody (his brother) simply recognized his writing style.
- alex_anglin 8y agoTed Kaczynskis writings were probably influenced a great deal by his mental illness, which would make him an outlier. In contrast, those in bureaucracies will likely tend towards milquetoast language over time, making recognition more difficult.
- mythrwy 8y ago1) Unabomber was caught based on content more than style (his brother recognized the theme and called it in). 2) Unabomber sent a very long screed to NYT, not 140 characters. 3) Unabomber had a very unique theme, not an anti-Trump screed which is quite common. Which, based on that alone there is really no telling who did this because of volume of interested parties. ("I'm a senior administration official"). Ya whatever. Guess it provides something for bored people to get frothed up about, but it may be about as real as Cinderella's Castle at a Disney World. Except not nearly as well done.
- sn41 8y agoI have a suggestion, which others may have thought of: can you randomize the style using programs? 1. You write an essay. 2. [Word-level randomization] An algorithm looks at it, as a first cut, replaces uncommon words with common equivalents. 3. [Sentence-level randomization] Changes each sentence to a randomly selected grammatically correct equivalent. 4. [Paragraph-level randomization] Adds spurious repetitions, replace some paragraphs with automated summarized versions. i.e. destroy the rhetorical effect, but get the message across so as to defeat NLP-based identification. If the content of the message is important, and the final version is okay to the author, this might be safer for anonymous disclosures.
- tmalsburg2 8y agoPerhaps it would be enough to just Google-translate the text to some other language and back?
- sn41 8y agoYes, translating different paragraphs into different languages and back might work.
- clickok 8y agoThat might defeat NLP-based analysis, but the use of obfuscation and the methods employed still tells you something about potential authors. If someone in an organization writes an essay disparaging that organization, but does so in an unnatural style that seems intended to defeat author identification, then you know that they're aware of stylometric analysis, and have created or downloaded a tool designed to interfere with assigning attribution. Depending on the organization, that might shrink the candidate pool more drastically than clustering based on sentence length, favorite words, or other linguistic quirks.
- village-idiot 8y agoI think if I planned to write a piece like this, I'd give an outline to a close confidant, such as my spouse, and have them write it. Assuming said confidant doesn't have a ton of public writing samples, this would frustrate any textual analysis.
- the_watcher 8y agoOr write an outline, have someone at the NYT write it, then send it back to you and edit it for clarity and accuracy.
- village-idiot 8y agoEither way.
- pawelmurias 8y agoSomeone someone at NYT could just writing it without the outline.
- wodenokoto 8y agoI did a end-of-semester project on stylometry on my masters, and no-where in our literature review did we see anyone comparing TF-IDF's scores for authorship attribution. Other than some guy on twitter saying TF-IDF, on what grounds is this classifier chosen? Other mention that "This is about showing off the library". In that case, the authors should chose to show case their tools for a problem they are ment to solve. "Check out my hammer, here's how to use it: First, you get some screws..."