4 ms·

Lead author here. Since my serious thinking on this topic started when I responded to this Ask HN post[1] Π years ago[2], it's nice to see this posted here, to

Lead author here. Since my serious thinking on this topic started when I responded to this Ask HN post[1] Π years ago[2], it's nice to see this posted here, to come full circle in a sense. Happy to answer any questions. [1] http://news.ycombinator.com/item?id=413730 http://news.ycombinator.com/item?id=413730 [2] No, really, it's been exactly Π years to the day :-)

saulrh 15y ago
Do you expect your system to scale easily to the "the entire internet" data set? You mention that you degrade gracefully as the problem size increases; is getting the accuracy up to a usable figure only a question of more signals and processing power and dirty ML tricks, or do you think that you need a fundamental improvement?
- randomwalker 15y ago
  The algorithm achieves significantly higher accuracy if it has more text per author. Also, if you're willing to do human analysis on a few dozen candidates after algorithmically shortlisting them, that gives you a further advantage. Finally, there is much room for straightforward algorithmic improvement (e.g., ensembles of classifiers) that we didn't have time to fully investigate. In short, IMO it's just a matter of more data and slightly better ML, not fundamental improvements.
  - saulrh 15y ago
    Pretty much what I figured. I look forward to seeing your future results! Or to seeing you to get shut down by the NSA. :P
loboman 15y ago
Well, not exactly, just 3.142 :) http://www.wolframalpha.com/input/?i=1147+days+to+years http://www.wolframalpha.com/input/?i=1147+days+to+years
- randomwalker 15y ago
  Sure, Π is irrational, so it can't be exact :) What I meant is that 1147 is the closest integer to Π*365, which IMO is still an awesome coincidence!
dstorrs 15y ago
I'm impressed by the skill that goes into this but it doesn't seem like an even-handed technology -- it empowers governments, major corporations, and other large organizations more than it does private individuals. As a specific example, people writing political blogs in China could be seriously harmed by this technique even at the levels that it's at now. I applaud you for including the link to "manually changing your writing style will defeat these attacks" but that's a link to an academic paper. Could you please also write some good, layperson-oriented docs on "how to beat this"? For that matter, I'll do the writing grunt work if you'll provide the expertise. If you're interested, use the GMail address in my profile.
- Jach 15y ago
  I'd also be interested in what anonymizing techniques come from this, but the way to specifically beat the Chinese government here is to carefully guard the border between your online presence and your offline government-issued identity. At minimal this means anyone in China should be encrypting everything before it leaves their machine and goes to the network through a connection with their name on it, and if they're in the business of writing anti-China political blogs, they need to treat everything they do publicly as a threat to their identity. You can't expect to carelessly leak your identity today and try to hide it tomorrow. You either don't leak your identity at all or what you're trying to keep anonymous has to be so low-bit in information content that it's probably worthless because it would be like everything else.
  - gtani 15y ago
    http://news.ycombinator.com/item?id=3582490 http://news.ycombinator.com/item?id=3582490
- randomwalker 15y ago
  That's a good question. First, I believe that intelligence agencies are already well aware of the potential of technology like this, and at least some, like the NSA, could very well be ahead of public research. Second, research such as ours is intended to demonstrate a proof of concept, and it takes a lot of work to turn it into a reliable tool — for example, we restrict ourselves to English text. For those two reasons, I think our work does little to directly help governments and other oppressive entities. On the other hand, publicly available research is effective (we hope) in raising awareness of the threat, so on balance it does more good than harm to people writing political blogs. As for practical tips to defeat stylometry and such, organizations like the EFF specialize in doing that, so I will leave that to them. Comparative advantage, etc. If you would like to help, you are more than welcome.
staunch 15y ago
I asked that question! I was going to post a link to that when I saw the headline here. Ask HN's are awesome but you don't usually get a PhD thesis in reply :-)
JesseAldridge 15y ago
So did you every figure out who onetimetoken was? http://news.ycombinator.com/item?id=1203836 http://news.ycombinator.com/item?id=1203836
robrenaud 15y ago
Is this really that hard to break? Imagine I want to post anonymously about some sensitive subject. Whenever I do it, I write the article normally, then Google translate from English to French to English, and then I clean up obvious errors in the retranslation.
- dhx 15y ago
  Practically that would work, at least for the foreseeable future. Theoretically it's not so easy. If you know how Google translates from English to French (and vice-versa) you can at least partially reverse the process. Or perhaps authorship detection can be performed on the output from Google translate? The authoriship entropy contained within the original English text is likely to be carried through (at least partially) to the post-translated output.
- DanBC 15y ago
  Some people could be tracked by their weird punctuation. I'm not sure how to deal with that.
DennisP 15y ago
Is your software and the dataset it uses publicly available? If so, perhaps it would be possible to use it to engineer countermeasures. In the example you used ("since" vs "because"), it would be fairly simple to alter the ratio. Of course I don't know what more complex indicators you may be using, but I'm having a hard time imagining what would be easy to measure, but difficult to alter.
neilk 15y ago
Great paper, glad to see the fruit of your efforts. I guess you did not look at texts with multiple authors, or professionally edited texts? I am curious if a different editor, or publication house style, can be detected.

saulrh 15y ago

Do you expect your system to scale easily to the "the entire internet" data set? You mention that you degrade gracefully as the problem size increases; is getting the accuracy up to a usable figure only a question of more signals and processing power and dirty ML tricks, or do you think that you need a fundamental improvement?

randomwalker 15y ago
The algorithm achieves significantly higher accuracy if it has more text per author. Also, if you're willing to do human analysis on a few dozen candidates after algorithmically shortlisting them, that gives you a further advantage. Finally, there is much room for straightforward algorithmic improvement (e.g., ensembles of classifiers) that we didn't have time to fully investigate. In short, IMO it's just a matter of more data and slightly better ML, not fundamental improvements.
- saulrh 15y ago
  Pretty much what I figured. I look forward to seeing your future results! Or to seeing you to get shut down by the NSA. :P

loboman 15y ago

Well, not exactly, just 3.142 :) http://www.wolframalpha.com/input/?i=1147+days+to+years http://www.wolframalpha.com/input/?i=1147+days+to+years

randomwalker 15y ago
Sure, Π is irrational, so it can't be exact :) What I meant is that 1147 is the closest integer to Π*365, which IMO is still an awesome coincidence!

dstorrs 15y ago

I'm impressed by the skill that goes into this but it doesn't seem like an even-handed technology -- it empowers governments, major corporations, and other large organizations more than it does private individuals. As a specific example, people writing political blogs in China could be seriously harmed by this technique even at the levels that it's at now. I applaud you for including the link to "manually changing your writing style will defeat these attacks" but that's a link to an academic paper. Could you please also write some good, layperson-oriented docs on "how to beat this"? For that matter, I'll do the writing grunt work if you'll provide the expertise. If you're interested, use the GMail address in my profile.

Jach 15y ago
I'd also be interested in what anonymizing techniques come from this, but the way to specifically beat the Chinese government here is to carefully guard the border between your online presence and your offline government-issued identity. At minimal this means anyone in China should be encrypting everything before it leaves their machine and goes to the network through a connection with their name on it, and if they're in the business of writing anti-China political blogs, they need to treat everything they do publicly as a threat to their identity. You can't expect to carelessly leak your identity today and try to hide it tomorrow. You either don't leak your identity at all or what you're trying to keep anonymous has to be so low-bit in information content that it's probably worthless because it would be like everything else.
- gtani 15y ago
  http://news.ycombinator.com/item?id=3582490 http://news.ycombinator.com/item?id=3582490
randomwalker 15y ago
That's a good question. First, I believe that intelligence agencies are already well aware of the potential of technology like this, and at least some, like the NSA, could very well be ahead of public research. Second, research such as ours is intended to demonstrate a proof of concept, and it takes a lot of work to turn it into a reliable tool — for example, we restrict ourselves to English text. For those two reasons, I think our work does little to directly help governments and other oppressive entities. On the other hand, publicly available research is effective (we hope) in raising awareness of the threat, so on balance it does more good than harm to people writing political blogs. As for practical tips to defeat stylometry and such, organizations like the EFF specialize in doing that, so I will leave that to them. Comparative advantage, etc. If you would like to help, you are more than welcome.

staunch 15y ago

I asked that question! I was going to post a link to that when I saw the headline here. Ask HN's are awesome but you don't usually get a PhD thesis in reply :-)

JesseAldridge 15y ago

So did you every figure out who onetimetoken was? http://news.ycombinator.com/item?id=1203836 http://news.ycombinator.com/item?id=1203836

robrenaud 15y ago

Is this really that hard to break? Imagine I want to post anonymously about some sensitive subject. Whenever I do it, I write the article normally, then Google translate from English to French to English, and then I clean up obvious errors in the retranslation.

dhx 15y ago
Practically that would work, at least for the foreseeable future. Theoretically it's not so easy. If you know how Google translates from English to French (and vice-versa) you can at least partially reverse the process. Or perhaps authorship detection can be performed on the output from Google translate? The authoriship entropy contained within the original English text is likely to be carried through (at least partially) to the post-translated output.
DanBC 15y ago
Some people could be tracked by their weird punctuation. I'm not sure how to deal with that.

DennisP 15y ago

Is your software and the dataset it uses publicly available? If so, perhaps it would be possible to use it to engineer countermeasures. In the example you used ("since" vs "because"), it would be fairly simple to alter the ratio. Of course I don't know what more complex indicators you may be using, but I'm having a hard time imagining what would be easy to measure, but difficult to alter.

neilk 15y ago

Great paper, glad to see the fruit of your efforts. I guess you did not look at texts with multiple authors, or professionally edited texts? I am curious if a different editor, or publication house style, can be detected.