4 ms·
One under-discussed component of Snowden's XKEYSCORE leak was the claim made in one of the presentations that the NSA has enough stylometry data collected that,
by px43 4y ago
One under-discussed component of Snowden's XKEYSCORE leak was the claim made in one of the presentations that the NSA has enough stylometry data collected that, give some piece of text, they can not only tell you who wrote it, but they can also tell you where they wrote it (in 2008).
Inferred from the "Document Tracking" slide about 2/3rds of the way in.
https://www.theguardian.com/world/interactive/2013/jul/31/nsa-xkeyscore-program-full-presentation https://www.theguardian.com/world/interactive/2013/jul/31/ns...
It will be interesting to see if any research projects start seriously looking into styleometry using LLMs, but that's basically what you'd need to do to actually generate something useful detection-wise. Ideally it would light up a document identifying the style of known humans, as well as the style of various LLMs. If teachers actually start cracking down on this sort of thing, I'm sure people will fine tune models on their own text, ensuring that anything written uses their personal style, but we can tackle that once we get there.
- mistymountains 4y agoYou could take a similar approach to detection (hypothetically). You provide writing samples a LLM is fine tuned on, then future texts could possibly be filtered by perplexity or similar to detect obvious differences in style. Of course, there would be side effects, and ideally we don’t start creating incentives where we don’t need to. Man, I’m glad I learned to write before all this.
- polishdude20 4y agoHow does it know where it was written?
- zx8080 4y agoGeo-ip? Joking.
- Atlas22 4y agoJust a guess, but it would not be surprising if there are variations in stylistic writing that are influenced by environment (location, people interacted with, air quality, etc). I have certainly observed it happen with myself based on who I've interacted with recently.
- px43 4y agoMy assumption was that it would first detect when it was written using stylometry, and the change in style over time, and could use that, in addition to other location tracking data in XKEYSCORE to figure out where the target was at that time. That might be giving them a bit too much credit. They could just be using some other simple heuristics to guess at location data. Hard to tell. XKEYSCORE is/was basically the NSA's Google, and in theory contains nearly all of the data ever collected from the various collection points all over the internet.
- sebzim4500 4y agoUnless I am misunderstanding the document, they aren't claiming to be able to read a document and tell you who wrote it. They claim they have a massive database of when documents have been shared and then they go back to find the original source that way. Like chainalysis for evil people.
- WalterBright 4y agoI'd probably use chatgpt to mask the text coming from me!