9 ms·
US Government building AI tech to unmask anonymous writers
- Minor49er 4y agoI wonder how this would fare against an astroturfing campaign that's using something like OpenAI[1] to generate human-readable text [1] https://beta.openai.com/examples https://beta.openai.com/examples
- no_identd 4y agoBUILDING? The NSA announced they have this tech and used it to identify the real Satoshi like half a decade ago lol
- egberts1 4y agoIt is only a matter of time before some group starts developing and then expanding a jumbler of incorrect grammars, useless punctuations and poorly substituted adverbs before posting … as a plug-in/service. Origination still remains the better datapoint class to analyze.
- cronix 4y agoungrammarly.com /s
- homonculus1 4y agoindi.an
- ravenstine 4y agoDamn it, someone owns that domain.
- deleted 4y ago[deleted]
- LinkLink 4y agoTechnology seems to create more arms races than solutions to problems these days.
- dylan604 4y agoHumans don't deserve the technology is classic sci-fi trope. Sadly, it's more art imitating life though. How long does it take for a society to be considered an ancient society? We've only not destroyed the planet with nuclear weapons for less than 100 years. That seems like such a baby step. Every bad thing we've done with tech since the atomic bomb has been misused at an even faster rate which seems to be gaining pace.
- donkarma 4y ago
- tartoran 4y agoIf they did so behind a backdrop it could yield some results. How long till anonynous writers run their text through some fuzzers or some other AI system to junble the style or emulate other’s styles, replace words with synonyms, change sentence structure and such ending up with AI systems fighting other AI systems in futility.
- starlust2 4y agoWhat you do is have your sister to re-write your posts and you re-write hers. No one will suspect you two are plotting to bring about the first Hegemon.
- rebelos 4y agoThis has been possible if not easy to accomplish for a very long time, so calling it "AI" seems like a bit of a stretch. I suspect most people can be fingerprinted according to a handful of less typical grammatical flourishes and reuse of certain uncommon words and turns of phrase. The only real challenge is scaling that up.
- acjohnson55 4y agoSure, but there's always room to advance the state of the art. And in the article, they mention using the adversarial AI approach, which I would say is firmly in the world of AI, and not just statistical analysis.
- EamonnMR 4y agoI've seen this as inevitable for a while, and that's why I'm writing under my real name right here right now.
- supermatou 4y ago> be me > be anonymous writer > be fearful of AI de-anonymizing my contrarian self > idea > write only greentexts, like other millions > AI can't single me out > AI defeated IFW
- throw10920 4y agoBeyond the meme, this seems to be a brilliant idea - just like how we have coding standards, why not make writing standards, several levels beyond what we have currently? Constrain the "syntax" of expression (the way you communicate, as opposed to the ideas you want to communicate) to counteract fingerprinting. Sure, you'd have to restrict your creativity and prose, but you'd get anonymity in return.
- omniglottal 4y agoAll natural languages have fundamental ambiguity built in. This increases "fitness" both for the languages and those using it. Precise disambiguation requires an ungodly amount of context which, notably, we don't all share between our distinct moral communities. This is a feature, not a bug. See (and practice) lojban if you intend to disambiguate. Next, just try to imagine prose or poetry or, hell, even a compelling speech or nuanced opinion...
- throw10920 4y agoI agree that disambiguation requires context and that this is a feature. Wouldn't that mean that you could write in a homogeneous, but still unambiguous, style by manually, laboriously, providing context longform, alongside your main text, written in the same homogenized style?
- stevewatson301 4y agoInstructions unclear, got a society whose language is Orwellian Newsspeak.
- throw10920 4y ago
- thenerdhead 4y agoBut what if those anonymous writers use AI to keep their writing anonymous?
- ravenstine 4y agoIf they're using the same AI for every anonymous identity, maybe that's still identifiable to an extent? But yeah. Hopefully any such AI used by the government can't be admissible as evidence in a court of law (assuming it's neural based and not a bunch of if statements), or at least is only used in pursuing federal crimes. But with the way things are going, who knows.
- thenerdhead 4y agoI was thinking of those writing AIs that rewrite whatever you write to a consistent voice. The ones that students are using to cheat in schools nowadays.
- ravenstine 4y agoI wonder how difficult that would need to be in practice. Obfuscation just has to be enough that it creates enough ambiguity. All you'd have to do is make things like basic punctuation, capitalization, rate of typos, advanced word count, etc. consistent. Those are things that can be "statically analyzed" from one's genuine writing and be changed on the fly without needing an AI to know the meaning of a body of text.
- pessimizer 4y agoIt's becoming a question that if you have any sort of beliefs that may not be looked upon kindly by this government or future governments, maybe you shouldn't be writing anything on the internet at all. Maybe you shouldn't be texting or writing emails, either. Over the past year, I've certainly been debating not expressing myself at all in writing. It's starting to feel very dangerous.
- gatonegro 4y agoExpressing certain thoughts is already dangerous in some contexts. Not state-actor-is-coming-to-get-you levels of danger just yet, at least for us common folks, but it's already happening and it ain't gonna get better.
- pessimizer 4y agoIt certainly is, but anonymity and pseudonymity tend to protect you unless you get some sort of celebrity or need to communicate for your job. Anyway, one expresses one's opinion for the sake of other people. Maybe instead of worrying about what other people think, I should concentrate on my own safety. I can do other things for people.
- 2OEH8eoCRo0 4y ago> if you have any sort of beliefs that may not be looked upon kindly by this government List them. Which beliefs? Edit: Not trying to "out" anybody's beliefs, I just don't think the govt cares.
- BitwiseFool 4y ago>"List them. Which beliefs?" I have a feeling the person you're asking won't want to answer this question!
- 2OEH8eoCRo0 4y agoI'm not asking for their beliefs, just an example.
- Randor 4y agoHmmm, Not sure how effective it will be against the latest NLP tools. You can also generate writings in the same style as others.
- tolmasky 4y agoOK, so before you publish something you give it to GPT-3 with the prompt "write the following in the style of $X". Run the resulting text through a few AIs to confirm it says that it was probably written by $X. So, either to anonymize the text or to implicate someone, it seems like we're just in an arms race now.
- MrWiffles 4y ago"Write $POLITICAL_OPPOSITION_PIECE in the style of Hemmingway". AI fall down go boom.
- dylan604 4y agoThen BigBrother starts getting logging info from the GPT-3 based SaaS you used to do that translation. Better to run your own AI from home. I hear Nvidia has a glut of inventory you can utilize.
- MrWiffles 4y agoI wonder if this will turn out to be something of an antipattern (for lack of a better term). The set of vocabulary in a given language is fairly limited by the human mind, enough such that it's relatively uniform enough that subtle alterations are achievable enough to the individual that they could conceivably, of their own volition, intentionally fool emergent AIs leading to false attribution. I mean, if the AI can alter the text subtly enough to fool other AIs, given the limited number of words known to other humans (which is kind of the point of speech - what use is a massive vocabulary if nobody else knows those words?), why couldn't humans do it too? So, here we have AIs erroneously pinning the author of heretical text X as human Y, thus triggering punishment Z. All the while, it was really written by bastard A, who is really a member of political party B: the opposition of human Y's party, the C's.
- yonaguska 4y agoEventually, we'll need an AI that we write through, that then anonymizes our writing to make it unidentifiable.
- wyldberry 4y agoDepends on who the audience is. If you're writing a persuasive political message, you will have to have your fingerprint on it to generate the human reaction you want. Although if we are at that point with AIs, it could also just be trying to game other AIs into interpreting writing into what you want before it's relayed to the individual.
- noncequitur 4y agothings like privacy protection don't apply to the Big Brother their psycopaths have untied hands, full control on citizens and "tools" to protect themselves against wrongthinkers
- abalaji 4y agoSurprised, stylometry is not mentioned once in the article. [1] Unmasking anonymous writers has a rich history, and there are approaches writers with a large corpus have taken to conceal their writing, as well as approaches folks have taken to deanonymize those same writers. [1] https://en.wikipedia.org/wiki/Stylometry https://en.wikipedia.org/wiki/Stylometry
- jerf 4y agoI find it absolutely, utterly, and completely implausible that the intelligence community just now started to research this possibility. It's been an obvious idea for decades now, and the tech has visibly obviously been roughly up to the task for decades as well. The last couple of years may have given an incremental improvement but it is completely implausible to me that they don't have workable solutions to this already. It's not like the system is useless to them until it 100% fingers exactly one person.
- 3pt14159 4y agoDon't believe it jerf. It's obvious that this isn't the cutting edge that USG has to offer. Just a random department re-inventing the wheel. Disclaimer: Worked on CIA / FBI stuff two decades ago.
- TheJoeMan 4y agoI know that someone in the intelligence community was giving a lecture at the University of Florida on using UF’s fancy new donated Nvidia cluster to do this analysis.
- gaitthe2ndface 4y agoThis sort of writing style analysis has absolutely been used to ID pseudonymous / anonymous individuals online for over a decade. The new twist seems to be using "AI" instead of more traditional algorithms. Personally, I would expect that to make the process more error-prone, but it makes sense to try.
- mvidal01 4y agoCant someone also build an AI to mask anonymous writers?
- mvidal01 4y agoCant someone also build an ai to unmask anonymous writers?
- happytoexplain 4y agoFor years I've tried to obfuscate my own style/word choices/idiosyncrasies when writing anonymously. Not because I fear that I'm saying things my government would like to punish me for, but because I fear these tools will become accessible enough for anybody who would like to punish me to identify and harass me.
- timmytokyo 4y agoIn the process, you've established another identifiable pattern.
- throw10920 4y agoYeah, you can't just change from one fingerprint to another - you either have to erase your fingerprint (such that you look identical to someone else), or generate random ones and cycle through them. The anti-web-tracking people have been wrestling with this one for a bit.
- happytoexplain 4y agoWell, the point is to keep that fingerprint isolated to anonymous accounts. I don't mind if anonymous accounts are linked. Luckily I don't write much outside HN, and even less under my real name. I am, however, not under the illusion that this practice makes me safe. It's just something I do out of nervousness, since the only other option is not to write, and I'd rather not give it up.
- mutt2016 4y agoI read about a child porn guy on the dark web once, he was an auzzie I think. He always said some strange greeting on every post. He would say hiyas. They found Facebook posts where some dude said hiyas, did some police work and got him. Found an article: https://www.bbc.com/news/uk-36437856 https://www.bbc.com/news/uk-36437856 This isn't quite the same, but validates the point you are making for sure.
- claudiulodro 4y agoEasy thwart: Run a document through a couple different languages in Google translate then back to English. The writing will be understandable if not especially elegant, and my understanding is it's like a hash: after a couple hops in Google Translate it's not possible to "un-translate" it back to the original source document. e.g. English -> Spanish -> Esparanto -> English won't yield the original document if you then run the translations in the reverse order, and it wouldn't really be possible to determine which languages it was run through in the first place using only the end result document. Of course, I'm sure the government could just get the Google Translate logs if they really wanted to . . .
- Ajedi32 4y agoLanguage models are becoming more accessible all the time. I wouldn't be surprised if there's a similar system out there right now that can be run locally on a consumer grade GPU.
- marktangotango 4y agoI think this comment is a good example of why such an AI would be nigh impossible to implement, at least in a way to be meaningful. When I read this comment, at a high level it's very original like something GPT-3 could create. But this comment has meaning that GPT-3 gibberish wouldn't. Basically I think I'm implying that such an AI would effectively be AGI.
- ravenstine 4y agoMost people won't use a language-mixer of sorts. Do you think Apple or The Google are going to provide this as a feature on their devices? Of course not. The vast majority of people wouldn't even care to want such a feature. The whole point is to use this technology against everyday people, not the criminals who might be savvy enough to obfuscate their identity.
- bolasanibk 4y agoJust translate the text through Google translate a few times. "I plan on not paying my taxes" becomes "I intend not to pay tax."
- seeraan 4y agoSatoshi here we come
- deleted 4y ago[deleted]
- wyldberry 4y agoThis certainly feels a natural extension from being able to do it off of code already. https://media.defcon.org/DEF%20CON%2026/DEF%20CON%2026%20presentations/DEFCON-26-Rachel-Greenstadt-and-Aylin-Caliskan-De-anonymizing-Programmers.pdf https://media.defcon.org/DEF%20CON%2026/DEF%20CON%2026%20pre...
- TacticalCoder 4y agoI do believe a sizeable percent of all blogs and comments on the big bad internet are already generated by AI / models / etc. Especially so for the anonymous ones. So it's an AI buddy that's going to unmask lots of its AI buddies. Also, as others have commented, it's going to be hilarious when the AI shall unmask comments written in the style of Obama or Trudeau. But, as always, it's amazing to see taxpayer dollars hard at useful work.
- itronitron 4y agoVarious research groups have been looking at computational methods for determining authorship attribution over the past thirty years. One of the more valuable applications has nothing to do with anything IARPA would care about. Unless an author is rehashing the same content over multiple publications then there isn't much that can be done to attribute writing to the same source, and even then if it's publicly available then attributing it to a singular person is dubious. A great way to freak people out though and potentially stifle freedom of speech.
- killjoywashere 4y agoYet shockingly there will be some other shrill complaints from the other side when the government fails to identify <future event> because, in retrospect, all the information was online. "They should have known!" There are all sorts of reasons work on this. Mental health (think force health protection). Intel. Counter-intel. Criminal investigations.
- MauroIksem 4y agoShocked they're not already doing this.
- ashwagary 4y agoThey probably already deanonymized a large % of the internet and are using this as an excuse to ramp up parallel construction.
- bradmcgo 4y agoSo will we live in a world where anonymous and pseudonymous writers hide behind AI generated translations of their original writing? Will this become a new cat and mouse game?
- amadeuspagel 4y agoThis is another reason I wish we had more anonymous forums - truly anonymous, not pseudonymous.