18 ms·
US Government plans to develop AI that can unmask anonymous writers
- hilbert42 4y agoIn my case and I suppose for most HN posters there'd be little point—for on demand Ycombinator would be compelled to hand over email and IP addresses to Government—as I'd reckon in most instances that'd be a much easier and faster way of obtaining relevant information. In an era when privacy has become hugely diminished under the hands of both governments and corporate interests it raises the question of what rights to anonymity anyone has in either a public or private forum, and at present there's little if any consensus on this which ought to signal that any such project is premature. Unlike yours truly—who usually speaks his mind irrespective of whether he's known to his audience or does so anonymously—many will not speak their minds out of fear of being ridiculed, or humiliated, or exposed, or out of the risk of offending—risking the breakup of a friendship, etc. Same goes for whistleblowers whose public utterances, if not done anonymously, usually costs them their jobs. If people fear that their autonomy to act in an anonymous manner has been removed then they're unlikely to act at all, silence being the better part of discretion. This would have huge negative repercussions for society, our institutions and our governance—after all, the secret ballot is one of the cornerstones of our democracies. If we're not careful AI could undermine the ballot by unmasking what users think or how they actually vote and it's not hard to see how this would lead to coercion thence totalitarian government. That said, in this world of widespread almost instant communications, actors who intentionally act out of bad faith can do widespread damage, especially so when they do so anonymously. Knowing who they are would minimize the damage they are able to cause. Similarly, in a distantly-related post on HN a few days ago I referred to the increasing loss of respect for our important institutions and for the way we're being governed and how I thought that faith could be restored. There, I suggested that as a part of that process we need to unmask the hidden processes of government and that this would also include the naming of those who originate policy, law, etc.: "If we're to restore any faith in our governance then this protection [hiding originators of policy] must stop. Decisions made by government employees must be open to public scrutiny, similarly, the origins of government policy—laws, regulations etc.—must be traceable back to its source (those who initiated said policies). Systems without accountability will always become corrupt." Thus, there's a real dichotomy at work here. For some things anonymity is essential, at other times it's a curse. And from the many recent instances of where the gnomes within government haven't acted in our best interests then I'm damned sure that putting AI to work here won't bode well for us either. I've little doubt that the technology will be abused, and by virtue of the fact it will automatically silence a large proportion of the population who need speak out and who should do so anonymously in the interests of all. Even if they aren't targeted directly just knowing that there are systems in place that have the potential to expose them would be sufficient to silence many—as AI analysis of their words could be used to determine their identity at any future time (living with ongoing stress from potential exposure of one's ID would likely be unbearable for some). Given past history and current bad behavior of governments in these areas, I do not believe that it is possible to put such a system in place that would gain the full confidence of all players involved. It would have to have sufficient protections locked in place to provide full public accountability as well as having inbuilt mechanisms that would ensure the system could not be abused by governments. At present, such conditions cannot be realistically met—not by a long shot. Before anyone or any entity could let AI loose on this project and simultaneously state with all honesty that sufficient protections were in place for the project to proceed with safety would require many other prerequisite protections and 'safety measures'—which currently do not exist—to be incorporated (locked) into our governance. For instance, a whole raft definitions and concomitant laws pertaining to privacy are needed—and that's just for starters. No doubt this project will proceed without those prerequisite protections, ipso facto, it will also be abused. PS: note my quoted point about government policy etc. being open to public scrutiny. Here such questions arise such as where did this idea originate, what are the names of its instigators and what are their motives for instigating this development—not to mention others such as what are their qualifications, experience, etc. (perhaps, given the enormous potential of this AI application to damage society, we may even need to pose questions concerning their political beliefs and allegiances). It's no accident that this information is missing with this announcement.
- egao1980 4y ago
- michaelwww 4y agoI'm old so I've been planning on this for awhile. I have a folder that contains all my personal data: photos and videos, journal, all my saved social media posts, all my emails and all my anonymous handles leading to everything I've ever written online. My thinking is an AI could create a reasonable facsimile of myself that my descendants could have a conversation with. I think it'd be better than a autobiography since Joyce Carol Oates convinced me by something she tweeted that no one reads autobiographies, not even close family, unless you are famous.
- oneoff786 4y agoIf I had a bot that replicated my great great ancestor I’d probably get bored quickly and then try to prod it into revealing it’s deeply outdated and inappropriate social views
- michaelwww 4y agoYou're right, the novelty would wear off quickly, but it doesn't hurt anything for me to organize a data set about myself just in case
- eftychis 4y agohttps://news.ycombinator.com/item?id=33037319 https://news.ycombinator.com/item?id=33037319 https://news.ycombinator.com/item?id=33034918 https://news.ycombinator.com/item?id=33034918 Am I the only one seeing the irony and contradiction here? Not the same people at all -- subgroups at best , but the Director of National Intelligence is part of the administration. Perhaps I am missing something -- feel free to comment -- I am curious what everyone thinks.
- bitL 4y ago"Hey Joe, run that article of yours through the anonymizing AI first!"
- dathinab 4y agoWriters plan to develop AI which makes their text un-umaskable.
- ta988 4y agoThat's indeed the best defense. But the purpose of that one is to catch the easy ones.
- Animats 4y agoThat was working 20-30 years ago.[1] But not at scale. Mostly it was used on literary works. [1] https://en.wikipedia.org/wiki/Stylometry https://en.wikipedia.org/wiki/Stylometry
- patcon 4y agoThere's a tool called Anonymouth, written by folks at Drexel's PSAL lab, which is intended to help writers with anti-stylometry analysis. If anyone is looking for an impactful project, Tails OS maintainers and it's author academics seem receptive to bringing it onto that privacy-minded platform: https://gitlab.tails.boum.org/tails/tails/-/issues/5726 https://gitlab.tails.boum.org/tails/tails/-/issues/5726 https://github.com/psal/anonymouth/issues/6 https://github.com/psal/anonymouth/issues/6 https://github.com/spencermwoo/anonymouth https://github.com/spencermwoo/anonymouth
- ilamont 4y agoI suspect it’s how Fake Steve Jobs was unmasked 15 years ago: The New York Times found Mr. Lyons by looking for writers who fit those two criteria, and then by comparing the writing of “Fake Steve” to a blog Mr. Lyons writes in his own name, called Floating Point https://www.nytimes.com/2007/08/06/technology/06steve.html https://www.nytimes.com/2007/08/06/technology/06steve.html
- daniel-cussen 4y ago
- jamiek88 4y ago
- daniel-cussen 4y ago
- Waterluvian 4y agoCan’t I just feed my missive into an AI that reshapes the language but retains the data?
- bergenty 4y agoSure but your output will probably have the same signature as well unless you keep updating the model each time. It’s already hard to do the first thing, almost no one is going to do step 2 for a comment on the internet.
- jgalt212 4y agoI wonder if we'll finally find out if Harper Lee or Truman Capote wrote To Kill a Mockingbird.
- dvhh 4y agodidn't South Park already made a joke about it
- astrange 4y agoIf it's an actual ML model then someone can leak it and develop countermeasures. If it's a contractor who claims to be AI but is actually some guy in a room in Maryland, that's not possible but also the results wouldn't be very reliable. Of course, the article just says they're planning to do this. It doesn't say it's going to work, and our closest examples, forensic science like handwriting and blood spatters and polygraphs and all that, generally don't actually work.
- club_tropical 4y agoPresumably they can already unmask 99.99% of high-value persons of interest already, this sounds like they want a tool to unmask all the lower-value anons for the journalists to pick off.
- bergenty 4y agoI hope they open source it.
- club_tropical 4y agoI always wondered about voice deanonymization as well. I feel like between accent, gap between words, tonal variation, and rhythm, there's enough there to fingerprint speech, even with voice changers.
- c7b 4y ago
- fareesh 4y agoThe US inners bossmang want unmask but we know better sasa ke for beltalowda Perhaps it is time for Creole
- RadixDLT 4y agoI think they will secretly use this service https://www.typingdna.com/ https://www.typingdna.com/
- culi 4y agoWouldn't it be comparatively trivial to train an AI that can minimally modify your writing to be immune to this other AI? Basically a GAN
- PointyFluff 4y agoThey already do this, have since at least WWII. It's known as your "hand".
- godmode2019 4y agoIBM had this 7 years ago
- whitexn--g28h 4y agoSo South Park episode?
- golem14 4y agoOf course, there now is a business opportunity to make style obfuscating AI transformers ...
- xbar 4y agoThey come for us.
- submeta 4y agoBeing able to be anonymous is actually a valuable thing. Think of people in oppressive regimes. I believe we need countermeasures, so we need to create algoriths that add fuzzyness to our writing to protect privacy.
- spoonjim 4y agoThis already exists. There was actually someone who posted something on HN that would use your public profile to find your anonymous Reddit username and it found mine. That’s when I stopped anonymously using Reddit.
- meltyness 4y agoAs long as it doesn't run a foul of reasonable expectation of privacy.
- StanislavPetrov 4y agoWhat's almost as bad as the increased powers of government surveillance is thinking about the people who will be jailed, waterboarded and/or shipped to Gitmo for horrific tortures because our benevolent overseers in government make a mistake and think they are someone else due to the flawed algorithm fingering the wrong person.
- enviclash 4y agoZipf-based maths could already say lots.
- graderjs 4y agoSurely folks already have this? Authorship attribution has been a thing in academia for at least 10 years Tho I guess I could just use gpt3 and say hey can you write this for me in your own words or summarize. Or a few rounds of google translate roulette
- paulpauper 4y agoOne could possibly insert random invisible characters to fool this
- allisdust 4y agoThis seems to be a particularly bad time to implement this dumb idea. If it does become successful and prevalent, who do they think is going to be the writers apart from gpt3 clones.
- black_puppydog 4y agoAs someone who expects this to have happened years ago (maybe not under the moniker of AI but who cares) I'm more shocked by the fact that they'd publically announce this. The chilling effects of this will be all too real. If this works it's pretty much the equivalent of a mandatory state ID on every online interaction. If it doesn't work very well, then it's going to be that, plus the risk of randomly being flagged. As a society I don't think we have anything to gain from it; it's certainly tech that's put of Pandora's box, but that doesn't mean we shouldn't use all our cultural/legal means to prohibit/control it. Anyone in their basement could build this today, but it only becomes a problem if that person has control over police and intelligence forces. Which means the potential for good regulation is bigger than with other tech. Edited to add point about regulation
- vintermann 4y ago> The chilling effects of this will be all too real. Maybe that's the point. It could also be announced because they plan to use it publicly soon, to "prove" that some person they want to get for political reasons is the same as some evil terrorist/pedophile/serial killer.
- ergodic1 4y agoWon't there be a countermeasure? A grammarly for privacy that takes your intent and outputs an un-anonymizable text?
- zimpenfish 4y agoWouldn't surprise me if, eventually, it turned out that things like Grammarly were being used to train it in the first place.
- Mordisquitos 4y agoOne relatively straightforward countermeasure would be to feed your text through multiple rounds of machine translation using DeepL, Google Translate, and other similar services and applying corrections when necessary. That should break any personally identifying features in your original text. That of course has its own risks, if you assume full government access to providers and unlimited surveillance capabilities. If you are fully paranoid you could even run your text through a local instance of (less accurate) Apertium[0] for the first few rounds, or even for the whole process if you find the result is different enough from your original. [0] https://www.apertium.org/ https://www.apertium.org/
- achenatx 4y agoi thought they had done this with the federalist papers to determine which founding fathers wrote which papers (1996) https://www.jstor.org/stable/30204514#:~:text=They%20were%20written%20in%201787,under%20the%20name%20%22Publius%22 https://www.jstor.org/stable/30204514#:~:text=They%20were%20.... This is 2017 https://towardsdatascience.com/hamilton-a-text-analysis-of-the-federalist-papers-e64cb1764fbf https://towardsdatascience.com/hamilton-a-text-analysis-of-t...
- jongjong 4y agoI'm sure this has already existed for years. The AI technology needed to implement this has exited for years. You just need to do web crawling and keyword and link matching. You don't even need a 'writing fingerprint', you just need to parse comments which reveal identifying information such as 'I participated in project x', 'I taught at university y', 'I invested in startup z'... then when you combine all the identifying information, you can narrow down the pool of possible matches to a single person. You could probably do it with just basic text matching, no AI required.
- iinnPP 4y agoI don't go around as an anonymous user and post a bunch of identification information.
- bongoman37 4y ago
- gonzo41 4y agoOk, so would the simplest way to solve this be an open source AI that can normalize specific text / writing to force it to be too similar to all other text so it's less identifiable. Such as, you write a comment or an essay, feed it in and it just dumps all your styles, idioms etc and makes all your stuff sound bland and normal. Seems like it would only work if you had a targeted population and a writing / style sample ffor a lot of people.
- iinnPP 4y agoIf you can use this to identify anonymous people, you can similarly use a tool to change your own text to appear as though someone else typed it. Then you get to accuse anyone of anything.
- janejeon 4y agoHang on, is this rooted in the same technique (obviously they didn't have "AI"/technological processes like this back then) the FBI used for Ted Kaczynski (in which they literally analyzed his text to figure out who he was)?
- midmagico 4y agoWell, luckily it turns out that literally every person who attempts it can fool humans' best stylometric fingerprinting, so, meh?
- IYasha 4y agoPlot twist: anonymous writer is also AI.
- pabs3 4y agoSome other posts about Stylometry: https://www1.icsi.berkeley.edu/~sadia/papers/oakland2014-underground.pdf https://www1.icsi.berkeley.edu/~sadia/papers/oakland2014-und... https://news.ycombinator.com/item?id=18328270 https://news.ycombinator.com/item?id=18328270 https://serhack.me/articles/unveiling-anonymous-author-stylometry-techniques/ https://serhack.me/articles/unveiling-anonymous-author-stylo... https://news.ycombinator.com/item?id=30571932 https://news.ycombinator.com/item?id=30571932
- ezekg 4y agoDon't talk bad about your government, folks. We're going to be entering a new age of technological tyranny.
- imglorp 4y agoThe government appears more worried about managing dissent than anything else, like what actual threats people are talking about.
- alexbiet 4y ago@ezekg, GovAI has detected unlawful talk posted from your account. Your CBDC account is locked for 48 hours.
- brippalcharrid 4y agoFurther violations will lead to the balances of your close friends and family being adjusted by -20%, and the balances of acquaintances being adjusted by -5%. Help protect against the threat of misinformation and safeguard your Balance for up to 28 days by reporting anything that you think could lead to harm. Remember, We're All In This Together.
- Psychoshy_bc1q 4y agobitcoin fixes this.
- ezekg 4y ago> We're All In This Together. Sent chills down my spine.
- dmix 4y agoMove along citizen.
- orangepurple 4y ago@ezekg, GovAI has detected that your post violates community guidelines. Your COVID pass is RED for 72 hours to protect yourself and others.
- LinuxBender 4y agoHere [1] is a previous discussion on this as well. [1] - https://news.ycombinator.com/item?id=33009545 https://news.ycombinator.com/item?id=33009545
- unethical_ban 4y agoSo we'll have AI that can mask writer's identity in about three months.
- lm28469 4y agoText to text AI, it should be relatively easy to do too
- MengerSponge 4y agoBasically exists already, right? A lot of college kids are using it for their writing assignments.
- ezekg 4y agoIt'll be AIs all the way down.
- DharmaPolice 4y agoIf precision isn't super important you can already run your text through machine translation into another language and then back again. Spammy content sites already seem to do that to avoid copyright detection.
- yamtaddle 4y agoBut it'll erase all my anachronistic grammatical and style preferences! Then what's even the point of writing? (it's rôle and &c., goddamnit)
- Kukumber 4y agoThey demonized China for doing things like that Now they'll copy China I find it very funny
- mike_hock 4y ago> They demonized China for doing things like that > Now they'll copy China Rinse and repeat.
- xani_ 4y ago"How dare they do it before us" USA next decade: "Posting bad things on twitter reduces your credit score"
- hardnose 4y agoCredit scores are not determined by the government. If a credit ratings agency took that step, it would harm them because Twitter posts are unlikely to represent a meaningful variable when predicting someone's creditworthiness. Don't confuse that with "social credit" systems, whereby China prevents you from riding trains if you say something naughty.
- wahnfrieden 4y ago(citation needed - that’s not actually implemented in china at scale, though it’s a convenient talking point in the west)
- challenger-derp 4y ago> that's not actually implemented That's my understanding as well. > it's a convenient talking point in the west When online comments get worked up about The Social Credit System, a key thing I believe they're trying to do is spread awareness of how disturbing it is that a government is even considering such a thing that, as we understand it, is closely related to being a core technology in an authoritarian dystopia. While it's not implemented at scale, the unnerving fact is that govt policy makers did a careful enough take on a social credit system to decide that it was worthwhile investing (probably non-trivial amounts of ) money and resources into exploring it and did eventually reach a point where they were a handful of steps short of wide-scale implementation.
- dehrmann 4y agoPlans? I assumed lots of people were already working on this. There's already a lot of training data out there, and I suspect most users can be identified by use of a handful of uncommon trigrams and sentence stats. I know you can recognize things I've written at work because they have real em dashes—people rarely type with those.
- RobRivera 4y agoI'm a big counter intelligence man, so I love deliberately inserting randomness into potential identification metrics. In the words of martha stewart, it's a good thing
- boomboomsubban 4y agoYou probably don't want to associate yourself with a convicted criminal as a deflection technique.
- matheusmoreira 4y agoI wonder if there is a way to do that automatically.
- RobRivera 4y agorng shuffling of style and library usages, paradigms. and naming convention. camelCase vs NOCAMELCASE based on a coinflip, etc. leverage Lint static analyzer tools to enforce prior to submission. lot of fun ways to stick it to those bulk surveillance data lakes that may be the targets of such training sets
- causi 4y agoWouldn't it be more accurate to say it links anonymous writings together? If you don't have any writings under your real name there's nothing it can do except indicate two pseudonyms belong to the same person.
- andy_ppp 4y agoYou’re assuming they don’t have copies of every email you ever sent…
- ortusdux 4y agoI wonder how things like Gmail's smart auto-complete would affect these efforts.
- xani_ 4y agoI'm sure it will not be used in malicious way
- swayvil 4y agoWhat we need is a speaking style anonymizer. Like a language translator except it translates your text into some kind of stylistic uniformness. We're flexible on that format. Anything legible and relatively easy to translate into. Call it ANONSPEAK. It will probably be, aesthetically, horrible.
- __jambo 4y agoSeems very doable given the state of google translate. Trouble is if you are a revolutionary leader of some kind you are probably going to be saying new things that no-one else talks about - which renders both anonspeak and the ai detection kind of redundant. I guess the application for this then is in the interim to stop people or online groups becoming revolutionary by tracking and deradicalising them with targeted manipulation.
- hoosieree 4y agoThere exists such a practice, and it is known as academic publishing. Every verb is done in passive voice, punctuation is added - wherever possible - to make sentences appear more complex than they need be, and of course there is an effervescent use of sesquipedalian terms where shorter similes would otherwise suffice.
- lwneal 4y agoThe best protection against this type of de-anonymization is to take measures now, while you still have time, to prevent it. It is possible to change the style of one's writing by using a language model which alters the original text in order to create a new piece with a different style. For example, to translate your text into the grandiose and flowing diction of a bygone era, you might consider the project below. [1] https://github.com/lwneal/victorianhackernews https://github.com/lwneal/victorianhackernews
- doliveira 4y agoAren't GANs all about creating both the generator and the discriminator? Seems to me you can also build the "reverser" quite easily.
- ok_dad 4y agoI someday hope for style transfer for language, so I can write a paragraph and convert it to Michael Creighton's writing style.
- KennyBlanken 4y agoKennyblanken opens up the fifteen inches of aluminum that is his aging Macintosh Book Pro, produced by a company laughably named after a piece of fruit and formerly headed by a brilliant but somewhat psychopathetic man, now long pushing up the daisies, and taps out the commands to login to his account on the antiquated and increasingly inaccurately named hackernews, bathing in the yellow-orange theme as he stretches across the pale green couch in his small apartment in the big city. Outside, a 2007 Neopolitan Flyer bus roars by, half full with tired commuters. He expertly massages the worn keys to type out a lengthy and witty response about how he'd imitate the style of a famous novelist whose prose exists mostly to result in the sacrifice of as many innocent trees as possible - demonstrating how to stretch "I'd use Dan Brown's writing style" into nearly two full paragraphs of text.
- zimpenfish 4y agoTo be a proper Dan Brown, you'd need to start with "Internet Commentator KennyBlanken" (or some other prefixed descriptor.) cf [1] "I think what enabled the first word to tip me off that I was about to spend a number of hours in the company of one of the worst prose stylists in the history of literature was this. Putting curriculum vitae details into complex modifiers on proper names or definite descriptions is what you do in journalistic stories about deaths; you just don't do it in describing an event in a narrative." [1] http://itre.cis.upenn.edu/~myl/languagelog/archives/000844.html http://itre.cis.upenn.edu/~myl/languagelog/archives/000844.h...
- throwamon 4y agoDidn't they already use something like this to supposedly unmask Satoshi Nakamoto?
- Grimburger 4y agoNakamoto Satoshi has not been unmasked despite the very short list of people capable/interested in creating what he made. Stylometric analysis did suggest a single person on that list. The easier thing for governments to do at the time would have been to just spin up a node in the first year and look at the IP addresses. He had no desire to become known back then and likely never will. It's only more dangerous now compared to the threat before of being locked up like the LibertyCoin guy (who just got released a year ago). NS is happy to stay in the shadows, nearly everyone respects that decision especially in a world of crypto scams and ponzis. Surprised they never linked the domain name purchase to him though.
- daniel-cussen 4y agoThis is why I write my shit under my own legal name, even going to notarize this account at some point. Yeah throwaway yeah. Anonymous speech. Oh yeah darknet, Tor, cryptography, like yes sometimes, but it's a game of cat and mouse, it's purely a question of cost. Furthermore I consider games like poker or Magic the Gathering unplayable, that is the extent to which there is literally absolutely no privacy. But don't mind me, just got lobotomized is all.
- blakesterz 4y agoThere's a great book on this kind of thing, Author Unknown: On the Trail of Anonymous, by Don Foster. This was written way back in 2000. People have been doing this for decades.
- runjake 4y ago> People have been doing this for decades. The key point here is that it's AI-driven and at scale -- in other words, mass surveillance. Personally, I see this as part of the US IC's mission, despite the potential domestic detriment.
- Zigurd 4y agoMoreover, a stylometry analysis will reveal when I started to limit myself to one "moreover" for every two or three chapters.
- 1970-01-01 4y agoVoynich manuscript ---> US Govt AI stylometry machine ---> 42
- PointA2B 4y agoContent marketers in the digital marketing space commonly put blog posts through "spinners" that take your text and modify it through replacing words/phrases with similar equivalents. This lets you take one article and turning it into 5-10+ unique ones, even though they still discuss the same things. It would be a shame if a service like this was marketed towards those interested in privacy, it would probably break this entire system...
- ElementaryElk 4y agoI’ve found plenty of articles that seem to be run through these spinners, but hand made corrections are likely to be necessary (unless it can be automated with ML for example) as you can almost always tell that something is odd based on context lacking word choices
- PointA2B 4y agoAnd thats exactly what newer programs do, look at AppSumo and its practically all of them. The older gen simply used a giant dictionary, then picked a random option from the list of acceptable choices.
- ElementaryElk 4y agoOk, I wasn’t familiar with the newer gen of spinners
- frozenlettuce 4y agoThe thing is, AI is a good mask for a "backend process that you don't need to explain how it works". Assuming that the US government already has private conversations on multiple content and messaging platforms, this AI will provide the perfect excuse to connecting a blog post with a given id in a process.
- philipkglass 4y agoMy first cynical take was that this will be used for "hunch laundering." There could be no indication that user A is an alias for user B, other than someone's hunch, but getting a computer to say that they match might be good enough to get a warrant when someone's hunch wouldn't be. It would be similar to having drug sniffing dogs affirm their handlers' feelings.
- vmoore 4y agoIf I want to write anonymously, I cycle my text through Google Translate multiple times and keep all the grammatical errors intact. So, English > Italian, and then Italian > French, then back to English. I also pass it into Hemingway[0] first to make my text lean and non-superfluous. [0] https://hemingwayapp.com/ https://hemingwayapp.com/
- tenebrisalietum 4y agoFail, now Google and its friends and AI models have a copy of all your original text as well as the new text.
- club_tropical 4y agoAre you satisfied with the level it preserves your "voice"? Humor, style, metaphors, etc?
- challenger-derp 4y ago^ In case anyone is interested in learning more, the first part of what vmoore is doing is termed back-translation.
- glitchc 4y agoWe can get around this by writing something and sending it through GPT-3 for “style correction”.
- hoosieree 4y agoCreepy factor aside, a similar tool for attribution would be very useful for content creators (or copyright holders) currently worried about stable diffusion.
- Zak 4y agoI built something like that more than a decade ago to identify alter-egos in an online game from in-game chat. It was reasonably successful and I thought about commercial applications for it, but ultimately decided most things that could be used for are creepy or evil. I remember hearing DARPA was actively seeking research in the field around that time. In principle, I'm not absolutely against my software being part of the chain of events that leads to the decision to kill someone, but I don't trust the US government (or, realistically, anybody else) to independently verify an identification made by such a system. I'd be surprised if the three letter agencies aren't using something at least as good as what I wrote by now.
- copperx 4y agoHow sophisticated was your system? It sounds like you used cutting edge NLP techniques at the time.
- Zak 4y agoI'd describe it as fairly simple. It was just a classifier where each account name was a category: there was no fancy NLP. It used a single feature type and an algorithm from a well-known family. I don't want to say what either was lest I further proliferate the technique. I cross checked using statistically improbable words, which helped confirm or exclude weak matches.
- narrator 4y agoHow about using this to find bot accounts?