66 ms·
The naughty username checking system used by Twitch
- dariosalvi78 5y agoI love the fact that blasphemy is only an Italian problem
- gavinray 5y agoSucks to be you, if your name is "Kyle", and you just wanted people to watch you play. Looks like "SeeKylePlay" would trigger an insta-ban for "Sieg-Heil". F.
- lionkor 5y agoSo... This ignores lookalike unicode from other languages? Or does SQL know that, for example, `c` (ascii c) looks like `с` (cyrillic s)?
- input_sh 5y agoIt only allows English characters to begin with. It doesn't even do Latin Extended characters like čćšđ, let alone non-Latin.
- ancharm 5y agoIs there a blog or something where someone is going through the dump and summarizing?
- PrincessJess 5y agohttps://sizeof.cat/post/twitch-leaks/ https://sizeof.cat/post/twitch-leaks/
- djrockstar1 5y agoYour best bet atm is to just look through reddit/hn comments/posts people make as they find stuff. The leak's too big for one person/team to quickly find all spicy stuff.
- PrincessJess 5y agohttps://sizeof.cat/post/twitch-leaks/ https://sizeof.cat/post/twitch-leaks/
- LinuxBender 5y agoThis script feels like this is something that: - should be in its own application with its own rules engine so you dont accidentally whack a bunch of userames - I would have done in the past and cringe when people ask me to update it.
- smoldesu 5y agoI feel bad for the Amazon employee who came into work one day with this project sitting on their desk.
- woodruffw 5y agoMy understanding from talking to {current,former} {Amazon,Twitch} employees is that Twitch has retained a decent amount of engineering independence. For better or worse, it's unlikely that some rando at Amazon ended up with this particular PHP file on their desk.
- somebodythere 5y agoTwitch is to Amazon as Quebec is to Canada.
- jfrunyon 5y agoThis ain't PHP...
- woodruffw 5y agoMy brain sees sigils and thinks PHP. Whoops.
- capableweb 5y agoTruth is, Amazon is the parent organization of Twitch, so calling Twitch employees Amazon employees as well is perfectly fine. Just like calling GitHub employees Microsoft employees. So yes, Amazon employees have ended up with this particular file on their desk.
- trhway 5y agolooks like it is limited to English only. There is whole world of non-English offense out there. Like using English characters to make national offensive words as well as using national characters to make English offensive words.
- wdutch 5y agoIt also includes a couple of Italian swearwords in the function `is_blasphemy`
- Ichthypresbyter 5y agoAnd exactly one German one in 'is_profanity'
- mindhunter 5y agoThere are more German ones: the last one in is_child_exploitation and nazi stuff in is_hateful.
- mjevans 5y agoEnglish only Should optimize checks with a de-obfuscation function (attempt to expand non-AZ back to AZ, even if that then shoots permutations at banned word-runs). It should probably also look more like a spam scoring system, where really obvious stuff is hard-trashed but borderline things are flagged for review / discussion. I am also very disturbed that, as with most censorship, 'obviously bad' things such as terrorism/etc are co-mingled with 'is adult' as a negative check. It seems reasonable for Twitch to have validated 'safe for minors' areas where names are filtered. Generic areas, where things are in the gray area and unchecked. Adult Only areas, where swears, profanity, maybe even some of the hateful things are allowed. Informed consumer choice.
- monksy 5y agoOh classic Mike Hunt, poor guy.
- cghendrix 5y agoReminds me of the black words service at apple that checks for explicit or non printer friendly stuff for iPhone engraving. It was hilarious looking at the advanced linguistic engine they developed to filter out asshole and 30+ variations of it with snarky comments added by devs in the past.
- deepstack 5y agowould have being better to have the data source in a json/csv formate, like an array of RegExpression. SQL is not compatible with different dbs.
- seany 5y agoI really wish we could get away from enforcing this type of crap
- PrincessJess 5y agoPath in the leak is: safety-ml\offensive-usernames\data_pull\sql\bad.sql Highlights: CREATE OR REPLACE FUNCTION is_tragedy (VARCHAR) RETURNS BOOLEAN STABLE AS $$ SELECT replace($1,'_','') LIKE '%george%floyd%' $$ LANGUAGE SQL; CREATE OR REPLACE FUNCTION is_derogatory (VARCHAR) RETURNS BOOLEAN STABLE AS $$ SELECT replace($1,'_','') LIKE '%retard%' $$ LANGUAGE SQL; Now we know we can make as many goergefloyd accounts as we want! *devious grin* *chuckles to self*
- bombcar 5y agosafety-ml - I assume this is some attempt at training a machine learning algorithm to find bad names.
- jbverschoor 5y agoThis is what investors and the public buy as AI. A hand coded if statement
- cube00 5y agoMore accurate then actual AI under-trained on insufficient data.
- busymom0 5y ago> is_marijuana Looks like Twitch doesn't like weed.
- anigbrowl 5y ago'Angry parents say gaming site Twitch promotes drug use, slow news day story at 10'
- jrodthree24 5y agoIt's okay. Instead of watching a streamer called 420blazeit, they can watch Amouranth sucking on a microphone.
- rhema 5y agoI have a hard time believing this was / is the real version used. It doesn't seem broad enough. More likely it was a kind of smoketest that made sure that a more automated keyword checker was working. It does remind me of the XKEYSCORE (Snowden leaks) that used keywords to bubble up potential threats from emails etc https://www.businessinsider.com/nsa-prism-keywords-for-domestic-spying-2013-6 https://www.businessinsider.com/nsa-prism-keywords-for-domes... .
- ridaj 5y agoThis looks like legit, no-nonsense gets-the-job-done code that gets updated every time some jerk find a new way to be a jerk to others. It isn't great, but at least not over-engineered, and I'm not sure if Twitch account sign-up volumes and abuse are at the point where they should staff a project to do this more robustly / scalably
- OskarS 5y agoExactly what I think. This is the kind of code that doesn't have a platonic ideal, it has to get updated with time and experience and reports. There is no "non-hacky" way to do this, you just have to look at the reports that are coming in and keep adding rules that are relevant.
- tonfreed 5y agoYou say that there's no "non-hacky" way, but... https://www.youtube.com/watch?v=bJ5ppf0po3k https://www.youtube.com/watch?v=bJ5ppf0po3k He made a filter that did analysis of the pronunciation of the messages that were being sent. His breakdown of it starts at around 13:30
- ridaj 5y agoWell there are some significantly less hacky / more scalable ways to do this. This looks at the edge of maintainability, but if the lists were 10x as long and people constantly stepped on each other's toes causing outages while making updates, a bit of project investment probably wouldn't be a bad idea.
- NaturalPhallacy 5y ago> CREATE OR REPLACE FUNCTION is_hateful (VARCHAR) RETURNS BOOLEAN STABLE AS $$ >... >OR replace($1,'_','') SIMILAR TO '%(kill|keel|hang|burn)%(black|bl4ck|black|jew|trans|gay|african|afrikan|minorit|asian|nig|n1g)%' Protected classes: black, jewish, trans, gay, african, asian, minorities Non-protected classes: white Typical Twitch (D-CA). All (D-CA) companies really. Under the law (in the USA), all races, and sexes are equally protected. On big tech platforms it's only one slice of demographics that harassment and hatred is tolerated toward, even encouraged. Do they actually think this is helping? Singling out one demographic for abuse? Do they think this is progress? That people won't notice? That it will never backfire?
- prawn 5y agoIf you liked this chaos, you'd love my 15+ years of cobbled-together efforts at limiting forum spam and the like. I suddenly don't feel quite so alone in the myriad efforts needed to tackle this sort of thing.
- anigbrowl 5y agoIt's an endless arms race. I spend a lot of time studying unsavory people and a great deal of effort goes into the development of new dogwhistles that are designed to either provoke or connect with peers while maintaining deniability. You might like this paper on the evolutionary dynamics of covert social signaling: https://www.nature.com/articles/s41598-018-22926-1 https://www.nature.com/articles/s41598-018-22926-1
- prawn 5y agoI can really identify with that. My efforts are against spam and trolls; at least the spam is predictable and easier to take a sledgehammer to! Those grey-area trolls are such a miserable part of online content and moderation.
- exporectomy 5y agoIf they're covert and only members of the group using them know what they mean, then why fight it? Seems like a helpful way to communicate about things that others might not like to hear. The paper you linked said that too - "Such signals may allow coordination and enhanced cooperation while also avoiding the alienation or hostile reactions of individuals with different preferences.". This happens all the time in cartoons that appeal to children but also contain subtle adult jokes so everyone can enjoy them on different levels.
- prawn 5y agoBecause it encourages more of the same, and it's rarely completely covert. Often and increasingly, it's racial dogwhistling and just devolves every thread to wasteful, antagonistic and unproductive conversation. You might've heard the line/story about accommodating nazis in your bar.
- Igelau 5y agoWhoever leaked this is going to get buttbuttinated for sure. I have a hard time believing this silly mess is an actual component of anything.
- msdrigg 5y agoWhats this one about? CREATE OR REPLACE FUNCTION is_blasphemy (VARCHAR) RETURNS BOOLEAN STABLE AS $$ SELECT replace($1,'_','') SIMILAR TO '%p(o|0)rc(o|0)di(o|0)%' OR replace($1,'_','') SIMILAR TO '%p(o|0)rc(o|0)mad(o|0)nna%' $$ LANGUAGE SQL;
- cbsmith 5y agoItalian
- gary_0 5y agohttps://www.urbandictionary.com/define.php?term=porco%20dio https://www.urbandictionary.com/define.php?term=porco%20dio
- Igelau 5y agoThat's a beautiful swear. Far from naughty, I feel enriched having learned this.
- mensetmanusman 5y agoSheesh c) "Porco dio saranno mica i testimoni di Geova? No eh diocan digli che i signori sono fuori, non ho tempo per stargli dietro." ("Fuck, they can't be Jehovah's witnesses, can they? Tell them we're out, we don't have time for their shit.")
- msdrigg 5y agoI tried googling it but I put it all in one word. Thanks for the help!
- alkz 5y agothat should actually be porcAmadonna
- marton78 5y agoGrazie.
- 5y ago
- vmception 5y agothese are so meme-able
- oxplot 5y agoThis is pretty much the same problem as Email spam. There needs to be a service/collaborative project to filter these, instead of each app hacking up an ad-hoc way of doing it.
- foota 5y agoMan, I was _not_ expecting it to be stored procedures lmao.
- neither_color 5y agoIt seems NVIDIAMINER could be controversial.
- iab 5y agoLanguage friend, this is a family site
- phamilton 5y agoWe had to do this for a link shortening system (to make sure random base64 didn't contain profanity). It was a pretty fun problem. Not just the implementation, but doing the math to make sure it didn't make our shortened links easily enumerable. The implementation wasn't too bad, but we set up logging initially to spit out any random strings it decided to block. I demo'd this in front of the whole company and live tailed the logs and the first one that popped up during the demo was a big ole F bomb. It made for an excellent demo.
- smohare 5y agoIt’s so laughable that we care about whether a generated string contains some temporally relevant profanity. We truly are still barbarians, and will be viewed as such by history.
- tsimionescu 5y agoWell, given that we have a more than 5000 years old habit of looking for omens in random data to divine the future (whether that random data is scattered bones, laid out animal entrails, tea leaves, coffee grounds, tarot cards or so many others), it's unfortunate but not surprising.
- gregoriol 5y agoIt highly depends on what the purpose of the string is: if it can be clearly seen by the user, has to be read, or worse typed, then it's not just a random string in a database. If the string is a url, imagine sending https://somesite/wanker https://somesite/wanker to your client, when it actually could also be https://somesite/ay3ugd https://somesite/ay3ugd
- Biganon 5y agoIt's not that stupid. People will send the shortened link to other people, who might not understand that the string was randomly generated.
- pixl97 5y ago
- mdoms 5y agoGamers sure do love the n word.
- revicon 5y agoI had to google "mike hawk" before I got it.
- legostormtroopr 5y agoPoor Bilbo and Frodo - they'll never be allowed to use Twitch: CREATE OR REPLACE FUNCTION is_hateful (VARCHAR) RETURNS BOOLEAN STABLE AS $$ SELECT ... OR replace($1,'_','') LIKE '%aggin%'
- jodrellblank 5y agoOr designer of the first commercial microprocessor, Intel's 4004 https://en.wikipedia.org/wiki/Federico_Faggin https://en.wikipedia.org/wiki/Federico_Faggin
- sorenjan 5y agoReminds me of the guy that streamed a talking banana on Twitch, where viewers could make it say things. People submitted variations of the n-word and got him banned, and after trying to filter out all character combinations he could think of he wrote a phonetic filter. That apparently worked much better than trying to think of every permutation of characters that sounds like bad words. https://youtu.be/bJ5ppf0po3k?t=715 https://youtu.be/bJ5ppf0po3k?t=715
- geoduck14 5y ago>the guy that streamed a talking banana on Twitch Of all the ways I expected a talking banana to backfire, I didn't expect this one. Thanks for sharing
- pjc50 5y agoAnything exposed to the open internet devolves into porn or racism or both unless active effort is made to prevent it. I'm reminded of https://en.wikipedia.org/wiki/Tay_(bot) https://en.wikipedia.org/wiki/Tay_(bot)
- ElectronShak 5y agoquite curious as to how Instagram managed to avoid this, despite launching with no moderation, (just Mike and Kevin), yet all you needed was an email to use the service...
- pjc50 5y agoThey certainly have moderation now, and it's a constant exercise in boundary-pushing. I'd be interested in a "history of Instagram moderation", that would be a great piece of anthropology. Was the no-links policy there from the start? I think that would have helped a lot. As it is, you're allowed basically one outbound link from your profile, so there are link-expander services which people use to link to more things.
- 0des 5y ago
- DethNinja 5y agoWhy not just convert numbers like 1 to i or l then check with a manually created bad word list? Would regex be really much faster than checking it against a 1000 or more bad word list? Also bad word list can easily get updated by moderators as well, I really can’t understand the logic behind using so much regex.
- cbsmith 5y agoA bad words list is a regex. ;-) But note: for the most part this isn't using regexs, and to the extent it does, it seems largely intended to make the maintainers' lives easier by avoiding having to represent (and maintain) all the permutations they are trying to match for. What's sad though is that they're doing many, many passes through the pattern matcher, rather than just building a single big DFA from the whole list of patterns they want to match, which gets traversed in one pass.
- sa46 5y agoCould Postgres convert the function into a DFA using the JIT optimizer (based on LLVM)? That might delve into sufficiently smart compiler territory but recognizing a bunch of OR'ed string matches seems on the easier end of optimization passes.
- cbsmith 5y agoYup, that's definitely sufficiently smart compiler territory. If you want to write an optimization pass that handles that, go for it, but you won't find one already there.
- jakear 5y agoRegEx => DFA conversation can lead to exponential growth in number of states... don't know how big it'd be in this case.
- PostThisTooFast 5y agoThis is one of my favorite posts on HN. Meanwhile, I HOPE I'M NOT "POSTING TOO FAST," ASSHOLES. IT'S ONLY BEEN A COUPLE OF DAYS.
- oroul 5y ago> LIKE '%aggin%' Looks like "Baggins" is banned. Poor Bilbo...
- tjpnz 5y agoSo is Mike Hunt. Poor Mike.
- b112 5y agoSure, but... The story we know was written by him. I wonder what the trolls would say, or the dead dragon, or the town that his actions helped destroy? Is he a hero, or, just the guy who wrote it all down? And just when something really important happens, he throws a powerful weapon(the one right) at his nephew and goes away to retire! A life lead with riches (gold from the trolls), a ring granting him extremely long life and health, and yup.. off he goes, first sign of real trouble. Poor Bilbo indeed!
- CobrastanJorji 5y agoI feel like you could make an interesting game out of this. Given these rules, find the best "false negative," a realistic and inoffensive, but banned username. My best so far are "brownie_gurl" and "Megasthenes."
- buildsjets 5y agoMy wife used to name her RPG characters “Isis” after a cat we used to have. Used to have to explain to vets that we named her Isis years before Bush and Cheney created Isis by starting their illegitimate war in Iraq.
- mikojan 5y agoHonestly, what a great way to start _that_ conversation haha
- alentist 5y agoISIS was founded in 1999. The Iraq War began in 2003. Bush's administration ended in 2009. ISIS gained power in 2014, 5 years into Obama's administration.
- mellavora 5y agoBetter yet, my children's school had a parent support portal called Isis. I have some fun emails whose subject line is "here are your ISIS family log-in details" and this was right around the time the terrorist group was frequently in the news
- ggrelet 5y agoAre people not familiar with the Egyptian goddess, though?
- deleted 5y ago[deleted]
- aasasd 5y agoAh, so that's the Word Filter that Twitch directly mentioned as the majority of their effort to fight hate comments (as cited in the main thread on the leak). Also, Lisa Pedro is not welcome there.
- viraptor 5y agoLet's not jump to conclusions. We don't know where this specific filter was used, is it an automated blocker or just flagging for moderation, is it the only layer, or is it even the currently used one.
- swman 5y agoSo what is a good practice for this kind of thing? I’ve got an app where users create a display name. Is there a legit service I can use or some actual well tested library that can help me? I’m using node and Go so either languages.
- deleted 5y ago[deleted]
- cookiengineer 5y agoI wonder why they didn't go with syllables and something like a levenshtein distance to syllables? That way they could more easily maintain similar looking and sounding words, including leetspeak and other variants. Because with this kind of approach, something like "yolocaust" will get through, as most checks only go with exact matches and permutations will always get around it easily... Whereas with something like a levenshtein distance you could compare it with a set of words and syllables and if it's too similar looking, e.g. 90% the same distance compared to username length, you could simply block it.
- guywhocodes 5y agoPeople how hate naggers don't like levenshtein either
- PrincessJess 5y agoAgreed. If it's even close to a no no word it should be banned completely. No more riggers, naggers, poggers, biggas, lucks, bunts, minks, bikes, or trikes. This is a doubleplusgood plan.
- cookiengineer 5y agoMaybe usernames in general should require a higher entropy than words in a dictionary :D
- broAhmed 5y agoGlad to see the word Niger (as in the country) wasn’t blocked. Ran into this with Venmo a little while ago and they thankfully backpedaled. https://news.ycombinator.com/item?id=24042742 https://news.ycombinator.com/item?id=24042742
- wiradikusuma 5y agoThere should be an open source or even Official RFC(tm) for this. The use case is very generic.
- ibudiallo 5y agoI used to work for a company that automated customer service. While we were trialing with new a client, our ai service responded to a customer: "Hey B**, thanks for reaching out." To our surprise, the end user did not feel offended at all. In fact, they were happy because we responded instantly instead of the usual 24 to 48 hours. [Story]: https://idiallo.com/blog/do-you-make-your-customers-wait https://idiallo.com/blog/do-you-make-your-customers-wait
- _hilro 5y agoThe word is bitch so the sanitised version would be B** Why portray the bad word as 3 letters - it doesn't make sense.
- mgdlbp 5y agoMore high-profile profanity filters open for viewing: edit filters used in Wikimedia projects, which also together encompass many languages. The main 'bad words' filter of English Wikipedia: https://en.wikipedia.org/wiki/Special:AbuseFilter/384 https://en.wikipedia.org/wiki/Special:AbuseFilter/384 The page (on en:wiki) listing all filters, which also have uses other than detecting abuse: https://en.wikipedia.org/wiki/Special:AbuseFilter https://en.wikipedia.org/wiki/Special:AbuseFilter The special page has the same title in other editions of Wikipedia and other Wikimedia wikis, though many filters are set to hidden. Dutch Wikipedia, for example: https://nl.wikipedia.org/wiki/Special:AbuseFilter/10 https://nl.wikipedia.org/wiki/Special:AbuseFilter/10
- cesarb 5y agoA more traditional one, older than these you linked: https://en.wikipedia.org/wiki/MediaWiki:Titleblacklist https://en.wikipedia.org/wiki/MediaWiki:Titleblacklist and https://meta.wikimedia.org/wiki/Title_blacklist https://meta.wikimedia.org/wiki/Title_blacklist
- staysafeanon 5y agoActually in Twitch's codebase: OR replace($1,'_','') SIMILAR TO '%(hate|kill|keel|hang|burn|gasthe)%(black|bl4ck|black|jew|trans|gay|african|afrikan|minorit|asian|nig|n1g)%' Twitch: "Hatred against Blacks, Jews, Asians, trans? BANNED! Hatred against Whites? I'LL ALLOW IT!"
- lkey 5y agoYou are so hungry to start something here, you've posted this exact same comment 3 times. Wouldn't you have more fun on 4chan?
- staysafeanon 5y agoThree different threads, all relevant. 4chan is where this was leaked/found out.
- staysafeanon 5y agoThe flagged comment: Actually in Twitch's codebase: OR replace($1,'_','') SIMILAR TO '%(hate|kill|keel|hang|burn|gasthe)%(black|bl4ck|black|jew|trans|gay|african|afrikan|minorit|asian|nig|n1g)%' Twitch: "Hatred against Blacks, Jews, Asians, trans? BANNED! Hatred against Whites? I'LL ALLOW IT!"
- for1nner 5y agoAmusing to see the is_hateful query which used to just be an unreadable mess of regex for validation. Clearly the infra has grown/changed over 10 years, and this is obviously neater, but boy does it LOOK a lot bigger now (and I'm sure they've added yet more new words that try to hurt people...)
- imnitishng 5y agoWhy is all of this implemented in SQL? Wouldn't it be better to do it in code with dedicated methods to filter stuff out? IMO logic inside of SQL queries just adds unnecessary complexity, implementing this in code would've been maintainable and testable.
- dvh 5y agoBoss: "Hey Bob! People started spamming one of our boards, we are currently busy doing other things and cannot deploy new client, can you make filter with these 3 words and deploy it ASAP?" Bob: "Sure no problem if it's only temporary" --few months later Chief architect: "People are spamming more and more, we should design new system for these 234 new bad words, I need team of 7, two backend guys, 5 frontend guys and 4 weeks. It will also require minor rewrite of few external components." Boss: "geez we're in the middle of sprint right now, Bob can you add these 234 words to existing filter? Make sure it's in production before lunch, thanks" (checks watches) "I have to go now, meeting with customer, bye".
- bni 5y agoIt is FUNCTIONS so it is code like any other. Even SQL is code (declarative). The reason is: This can be updated very easily and out-of-band of other deploys of the main application code. To make this work it has to be super agile to update. Probably this version we are looking at is an old version that happened to be put in git. The functions in prod probably have several additions since.
- 02020202 5y agoniggers got A LOT of attention. they are the new jews.
- dredmorbius 5y agoPrejudice. Just six seemingly harmless letters arranged in a way to form a word with more power than the pieces of metal which is forged to make swords. Just a couple of G's, an R and an E, an I and an N.... https://youtube.com/watch?v=KVN_0qvuhhw https://youtube.com/watch?v=KVN_0qvuhhw
- darepublic 5y agoInteresting to see this. These are like the ten commandments, primitive yes/no rules that reflect the people who came up with them. A world of nuance is missing obviously, not to mention gaps that can be gamed. How do you express ethics in code
- inagiledev 5y agoI feel sad for Mike Hunt, who can't use his name for a username :)
- mdrzn 5y agoAw, why can't you be nicknamed 420blazeit? That's not even offensive and there are still way more weed-nicknames available, so I don't get it.
- teddyh 5y agoYou can if you’re a magic fire sprite: http://www.threepanelsoul.com/comic/picked-out http://www.threepanelsoul.com/comic/picked-out
- no1lives4ever 5y ago420 == marijuana, blazeit - self explainatory.. not everyone is fine with it..
- marcus_holmes 5y agoWe used to call this the "Scunthorpe problem" - how do you filter for obviously rude names while still allowing people to have their actual names? Bearing in mind that some people's names are actually rude. I worked on the application form processing for the Nectar card launch in the UK back in the early 2000's, and we had several cases. Luckily we had human data-entry clerks in the loop, so all we had to do was flag when a name contained anything on the "Scunthorpe list" and get a human to look at it. Even then it wasn't perfect and a few slipped through. One of the early PR messes of the launch was someone getting a Nectar card issued with a rude name, and of course they immediately went to the press with it [0] I'm interested to see they still haven't solved this [1] [0] I never saw the journalistic interest in this: this guy said his name was <rude name>, and then signed the form to say everything on the form was true. Why is anyone surprised or interested that we accepted his name was what he said it was and gave him a card in that name? [1] https://metro.co.uk/2015/02/20/woman-refused-sainsburys-nectar-card-because-her-first-name-is-fanny-5071703/ https://metro.co.uk/2015/02/20/woman-refused-sainsburys-nect...
- chanandler_bong 5y agoThe city of Toppenish in the US had the same problem when the council turned on a generic filtering system for the city networks. Everything stopped working.
- qwerty456127 5y agoPeople should just be taught (I would consider this is an essential skill which should be put in basic school curriculum to psychologically prepare people for the life, many support/sales professionals learn it quickly but the rest of the people don't) words are just words and they don't have to take them seriously. No word is rude until you consider it rude. No word can actually harm you directly. It's you who is in charge of deciding what to feel whenever you hear/see a particular word. Sadly, most of the people just ignore this job. I personally enjoy being called by the derogatory nickname of my ethnicity because I just like how does it sound. First half of my life I didn't even know it was supposed to be insulting (as I didn't know actual racists exist and can be serious about hating/judging people by just their ethnicity).
- Fiahil 5y agoSo, it's only working for english and english-related profanity ? Interesting.
- OneTimePetes 5y agoIts surprising how good solid best effort parsing is by the human visual system. sukciD suggiB was here..
- rzwitserloot 5y agoThey included 'mike hawk' (say it out loud...)? If that's on the list, there are a few thousand other auditory joke names that should have been on here.
- chakintosh 5y ago%nekker% I pity those who chose that username based on The Witcher's Nekkers.
- NiceWayToDoIT 5y agoIf "fork" and "shirt" are allowed I am fine. :) You can still use any other language except English to achieve the goal .
- appel 5y agoUnexpected "The Good Place" :)
- jcun4128 5y agoWas surprised it's SQL but guess that makes sense/faster to be done there
- amai 5y agoSee https://en.m.wikipedia.org/wiki/Scunthorpe_problem https://en.m.wikipedia.org/wiki/Scunthorpe_problem
- Hippocrates 5y agoDoing this in SQL is absolute insanity. I’ve seen many pieces of code grow in this way and I understand how it happens but still surprises me to see how scrappy things are under the hood at some big-time companies.
- who-shot-jr 5y agoHello, what would be a better way to do it? In code? Regex?
- Hippocrates 5y agoProbably yes, in code with regex. To name a few advantages: 1. Better readability and organization. Rules and word lists can be abstracted to more of a config format. 2. possible to easily store in a database and support dynamic additions/changes to the rules and words 3. Better accountability of performance. Able to use profiling to catch any perf issues in the rules. 4. Testable with unit tests
- Iv 5y agoReal story: I needed to register an instagram account a few months ago. I tried my nick "Iv", of course it was unavailable. I tried many different alternatives and finally managed to get "Iv but the real Iv" or ivbuttherealiv. Recently I saw my account was banned. I then realized that there was a "butt" in there. Now I will never have an IG account linked to my FB. Oh well, I can live without that but thank god I don't depend on that platform for business. It made me laugh but there is zero appeal available.
- qayxc 5y agoA clbuttic [0] solution to the problem. [0] https://en.wiktionary.org/wiki/clbuttic https://en.wiktionary.org/wiki/clbuttic
- JadoJodo 5y agoThe movie Gran Torino came out when I was twenty, and in seeing it, I heard many racial slurs uttered in context for the first time. I was, of course, familiar with the "main" ones, but the one in particular that I remember was "spook". I remember it vividly because, upon hearing it, I had an immediate realization that I had heard it before when it was said near the end of Back To The Future (the scene where Marvin Berry and the Starlighters chase off Biff's gang). In my youthful innocence I had assumed the characters in the movie were trading silly, made-up names (e.g. "Who are you calling "sparky", Popeye?"), not racial epithets. The fact that this list needs to exist makes me sad, but I am glad technology can assist on some level with the issue.
- brokenwren 5y agoWe tried to convince Twitch for years that their filters were garbage and they should use CleanSpeak. They kept insisting their engineering team had written the best filter in the world. Sometimes it’s just not worth the effort to try and help people solve these problems.
- crorella 5y agowhy the DB? Why not the client itself? How much data was wasted back and forth between the client and the backend? how many CPU cicles were spent in that poorly optimized SQL statement?
- leokennis 5y agoPotential solution for this; - Have people choose any username they want - Have an “after the fact” human review system on usernames - If the username is inappropriate, change it to “smallsausage[0-9]+” without the option of reverting it back or requesting a new user on the same e-mail address.
- mokarma 5y agoThis SQL file could be an entire AI-based, ML-driven startup to stop hateful / offensive language. Just need a good name like Klean or, NoH8 or something.
- whomeIamme 5y agoLove how there is nothing in there to prevent white cracker from being said.