2 ms·
I had occasion to write a chat server for a small gaming service once. The way I came up with to do the profanity filter ended up working a lot better than I ex
by tzs 7y ago
I had occasion to write a chat server for a small gaming service once. The way I came up with to do the profanity filter ended up working a lot better than I expected it to, and avoided the problem you describe.
It worked like this.
1. Make a copy of the text to be filtered, modified as follows.
-- change upper case letters to lower case
-- change 0 to o, 1 to i, 3 to e, 5 to s, and vowels with various accent marks to the corresponding unaccented lower case vowel
-- discard anything else
For example, if the input was
start the assault. let's f-u-c-k them
it would become
starttheassaultletsfuckthem
2. Scan this for any substrings that are on the bad word list. In this case it would find "ass" and "fuck" [1].
3. If no bad words are found, the original string is returned and we are done.
4. For each word in the original string, look it up in the dictionary (I believe I used /usr/dict/words). Note the ranges of character positions in the original string of the words that are spelled correctly.
5. For each bad word found in the filtered string, if all of its characters came from positions in the original string that were part of correctly spelled words, leave it alone. Otherwise, asterisk out its characters in the original string.
In the above example, "ass" would be left alone because all of its characters came from "assualt" which is recognized as a correctly spelled word. None of "fuck" came from correctly spelled words, so it would get zapped. The final result would be
start the assault. let's *-*-*-* them
[1] Well...actually not. I just checked my archives and found the chat server bad word list. It did not include "ass".
- woodrowbarlow 7y agojust... be sure you're at least aware of the scunthorpe problem that this font is named after. do not implement an obscenity filter without being aware of the inconveniences they pose. my name is woody (like the tom hanks cowboy, but also, i've been told, like the outdated euphemism for an erection) and i've been scunthorped out of creating accounts on dozens of sites because of my name. it's aggravating at the best of times, but doubly so when i want to be actually recognizable on the service in question. https://en.wikipedia.org/wiki/Scunthorpe_problem https://en.wikipedia.org/wiki/Scunthorpe_problem