5 ms·

Author here. I made this almost five years ago when I was in high school. The Scunthorpe problem is real - "shit" is used in a lot of good compound swears. The

by uiri 9y ago

Author here. I made this almost five years ago when I was in high school. The Scunthorpe problem is real - "shit" is used in a lot of good compound swears. The code isn't public because I was concerned about people taking it to make a more popular version of the same thing. Not that it is difficult to glue together the two APIs. It is also an embarrassing mess of around 150 lines of Python. One issue with linking to the commits or repo is naming & shaming. The other is, as I mentioned, people trying to get on the bot intentionally. It scans a GitHub API once a minute so as not to put noticeable strain on their API. I think the firehose of constant commit messages has only gotten worse.

painbody 9y ago
Some interesting collisions here: https://en.wikipedia.org/wiki/Scunthorpe_problem https://en.wikipedia.org/wiki/Scunthorpe_problem "Problems can occur with the words socialism, socialist, and specialist because they contain the substring Cialis, the brand name for an erectile dysfunction medication commonly advertised in spam e-mails."
Daviey 9y ago
>code isn't public because I was concerned about people taking it to make a more popular version of the same thing. Why would that be a bad thing?
rootw0rm 9y ago
this would not mix well with my alcohol fueled coding 'technique'. it would be fucking hilarious for almost 5 seconds a day tho.
flavio81 9y ago
When i saw this, i thought the reason for making this bot was obvious: Measuring code quality in "WTFs per minute" (as detailed on this famous comic - http://i.imgur.com/J1svNp7.jpg http://i.imgur.com/J1svNp7.jpg) The less WTF/min, the better the code is.
tzs 9y ago
I did the communications server for a small online gaming site once, and it had a profanity filter on the chat channels. The approach we took mostly avoided the Scunthorpe problem, and also caught when people tried to sneak profanity past it by splitting it across words (e.g., knowing that "motherfucker" would be filtered, someone might try "motherfu cker", or maybe even "m0th3rfu ck3r"). Here is what we did, as far as I recall. This worked a heck of a lot better than I expected it to. 1. Make a copy of the text and replace all the common non-letters that look somewhat like letters with those letters. E.g., 0 => O, 1 => L, 3 => E, 4 => A, 7 => T, @ => A. 2. Do a spell check on each word. Set a flag on each character that comes from a word that passed the spell check. 3. Scan the sequence of characters, ignoring all punctuation and white space, looking for sequences that match known profanity. 4. For each sequence that matches, mark it as profanity if any of its characters were not flagged in step #2. As described above this probably would not scale well to high traffice because of the spell check. That could be improved by changing the order. Look for potential profanity first. Only if some is found would you then do the spell check, and you would only need to spell check the words that span the potential profanity.

by uiri 9y ago

painbody 9y ago

Some interesting collisions here: https://en.wikipedia.org/wiki/Scunthorpe_problem https://en.wikipedia.org/wiki/Scunthorpe_problem "Problems can occur with the words socialism, socialist, and specialist because they contain the substring Cialis, the brand name for an erectile dysfunction medication commonly advertised in spam e-mails."

Daviey 9y ago

>code isn't public because I was concerned about people taking it to make a more popular version of the same thing. Why would that be a bad thing?

rootw0rm 9y ago

this would not mix well with my alcohol fueled coding 'technique'. it would be fucking hilarious for almost 5 seconds a day tho.

flavio81 9y ago

When i saw this, i thought the reason for making this bot was obvious: Measuring code quality in "WTFs per minute" (as detailed on this famous comic - http://i.imgur.com/J1svNp7.jpg http://i.imgur.com/J1svNp7.jpg) The less WTF/min, the better the code is.

tzs 9y ago

I did the communications server for a small online gaming site once, and it had a profanity filter on the chat channels. The approach we took mostly avoided the Scunthorpe problem, and also caught when people tried to sneak profanity past it by splitting it across words (e.g., knowing that "motherfucker" would be filtered, someone might try "motherfu cker", or maybe even "m0th3rfu ck3r"). Here is what we did, as far as I recall. This worked a heck of a lot better than I expected it to. 1. Make a copy of the text and replace all the common non-letters that look somewhat like letters with those letters. E.g., 0 => O, 1 => L, 3 => E, 4 => A, 7 => T, @ => A. 2. Do a spell check on each word. Set a flag on each character that comes from a word that passed the spell check. 3. Scan the sequence of characters, ignoring all punctuation and white space, looking for sequences that match known profanity. 4. For each sequence that matches, mark it as profanity if any of its characters were not flagged in step #2. As described above this probably would not scale well to high traffice because of the spell check. That could be improved by changing the order. Look for potential profanity first. Only if some is found would you then do the spell check, and you would only need to spell check the words that span the potential profanity.