7 ms·

I have a hard time believing this was / is the real version used. It doesn't seem broad enough. More likely it was a kind of smoketest that made sure that a mor

by rhema 5y ago

I have a hard time believing this was / is the real version used. It doesn't seem broad enough. More likely it was a kind of smoketest that made sure that a more automated keyword checker was working. It does remind me of the XKEYSCORE (Snowden leaks) that used keywords to bubble up potential threats from emails etc https://www.businessinsider.com/nsa-prism-keywords-for-domestic-spying-2013-6 https://www.businessinsider.com/nsa-prism-keywords-for-domes... .

ridaj 5y ago
This looks like legit, no-nonsense gets-the-job-done code that gets updated every time some jerk find a new way to be a jerk to others. It isn't great, but at least not over-engineered, and I'm not sure if Twitch account sign-up volumes and abuse are at the point where they should staff a project to do this more robustly / scalably
- OskarS 5y ago
  Exactly what I think. This is the kind of code that doesn't have a platonic ideal, it has to get updated with time and experience and reports. There is no "non-hacky" way to do this, you just have to look at the reports that are coming in and keep adding rules that are relevant.
  - tonfreed 5y ago
    You say that there's no "non-hacky" way, but... https://www.youtube.com/watch?v=bJ5ppf0po3k https://www.youtube.com/watch?v=bJ5ppf0po3k He made a filter that did analysis of the pronunciation of the messages that were being sent. His breakdown of it starts at around 13:30
  - ridaj 5y ago
    Well there are some significantly less hacky / more scalable ways to do this. This looks at the edge of maintainability, but if the lists were 10x as long and people constantly stepped on each other's toes causing outages while making updates, a bit of project investment probably wouldn't be a bad idea.
- baud147258 5y ago
  it's missing a few slurs, I'm not sure it gets updated (or it's a filter elsewhere which gets updated)
- notjustanymike 5y ago
  There are portions of my codebase that are intentionally "dumb" code. They contain cascading rules controlling what UI elements are visible that can be challenging to reason about. So I wrote it so simple that anyone can read it. I see the same here. It's not clever, but no one has any doubt what words are being checked.
b112 5y ago
If valid, this means virtually every phone call, and every email (with clients) is flagged. :P This must be more nuanced. Maybe, it's "additional algo processing when a word is hit", eg another layer before "involve human".
mc32 5y ago
It also mostly checks for English naughty words and not much else. People can have fun in lots of other languages, so it would seem this is a small sample.
- formerly_proven 5y ago
  Bunch of ineffective entries too, all patterns containing underscores won't ever match.
  - talkin 5y ago
    Underscore matches any single char, so those patterns work fine. It’s an efficient way to match any small separator, like space, dashes, etc.
    - formerly_proven 5y ago
      I didn't know _ is a metachar in LIKE expressions, thanks! Not the first time its syntax caught me off-guard.
      - CRConrad 5y ago
        It only has two, doesn't it? Underscore for "any single character", and percent for "any string of characters", AFAIK.
- Igelau 5y ago
  Some near the end looked like they might be in another language, but I won't be the one to find out. It used to be "if I search for this term, am I accidentally going to wind up getting goatse or something?" The good old days. Now it's "if I search for this term, is the FBI going to kick my door in?"
  - fsckboy 5y ago
    I was pedantic about it in the good old days too, it's goatse.cx and not goatse you need the domain to make the goat-sex joke work
comrh 5y ago
I can't find the article now but the developer who wrote these scripts said they were a singular effort from years ago before security was taken over by a more formal development team.
woodruffw 5y ago
Someone in another thread mentioned that these might be part of corpus generation for an ML model. That would make more sense to me.
- gipp 5y ago
  Based on the filepath given in this very thread, it seems plain that that's the case (safety-ml\offensive-usernames\data_pull\sql\bad.sql).
- cbsmith 5y ago
  Training an ML model to "learn" a rules engine strikes me as an incredibly bad practice. It'd make more sense to just have an actual corpus of labeled data.
throwaway984393 5y ago
It seems you are assuming that software is usually written well, or as well as it can be. It's much more likely to be the opposite.
- jldugger 5y ago
  Would be unsurprised if some poor engineer got assigned the project, realized it was an untractable mess of scunthorpe, and decided to check some boxes and move on to some ticket of higher value.
- MattGaiser 5y ago
  This seems like the kind of thing that would be horribly specced and be a user story along the lines of "the user must not be allowed to make an inappropriate username." The engineer would write something for every test case the product manager complained about, anything else computationally easy, and call it a day. I once had to implement an audit logging system. What was supposed to be logged? "Important actions." Nobody on the team could define it. We just logged every database write along with the username responsible and called it a day. Nobody ever followed up or inspected it. Same deal. Both exist mostly for compliance.
  - h2odragon 5y ago
    Bad, old memory: "Every transaction must have a line written to the printer" Tracking down "why is the antique system suddenly slow". Power went out, system came back up fine, everything but the one ancient but vital app is fine. Dig, dig dig, there's this old dot matrix printer in another room (because it used to be loud and annoying) that no on has fed or looked at in years. It finally died with that outage, and it not accepting data was the problem. It had cheerfully printed the ribbon through, then fed out the rest of the box of paper it had, and that might've been several years before i saw it. The roller the paper was supposed to ride had been eroded. The metal rods the print head rode on had a perceptible bump at the ends of the normal stroke. The fix was a little dongle for the printer port that held the appropriate "i'm alive" lines up. hardware /dev/null. I'm thinking it was 25 pin rs232 because I remember a lot of cussing over it.
gowld 5y ago
That list is a list of words chosen by William Knowles to taunt any NSA who may be listening. It's not a list of words used by the NSA or any spies. https://attrition.org/misc/keywords.html https://attrition.org/misc/keywords.html
- geoduck14 5y ago
  In a previous life, all of our code was scanned for "vulnerabilities". One of the issues they looked for was if passwords were being stored in local variables. Initially, LOTS of people would do something like: $Username=<username> $Password=<password> Connection.string=($Username, $Password) The parser would flag this - Password was being stored to a variable! So we just changed our code: $pw=<Password>. Problem solved!
makomk 5y ago
Seems plausible to me. One of the streamers on Twitch had an actual wooden board on the wall he lasered subscriber names onto, and some of the regulars had fun finding lewd usernames to gift subs to. There were quite a lot of them out there. It was kind of a running joke how much Twitch let through the cracks.

by rhema 5y ago

ridaj 5y ago

This looks like legit, no-nonsense gets-the-job-done code that gets updated every time some jerk find a new way to be a jerk to others. It isn't great, but at least not over-engineered, and I'm not sure if Twitch account sign-up volumes and abuse are at the point where they should staff a project to do this more robustly / scalably

OskarS 5y ago
Exactly what I think. This is the kind of code that doesn't have a platonic ideal, it has to get updated with time and experience and reports. There is no "non-hacky" way to do this, you just have to look at the reports that are coming in and keep adding rules that are relevant.
- tonfreed 5y ago
  You say that there's no "non-hacky" way, but... https://www.youtube.com/watch?v=bJ5ppf0po3k https://www.youtube.com/watch?v=bJ5ppf0po3k He made a filter that did analysis of the pronunciation of the messages that were being sent. His breakdown of it starts at around 13:30
- ridaj 5y ago
  Well there are some significantly less hacky / more scalable ways to do this. This looks at the edge of maintainability, but if the lists were 10x as long and people constantly stepped on each other's toes causing outages while making updates, a bit of project investment probably wouldn't be a bad idea.
baud147258 5y ago
it's missing a few slurs, I'm not sure it gets updated (or it's a filter elsewhere which gets updated)
notjustanymike 5y ago
There are portions of my codebase that are intentionally "dumb" code. They contain cascading rules controlling what UI elements are visible that can be challenging to reason about. So I wrote it so simple that anyone can read it. I see the same here. It's not clever, but no one has any doubt what words are being checked.

b112 5y ago

If valid, this means virtually every phone call, and every email (with clients) is flagged. :P This must be more nuanced. Maybe, it's "additional algo processing when a word is hit", eg another layer before "involve human".

mc32 5y ago

It also mostly checks for English naughty words and not much else. People can have fun in lots of other languages, so it would seem this is a small sample.

formerly_proven 5y ago
Bunch of ineffective entries too, all patterns containing underscores won't ever match.
- talkin 5y ago
  Underscore matches any single char, so those patterns work fine. It’s an efficient way to match any small separator, like space, dashes, etc.
  - formerly_proven 5y ago
    I didn't know _ is a metachar in LIKE expressions, thanks! Not the first time its syntax caught me off-guard.
    - CRConrad 5y ago
      It only has two, doesn't it? Underscore for "any single character", and percent for "any string of characters", AFAIK.
Igelau 5y ago
Some near the end looked like they might be in another language, but I won't be the one to find out. It used to be "if I search for this term, am I accidentally going to wind up getting goatse or something?" The good old days. Now it's "if I search for this term, is the FBI going to kick my door in?"
- fsckboy 5y ago
  I was pedantic about it in the good old days too, it's goatse.cx and not goatse you need the domain to make the goat-sex joke work

comrh 5y ago

I can't find the article now but the developer who wrote these scripts said they were a singular effort from years ago before security was taken over by a more formal development team.

woodruffw 5y ago

Someone in another thread mentioned that these might be part of corpus generation for an ML model. That would make more sense to me.

gipp 5y ago
Based on the filepath given in this very thread, it seems plain that that's the case (safety-ml\offensive-usernames\data_pull\sql\bad.sql).
cbsmith 5y ago
Training an ML model to "learn" a rules engine strikes me as an incredibly bad practice. It'd make more sense to just have an actual corpus of labeled data.

throwaway984393 5y ago

It seems you are assuming that software is usually written well, or as well as it can be. It's much more likely to be the opposite.

jldugger 5y ago
Would be unsurprised if some poor engineer got assigned the project, realized it was an untractable mess of scunthorpe, and decided to check some boxes and move on to some ticket of higher value.
MattGaiser 5y ago
This seems like the kind of thing that would be horribly specced and be a user story along the lines of "the user must not be allowed to make an inappropriate username." The engineer would write something for every test case the product manager complained about, anything else computationally easy, and call it a day. I once had to implement an audit logging system. What was supposed to be logged? "Important actions." Nobody on the team could define it. We just logged every database write along with the username responsible and called it a day. Nobody ever followed up or inspected it. Same deal. Both exist mostly for compliance.
- h2odragon 5y ago
  Bad, old memory: "Every transaction must have a line written to the printer" Tracking down "why is the antique system suddenly slow". Power went out, system came back up fine, everything but the one ancient but vital app is fine. Dig, dig dig, there's this old dot matrix printer in another room (because it used to be loud and annoying) that no on has fed or looked at in years. It finally died with that outage, and it not accepting data was the problem. It had cheerfully printed the ribbon through, then fed out the rest of the box of paper it had, and that might've been several years before i saw it. The roller the paper was supposed to ride had been eroded. The metal rods the print head rode on had a perceptible bump at the ends of the normal stroke. The fix was a little dongle for the printer port that held the appropriate "i'm alive" lines up. hardware /dev/null. I'm thinking it was 25 pin rs232 because I remember a lot of cussing over it.

gowld 5y ago

That list is a list of words chosen by William Knowles to taunt any NSA who may be listening. It's not a list of words used by the NSA or any spies. https://attrition.org/misc/keywords.html https://attrition.org/misc/keywords.html

geoduck14 5y ago
In a previous life, all of our code was scanned for "vulnerabilities". One of the issues they looked for was if passwords were being stored in local variables. Initially, LOTS of people would do something like: $Username=<username> $Password=<password> Connection.string=($Username, $Password) The parser would flag this - Password was being stored to a variable! So we just changed our code: $pw=<Password>. Problem solved!

makomk 5y ago

Seems plausible to me. One of the streamers on Twitch had an actual wooden board on the wall he lasered subscriber names onto, and some of the regulars had fun finding lewd usernames to gift subs to. There were quite a lot of them out there. It was kind of a running joke how much Twitch let through the cracks.