3 ms·

I'm not sure about processes at 'big' companies like Twitter but in my experience if an inexperienced developer sees a field in a DB or API it's fair game to us

I'm not sure about processes at 'big' companies like Twitter but in my experience if an inexperienced developer sees a field in a DB or API it's fair game to use it even if inappropriate, and this could be the source of the mistake. So perhaps one team forgot to secure it property (based on role authorisation) and another team saw the field and didn't think it was something they shouldn't use (maybe an OKR or some target as kicking them in the direction of being eager to use what they could get). Obviously nothing happened accidentally at a coding level, but at a business-constraint understanding level a mistake may have been made.

hazelnut 7y ago
Pretty sure there was a product manager who drove the project. This person needs to understand where the data is coming from and where it flows to. This person is also responsible for checking with legal to see that everything is in line. Besides that, there're cycles of product reviews with directors and VP's.
munk-a 7y ago
I could buy that marketing/sales snuck around dev's oversight to manually get a copy of that to use - possibly by enlisting a poor junior dev to do an export - but there was definitely intent. How many times have you, during the process of doing work, said "Oh hey, I think this might be a nice feature to work on for the next 3-4 days" and just not done any of your normally scheduled work?
- jethro_tell 7y ago
  That shouldn't be possible though. If you classify data in your database as pii and classified but then let any one with door access in there you're really doing something wrong. To add to that, what are the chances the team that owns that database didn't notice a significant uptick in requests from a new caller? I'd say a program like this is probably enough to tip over a service if it's not coordinated.
  - munk-a 7y ago
    I agree that it'd be nice if devs couldn't see this data at all, but perhaps the backups are less restricted than the live DB or it's standard practice for folks to run BI/marketing queries against prod there? Twitter probably has a serious BI department so I'd imagine they have ways of getting at various data vectors for analysis, and BI folks usually complain loudly when you block data that let's them "cohort" usage data to the point where it can be easier to just open the door for them on a read-only instance.
jethro_tell 7y ago
I work at a big company, every field in a database of user info is tagged with a data classification. Everyone knows the rules about which classification can be used for what. Products go through a review of data use and storage (i.e. some data should be encrypted, and encrypted correctly not some hand rolled hash) If twitter doesn't have common data classifications and everyone with a keyboard and an idea has access to the database without review, they are probably pushing criminally negligent. I understand at a company with just a couple employees. This shit happens all the time. I don't understand it with a publicly traded company that would be in big trouble with a class action data lawsuit.

hazelnut 7y ago

Pretty sure there was a product manager who drove the project. This person needs to understand where the data is coming from and where it flows to. This person is also responsible for checking with legal to see that everything is in line. Besides that, there're cycles of product reviews with directors and VP's.

munk-a 7y ago

I could buy that marketing/sales snuck around dev's oversight to manually get a copy of that to use - possibly by enlisting a poor junior dev to do an export - but there was definitely intent. How many times have you, during the process of doing work, said "Oh hey, I think this might be a nice feature to work on for the next 3-4 days" and just not done any of your normally scheduled work?

jethro_tell 7y ago
That shouldn't be possible though. If you classify data in your database as pii and classified but then let any one with door access in there you're really doing something wrong. To add to that, what are the chances the team that owns that database didn't notice a significant uptick in requests from a new caller? I'd say a program like this is probably enough to tip over a service if it's not coordinated.
- munk-a 7y ago
  I agree that it'd be nice if devs couldn't see this data at all, but perhaps the backups are less restricted than the live DB or it's standard practice for folks to run BI/marketing queries against prod there? Twitter probably has a serious BI department so I'd imagine they have ways of getting at various data vectors for analysis, and BI folks usually complain loudly when you block data that let's them "cohort" usage data to the point where it can be easier to just open the door for them on a read-only instance.

jethro_tell 7y ago

I work at a big company, every field in a database of user info is tagged with a data classification. Everyone knows the rules about which classification can be used for what. Products go through a review of data use and storage (i.e. some data should be encrypted, and encrypted correctly not some hand rolled hash) If twitter doesn't have common data classifications and everyone with a keyboard and an idea has access to the database without review, they are probably pushing criminally negligent. I understand at a company with just a couple employees. This shit happens all the time. I don't understand it with a publicly traded company that would be in big trouble with a class action data lawsuit.