4 ms·

The issue is the information was made public I don't know why people are upset about it being stored in git. If I store PII in a database and then that database

The issue is the information was made public I don't know why people are upset about it being stored in git. If I store PII in a database and then that database gets leaked am I dumb for storing my PII in a database or for letting it get leaked? Seems kinda obvious to me.

kahrl 5y ago
People are upset that it is being stored in git because it shows how moronically negligent this company is about the security of its customer data. Storing any secrets in version control is one of the stupidest things you can do, and the fact that this was allowed to happen reflects poorly on the entire company.
NovemberWhiskey 5y ago
If you really, really want to manage PII in a private git repo, you can probably do it - in approximately the same that you can probably make API-first web services in FORTRAN if you really, really want to. The fact remains that your choice of tooling hurts you. Putting PII into Github (even if the repo is private) is catastrophic. You just made Github into a third-party data processor by accident. Good luck explaining that from a CCPA/GDPR perspective.
selfup 5y ago
The idea is that git is for source code/configs. Not data. Data gets stored in databases. Secrets/Tokens are stored in secret stores/vaults. Blobs like PDFs/etc should be stored in blob storage. If a diff is not useful, don't store it in git. Doing a code review on PII is not useful. You also shouldn't add packages in a git repo if they can be downloaded immutably in CI/locally. It adds a lot of space. Every commit is a snapshot + new code. Commits are not diffs.
- iudqnolq 5y ago
  I disagree. They were using it as sample data, when they should have used synthetic sample data. It's totally sane to check-in a JSON (or similar) file with a few hundred [edit: non-confidential] samples so that you can run integration tests without needing to set anything else up. Similarly, I keep small pdf manuals in git. I add them in their own commit, which doesn't have a useful diff. In exchange, they're always there and I don't need to spin up some special one-off system.
  - selfup 5y ago
    Fixture data (for testing) is fine. Sometimes a few lines is the minimum viable to test, sometimes a hundred or so lines of JSON. As long as the data is being optimized for testing and you aren't just storing a GB because it's easier. I was replying to a comment asking why PII data shouldn't be stored in git vs a database. Just tried to give a general rundown of how to think about the tools. Totally agreed that the data should have been synthetic/scrubbed and that a small sample set (large enough to validate) would be ok. Sometimes a PDF or a few pictures are necessary (or just much simpler). I get that. I have seen some repos with an unreasonable amount of PDFs/Docs/pictures/etc.. and that's when a script that copies them into a gitignored directory from someplace (Dropbox/S3/etc..) is a better fit.

kahrl 5y ago

People are upset that it is being stored in git because it shows how moronically negligent this company is about the security of its customer data. Storing any secrets in version control is one of the stupidest things you can do, and the fact that this was allowed to happen reflects poorly on the entire company.

NovemberWhiskey 5y ago

If you really, really want to manage PII in a private git repo, you can probably do it - in approximately the same that you can probably make API-first web services in FORTRAN if you really, really want to. The fact remains that your choice of tooling hurts you. Putting PII into Github (even if the repo is private) is catastrophic. You just made Github into a third-party data processor by accident. Good luck explaining that from a CCPA/GDPR perspective.

selfup 5y ago

The idea is that git is for source code/configs. Not data. Data gets stored in databases. Secrets/Tokens are stored in secret stores/vaults. Blobs like PDFs/etc should be stored in blob storage. If a diff is not useful, don't store it in git. Doing a code review on PII is not useful. You also shouldn't add packages in a git repo if they can be downloaded immutably in CI/locally. It adds a lot of space. Every commit is a snapshot + new code. Commits are not diffs.

iudqnolq 5y ago
I disagree. They were using it as sample data, when they should have used synthetic sample data. It's totally sane to check-in a JSON (or similar) file with a few hundred [edit: non-confidential] samples so that you can run integration tests without needing to set anything else up. Similarly, I keep small pdf manuals in git. I add them in their own commit, which doesn't have a useful diff. In exchange, they're always there and I don't need to spin up some special one-off system.
- selfup 5y ago
  Fixture data (for testing) is fine. Sometimes a few lines is the minimum viable to test, sometimes a hundred or so lines of JSON. As long as the data is being optimized for testing and you aren't just storing a GB because it's easier. I was replying to a comment asking why PII data shouldn't be stored in git vs a database. Just tried to give a general rundown of how to think about the tools. Totally agreed that the data should have been synthetic/scrubbed and that a small sample set (large enough to validate) would be ok. Sometimes a PDF or a few pictures are necessary (or just much simpler). I get that. I have seen some repos with an unreasonable amount of PDFs/Docs/pictures/etc.. and that's when a script that copies them into a gitignored directory from someplace (Dropbox/S3/etc..) is a better fit.