5 ms·

PII data should never touch github, ever. Good job you're not in the EU with GDPR as this process alone implies a serious lack of security due diligence. "Ada

by xd 5y ago

PII data should never touch github, ever. Good job you're not in the EU with GDPR as this process alone implies a serious lack of security due diligence. "Adafruit team began the forensic process" - Checking audit logs is hardly a forensic process and the rest of the speel about "privacy lawyers and legal experts" feels like a poor effort to regain some trust.

andi999 5y ago
Why shdnt it touch github? If you think it is not secure than also your companies source code shdnt touch it, shouldnt it? Or is there a different reason?
- sdesol 5y ago
  Trade secret and PII are two different things. Trade secret is something you can risk. PII is not something most governments will allow you to risk.
- CSSer 5y ago
  You never know where that repo is going to go. It could be private today and public tomorrow with a totally different license. Git should be thought of as sticky in the sense that it's intentionally designed to preserve a history of everything for all time in the most efficient way possible. Anyone who's ever had to scrub an API key they accidentally pushed or optimize a monorepo can attest to this.
  - MattRix 5y ago
    If it’s your own repo presumably you know exactly where that repo is going to go. The vast majority of private repos are never going to go public. I have many repos for projects that there is no chance I would ever make public, they’re not that kind of project. And even if I really did want them to be public, I would make a new repo for them instead.
    - CSSer 5y ago
      Yes, but they're your repos, not an organization's. I'm certainly not going to question your self-knowledge. You may know that, and if you work with one or a couple other people closely they might know that too. The challenge is when you grow beyond that or don't always have the chance to communicate about these types of things, especially when they seem very obvious but might not be.
- dec0dedab0de 5y ago
  it would be ok to store it in github if it were encrypted, but storing encrypted data in git is not very efficient.
- BrS96bVxXBLzf5B 5y ago
  The laws account for limited retention time, and can be retroactively changed to make it so that companies have to get rid of historical data. Versioned history is supposed to be immutable and changing it is a pain.
- xd 5y ago
  It's personal data, would you want your personal data being uploaded to github - in this instance so someone can learn data analysis?
- 3np 5y ago
  Even in a private repo, putting it on GH means sharing it with the third-party Microsoft, which is illegal in many countries.
  - iudqnolq 5y ago
    Practically speaking, I don't think that's true. Many EU companies use Microsoft's cloud offerings.
    - 3np 5y ago
      If PII transfer to MS servers is part of that, there are certain situations and necessary steps to take to make that not a violation. Moving from one provider to the other can not be done legally without properly informing the individuals involved and (depending on the nature of the data and its purpose and use) getting the right explicit consent. Taking an extract including PII (names, usernames, IP addresses, or email addresses for example) from your customer prod db and dumping it in a csv in a private repo on GH is most likely a violation unless you have prior explicit consent for that very purpose and use. This is true even if it's "pseudo-anonmized" in a way that the original PII can be deduced by combination with other datasets. Finally, I wouldn't be surprised if many of those companies are operating illegally. Drinking and driving doesn't become legal just because a threshold of people start doing it. There is a lot of ignorance (willful or not) among EU businesses even today.
      - iudqnolq 5y ago
        I thought you needed consent for usage, not for technical details? As in "we use your purchase history to generate recommendations", not "we store you purchase history on these systems and run these algorithms on it". Are you arguing that if my colo provider burned down I'd need to get explicit informed consent from every user before restore a db backup somewhere else?
        3np 5y ago
        > Are you arguing that if my colo provider burned down I'd need to get explicit informed consent from every user before restore a db backup somewhere else? AIUI, it could swing either way depending on several factors, such as: the format and usage of the data; how and where the data is transferred, processed, stored and exposed; what access and role the colo provider has (are you purely renting a dedicated server in a DC with FDE that you unlock remotely with an HSM or is the data processed by one of their managed services?); how the consent you already acquired was formulated. If the colo provider has an outsourced support engineer in Asia looking at logs/coredump or temporarily transferring a backup where the PII appears, that would constitute a transfer, for example, and full compliance needs to be guaranteed throughout. It's years since I considered myself to have a clear and deep understanding of it and it's gotten a bit fuzzy since, so someone else might chime in with a more clear answer.
simonw 5y ago
How is checking audit logs not a forensic process? Surely that's what audit logs are for?
- bastardoperator 5y ago
  Exactly, without logs what else exists? Git itself IS an auditing tool.
- xd 5y ago
  It's not a "process" it's looking at logs to see if someone accessed X in a certain time frame. The words "forensic process" in this instance is used to make it appear there is something more involved going on.. if say there was 1 access to the data how will this "forensic process" play out tracing who accessed the data and whom they passed it on to - they don't know and have no "forensic process" to know.
xd 5y ago
I'm the OP on this comment. The fact I've been 50/50 voted for what is quite frankly common sense is disappointing. Just ask yourself a simple question; would you want your personal information uploaded to GitHub for someone to learn data analysis and would you be happy about that?
- kahrl 5y ago
  I got downvoted for saying you should never store secrets in version control. Insane. I think some sophomores from /r/programmerhumor may be visiting.
  - xd 5y ago
    I hope it's just people trying to cover for Adafruit.. which is fair enough and I get that. They have done way more good than this little hiccup - but it's so important to not play around with real PPI data or be exposing that or secrets to version control.
  - morelisp 5y ago
    PD and secrets are not the same thing. You can be following all technical best practices regarding secret management and still fuck this one up. You got downvoted because storing encrypted secrets is fine.
    - xd 5y ago
      "You got downvoted because storing encrypted secrets is fine." Storing where?
      - morelisp 5y ago
        ... in version control. The response is intended to be read in context of the sentences above it.
        xd 5y ago
        The security and integrity of data shouldn't rely on itself to prove such.. imagine trying to store a checksum within the data it represents for instance.

by xd 5y ago

andi999 5y ago

Why shdnt it touch github? If you think it is not secure than also your companies source code shdnt touch it, shouldnt it? Or is there a different reason?

sdesol 5y ago
Trade secret and PII are two different things. Trade secret is something you can risk. PII is not something most governments will allow you to risk.
CSSer 5y ago
You never know where that repo is going to go. It could be private today and public tomorrow with a totally different license. Git should be thought of as sticky in the sense that it's intentionally designed to preserve a history of everything for all time in the most efficient way possible. Anyone who's ever had to scrub an API key they accidentally pushed or optimize a monorepo can attest to this.
- MattRix 5y ago
  If it’s your own repo presumably you know exactly where that repo is going to go. The vast majority of private repos are never going to go public. I have many repos for projects that there is no chance I would ever make public, they’re not that kind of project. And even if I really did want them to be public, I would make a new repo for them instead.
  - CSSer 5y ago
    Yes, but they're your repos, not an organization's. I'm certainly not going to question your self-knowledge. You may know that, and if you work with one or a couple other people closely they might know that too. The challenge is when you grow beyond that or don't always have the chance to communicate about these types of things, especially when they seem very obvious but might not be.
dec0dedab0de 5y ago
it would be ok to store it in github if it were encrypted, but storing encrypted data in git is not very efficient.
BrS96bVxXBLzf5B 5y ago
The laws account for limited retention time, and can be retroactively changed to make it so that companies have to get rid of historical data. Versioned history is supposed to be immutable and changing it is a pain.
xd 5y ago
It's personal data, would you want your personal data being uploaded to github - in this instance so someone can learn data analysis?
3np 5y ago
Even in a private repo, putting it on GH means sharing it with the third-party Microsoft, which is illegal in many countries.
- iudqnolq 5y ago
  Practically speaking, I don't think that's true. Many EU companies use Microsoft's cloud offerings.
  - 3np 5y ago
    If PII transfer to MS servers is part of that, there are certain situations and necessary steps to take to make that not a violation. Moving from one provider to the other can not be done legally without properly informing the individuals involved and (depending on the nature of the data and its purpose and use) getting the right explicit consent. Taking an extract including PII (names, usernames, IP addresses, or email addresses for example) from your customer prod db and dumping it in a csv in a private repo on GH is most likely a violation unless you have prior explicit consent for that very purpose and use. This is true even if it's "pseudo-anonmized" in a way that the original PII can be deduced by combination with other datasets. Finally, I wouldn't be surprised if many of those companies are operating illegally. Drinking and driving doesn't become legal just because a threshold of people start doing it. There is a lot of ignorance (willful or not) among EU businesses even today.
    - iudqnolq 5y ago
      I thought you needed consent for usage, not for technical details? As in "we use your purchase history to generate recommendations", not "we store you purchase history on these systems and run these algorithms on it". Are you arguing that if my colo provider burned down I'd need to get explicit informed consent from every user before restore a db backup somewhere else?
      - 3np 5y ago
        > Are you arguing that if my colo provider burned down I'd need to get explicit informed consent from every user before restore a db backup somewhere else? AIUI, it could swing either way depending on several factors, such as: the format and usage of the data; how and where the data is transferred, processed, stored and exposed; what access and role the colo provider has (are you purely renting a dedicated server in a DC with FDE that you unlock remotely with an HSM or is the data processed by one of their managed services?); how the consent you already acquired was formulated. If the colo provider has an outsourced support engineer in Asia looking at logs/coredump or temporarily transferring a backup where the PII appears, that would constitute a transfer, for example, and full compliance needs to be guaranteed throughout. It's years since I considered myself to have a clear and deep understanding of it and it's gotten a bit fuzzy since, so someone else might chime in with a more clear answer.

simonw 5y ago

How is checking audit logs not a forensic process? Surely that's what audit logs are for?

bastardoperator 5y ago
Exactly, without logs what else exists? Git itself IS an auditing tool.
xd 5y ago
It's not a "process" it's looking at logs to see if someone accessed X in a certain time frame. The words "forensic process" in this instance is used to make it appear there is something more involved going on.. if say there was 1 access to the data how will this "forensic process" play out tracing who accessed the data and whom they passed it on to - they don't know and have no "forensic process" to know.

xd 5y ago

I'm the OP on this comment. The fact I've been 50/50 voted for what is quite frankly common sense is disappointing. Just ask yourself a simple question; would you want your personal information uploaded to GitHub for someone to learn data analysis and would you be happy about that?

kahrl 5y ago
I got downvoted for saying you should never store secrets in version control. Insane. I think some sophomores from /r/programmerhumor may be visiting.
- xd 5y ago
  I hope it's just people trying to cover for Adafruit.. which is fair enough and I get that. They have done way more good than this little hiccup - but it's so important to not play around with real PPI data or be exposing that or secrets to version control.
- morelisp 5y ago
  PD and secrets are not the same thing. You can be following all technical best practices regarding secret management and still fuck this one up. You got downvoted because storing encrypted secrets is fine.
  - xd 5y ago
    "You got downvoted because storing encrypted secrets is fine." Storing where?
    - morelisp 5y ago
      ... in version control. The response is intended to be read in context of the sentences above it.
      - xd 5y ago
        The security and integrity of data shouldn't rely on itself to prove such.. imagine trying to store a checksum within the data it represents for instance.