5 ms·

Detecting plain SSN numbers wouldn't be difficult with a combination of regex, machine learning and human verification. Even if hackers could just encode the S

• BoorishBears 7y ago
  Are you seriously entertaining the idea that Github should pour resources into developing a SSN detection pipeline? Why don't they also flag repositories with lots of names, those are PII. Too many addresses? That's a red flag. Why would you even for a second entertain this nonsense? Non-technical people who think this: https://github.com/Hack-with-Github/Awesome-Hacking https://github.com/Hack-with-Github/Awesome-Hacking is a smoking gun to prove Github as a company is supporting hackers are the ones putting forth this idea, no one with any technical knowledge should come close to supporting this.
  • antpls 7y ago
    Per EU's GDPR, and maybe the future California Consumer Privacy Act, yes : GitHub should do everything they can to preserve people's privacy. Including maybe flagging and reviewing projects which process PII. GitHub is popular enough that less technical people also browse it, in particular young people looking to learn.
    • cljs-js-eval 7y ago
      If I accept that technical difficulty and infeasibility is no defense, then I want that standard applied to lawyers as well. Overly litigious firms causing rising legal costs across industries? I don't care if it's hard to solve, the onus for fixing it is on the firms, they figure it out or face penalties. Perhaps in the interests of helping the disenfranchised we could institute something like what real estate has, where banks are required to sell a quota of mortgages in certain areas regardless of the financial viability. Lawyers could be forced to seek out clients they would ordinarily never entertain due to the risk of loss. If would stink for them, but what do I care? That's their problem.
    • danShumway 7y ago
      > everything they can Just the other day my mom texted my address to a friend that wanted to mail me something. I want to know why Verizon didn't stop that from happening! Verizon should be censoring all addresses from text messages, but I suppose it could allow them after a manual review process. HN is guilty here as well: 1600 Pennsylvania Ave NW, Washington, DC 20500 Why isn't the above address automatically censored?
      • antpls 7y ago
        The message from your mom is private and isn't supposed to be read by Verizon for Verizon to provide its service. If you don't want your mom to send PII about you to other people without your consent, and it incurs damage, you can surely sue your own mom.
        • deleted 7y ago
          [deleted]
        • BoorishBears 7y ago
          I can't tell if you actually mean what you're saying, or if this is just the time-honored HN tradition of being a contrarian for the sake of being a contrarian.
  • karma20 7y ago
    Agree. This kind of DLP-like scanning is totally infeasible at GitHub scale. In particular, one excerpt from the complaint reads like output from a noisy scanner that has detected “exposed SSNs” in client-side code: > [...] Social Security numbers are readily identifiable: they are nine digits in the XXX-XX-XXXX sequence.
• cgb223 7y ago
  Wait... Given enough SSNs as training data, could someone make an ML model that churns our mostly valid SSNs? Because that could be really really bad
  • JumpCrisscross 7y ago
    > Given enough SSNs as training data, could someone make an ML model that churns our mostly valid SSNs? Most SSNs are deterministically derived. Given a single valid SSN, you can often generate millions of valid preceding values.
    • ageitgey 7y ago
      That changed in 2011. Now they are random. But it is still true for older numbers. https://www.ssa.gov/employer/randomization.html https://www.ssa.gov/employer/randomization.html
  • pliny 7y ago
    Don't you need at least one other piece of identifying info to make that useful? Like, I could churn out mostly valid gmail passwords all day, but I have no idea which users have those passwords.
  • jakelazaroff 7y ago
    No need to get that involved: for (let i = 0; i < 1000000000; i++) console.log(i); Every single valid social security number, and several (e.g. 000-00-0000) that probably aren't. You can store the whole thing in about a gig.
  • throwawaywego 7y ago
    Worse. You can make the ML model spit out the SSNs it was trained on. That's a problem when you can't manually curate billions of documents. If you didn't look, you wouldn't even know they were there.
• bufferoverflow 7y ago
  You missed the whole point of the comment you replied to. You can detect the format of the SSN, but you can't reliably tell that it actually is a SSN, and not some other type of identifier.
  • colejohnson66 7y ago
    Exactly. At Target, their internal SKUs follow the format xxx-xx-xxxx. That happens to be the same as the SSN format
  • throwawaywego 7y ago
    Can you yourself reliably detect if something is a list of SSN's or some other type of identifier? I think you can. Now when we have neural nets capable of distinguishing 100s of different dog breeds, we still have to trip on the most basic and structured type of entity extraction? No. A simple regex, combined with heuristics, and a linear model on top can reliably detect SSN's. Just that there will be a few false positives (no matter if you automate this, or do this manually) does not mean it is a Herculean technical challenge to do this.
• ben509 7y ago
  Any automated detection scheme can be thwarted by encryption as trivial as ROT13. You could leave a comment in README: "To decrypt, just run 'l33t-xor-crypt --key bob --file hacked-data.txt'" and Github's scanner is none the wiser. Meanwhile, it will be finding all the false positives already mentioned, especially test suites with fake data.
  • giancarlostoro 7y ago
    Hell in Python its: >>> import codecs >>> codecs.encode('foobar', 'rot_13') 'sbbone'
• danShumway 7y ago
  In regards to your edit 1, cost benefit analysis isn't just about a company's bottom line, it's also about the types of architecture and services that are allowed to exist online. Github is not the primary way that PII is leaked online, you're thinking of Pastebin. Should Pastebin be allowed to exist? Should the Open Source developers behind Wordpress and Ghostery be liable for not scrubbing PII off of websites? The "cost" here isn't really money, it's a social cost. It's regular people's access to services that, on net, make their lives much better. I'm not worried about Microsoft making less money, I'm worried about damaging one of the best software repository services online and making it less useful to ordinary developers. In regards to your edit 2, Github is a collection of software repositories. Say you replace all social security numbers with <number-redacted>. If I'm a screen scraper, that doesn't block me -- I can just clone the repository. There is no way that Github can block this unless they delete or replace the numbers in the actual uploaded code, which would obviously be a bad idea. In regards to your edit 3, are you planning on linking Github repositories to real-life identities? Probably not, since that would be a huge privacy problem, and you're trying to improve privacy. So what happens when ownership transfers? Or when someone makes an innocuous repo and then later on pushes PII? Note that this is not an abstract problem, we've seen multiple malware attacks on packages, browser extensions, and phone apps that boiled down to, "it looked safe, and then somebody stole the credentials or just decided to push malicious code." I don't understand how a review process would help here unless it was a review process on literally every commit.
  • antpls 7y ago
    > Should Pastebin be allowed to exist? Yes, because as far as I know, Pastebin doesn't process sent data to provide its service. They just store it, they could simply store it encrypted and say they can't access data. On the other hand, GitHub process repositories to provide further services. They already read the sent data. > The "cost" here isn't really money, it's a social cost. It's regular people's access to services that, on net, make their lives much better. What about the cost for people whose data leaked? Are you saying we should treat them as collateral damage for others to have an "accessible" service? That sounds irresponsible. > Should the Open Source developers behind Wordpress and Ghostery be liable for not scrubbing PII off of websites? The "cost" here isn't really money, it's a social cost. No, as long as they don't operate public instances of those softwares, they don't need to include all the tooling for detecting and handling PII. However, anyone who wishes to operate those softwares with a way to publicly sent data to it should implement privacy safeguards. > ordinary developers. In 2000s, we were the "ordinary developers". All the leaks happening today is because we didn't care enough about that aspect of software engineering. The new wave of developers should always have privacy in mind before writing software for businesses, and to change the mentality, changing a platform like GitHub would help spreading a new culture of security and privacy-oriented mindset. > In regards to your edit 2, Github is a collection of software repositories. Say you replace all social security numbers with <number-redacted>. If I'm a screen scraper, that doesn't block me -- I can just clone the repository. There is no way that Github can block this unless they delete or replace the numbers in the actual uploaded code, which would obviously be a bad idea. Github is an opinionated centralized collections of software repositories. It should be simpler : If your repository is private, then there should be no filter or review at all. If you repository is public or becomes public, then it should be draconian about what is posted and shared. No hate speech, no PII, etc. If there is a positive detection, then the repository should automatically turn private or be suspended until it's resolved. > So what happens when ownership transfers? Unsolved issue so far, I don't have proposition about it > I don't understand how a review process would help here unless it was a review process on literally every commit. Regarding publicly sharing information on famous platforms like GitHub, it should be mandatory IMHO. I would happily trade a few false positives for a better peace of mind.
• throwawaywego 7y ago
  What I think happened: Someone contacted Capital One by email to responsibly disclose to them that there were SSN's and other data on a Gist. That person found them with a simple crawler or search. Then Cap 1 thought: If some rando can find this after a lot of damage has been done, why can't Github find these seconds after upload? And, really, there is no technical excuse. It is perfectly possible to do this, and lots of big companies do this (or hire security companies to do this for them). Mention their name on some deep web hacking forum, a pastebin, or inside Github code, and somewhere an alarm goes off. Github could (and should) warn if a user uploads loads of PII-like data. For the cost of running a search server and a few moderators. "Are you sure you want to upload your AWS credentials in a public repository?". Github is somewhere halfway between moderated and a content platform. They already have a history of taking down repositories if they link to PII data (or infringe copyright, or damage U.S. national security): http://web.archive.org/web/20180619172528/https://github.com/antiboredom/ice-linkedin http://web.archive.org/web/20180619172528/https://github.com... so not acting on this specific repo with SSN numbers could be seen as a poor/shoddy job on their part. Github is certainly in the dominant position to mitigate spread of PII data, so they should have their stuff in order.