4 ms·

I think the industry is going to soon look back on building with Wild West open-source repos like we looked back on not having absolutely everything running on

• Kaytaro 2y ago
  If you include commercial offerings Red Hat has offered this for awhile, and many semi-successful startups have tried creating a business model solving this.
• theteapot 2y ago
  > npm is a package manager for the JavaScript programming language maintained by npm, Inc., a subsidiary of GitHub. -- [1] and Microsoft own Github so Microsoft is the provider? Pretty sure they're running malware scanners over NPM constantly at the least. NPM also has (optional) provenance [2] to a Github build workflow which is as strong as being "assured" by Google IMO. Only problem is it's optional. [1]: https://en.wikipedia.org/wiki/Npm https://en.wikipedia.org/wiki/Npm [2]: https://github.blog/security/supply-chain-security/introducing-npm-package-provenance/ https://github.blog/security/supply-chain-security/introduci...
• giantg2 2y ago
  Any reasonable company already knows this and sets up a proxy repo of scanned/approved versions (this is important for licensing too).
  • 0cf8612b2e1e 2y ago
    How much is that automated scanning worth? Sure, we have mirrored repos, but I assume the malware authors pre test their code on a suite of detectors in CI. So infected packages will happily be mirrored internally for consumption.
    • poincaredisk 2y ago
      >I assume the malware authors pre test their code on a suite of detectors in CI. Maybe some do, but you give the average malware developer way too much credit.
    • thibaut_barrere 2y ago
      It is worth a fair bit. If you control the mirroring you can ensure the malware is flagged but not deleted, so forensics can assess how much damage has been done or would have been done, for instance.
    • giantg2 2y ago
      At least at my company, I think someone at least has to approve/verify the scan results. Of course it's still a risk, but so are external emails, vendor files, and everything else.
    • ohgr 2y ago
      Bugger all. We had something go straight through.
    • delusional 2y ago
      Not much. as you say, static scanning is pretty much a dead end strategy. Exploiters have long since realized that you can just run the scan yourself and jiggle the bytes around to evade the signature detection.
    • feross 2y ago
      Totally agree. Most companies using mirrors or proxies like Artifactory aren’t getting much real protection. - They cache packages but don’t analyze what’s inside. - They scan or review the first version, then auto-approve every update after that. - They skip transitive deps — and in npm, that’s 79 on average per package. - They rely on scanners that claim to detect supply chain attacks but just check for known CVEs. The CVE system doesn’t track malware or supply chain attacks (except rarely), so it misses 99%+ of real threats. Almost everything on the market today gives a false sense of security. One exception is Socket — we analyze the actual package behavior to detect risks in real time, even in transitive deps. https://socket.dev https://socket.dev (Disclosure: I’m the founder.)
  • pletnes 2y ago
    Not from what I’ve seen. What are the relevant products in this space? Can’t expect every random company to set up package scanning from scratch.
    • tsm 2y ago
      I worked for an IBM acquiree 13 years ago and as part of the "Blue-washing" process to get our software up to IBM spec we had to use their proprietary tools for verifying our dependencies were okay.
    • chrisweekly 2y ago
      JFrog / Artifactory is one very common provider of private npm registries. There are a ton of security-scan vendors out there (mend/whitesource, socket, black duck...)
    • deleted 2y ago
      [deleted]
    • giantg2 2y ago
      Well then I wouldn't expect to do business with every random company. TPRM is a big issue today, so I wouldn't expect any company not performing basic due diligence to service.
  • swatcoder 2y ago
    You're absolutely right, but you've just asserted that almost all companies making software are unreasonable. Distressingly, doing what you suggest remains the exception by orders of magnitude. Very few people have internalized why it's necessary and few of those have the political influence in their organizations to make it happen.
    • nukem222 2y ago
      [flagged]
• BrenBarn 2y ago
  So then instead of knowing nothing, we'll know that Google wants us to use it, which . . . is a different problem. :-)
• donnachangstein 2y ago
  > like we looked back on not having absolutely everything running on HTTPS in the Snowden era. Apples and oranges and this is far, far worse. You can absolutely ship signed, trusted code over standard HTTP. Microsoft did this for years and Debian and OpenBSD to name a few still do. HTTPS does not assure provenance of code. Anyone who doesn't understand this is very misinformed about what HTTPS does and doesn't do.
  • genewitch 2y ago
    I think this depends on one's definition of "code"
  • tedd4u 2y ago
    Sorry, I wasn't clear. I meant only in the general sense of in the not too far past, the industry was content with a huge hole like only running login under HTTPS and no site traffic, which in hindsight seems insane. What I mean is the situation (explored in the rest of this thread) where many in the industry seem to be content with consuming code extensively from public repos without many obstacles to prevent a supply-chain attack. What I'm saying is that soon the industry will probably look back on this in the same way: "what were we thinking!?"
• ohgr 2y ago
  Based on the staff I see at the average technology company I wouldn’t expect this to get any better any time soon. The state of things is definitely declining.
• the8472 2y ago
  This is a coordination failure. We have ways to distribute the source, but not the reviews. Every time someone does any level of reviewing that should be publishable too.
  • karlding 2y ago
    Things like cargo-crev [0] or cargo vet [1] aim to tackle a subset of that problem. There’s also alternate implementations of crev [2] for other languages, but I’m not sure about the maturity of those integrations and their ecosystems. [0] https://github.com/crev-dev/cargo-crev https://github.com/crev-dev/cargo-crev [1] https://mozilla.github.io/cargo-vet/ https://mozilla.github.io/cargo-vet/ [2] https://github.com/crev-dev/crev/ https://github.com/crev-dev/crev/
• nextts 2y ago
  How does https help with the problems Snowden uncovered? You don't run on https, https just does in transit encryption between 2 points of the service architecture. That is why you can (could?) slap cloudflare atop your http only site and get a padlock!
  • reactordev 2y ago
    Because one of the methods reported was scanning http packets. Easily read without ssl from any hop in the chain. More importantly, he blew the lid off the fact that governments had access to this via the very ISP’s everyone relies on for telecom. By making everything TLS, they can look all they want but they can’t read it. You could do tls offloading at your load balancer but then you have to secure your entire network starting with your isp. For some workloads, this is fine, you aren’t dealing with super sensitive data. For others, you are violating compliance.
  • tedd4u 2y ago
    I'm referring to programs like MUSCULAR [1] and PRISM [2] where NSA was tapping inter- and intra-datacenter traffic of major internet communications platforms like Gmail, Yahoo Mail, Facebook etc. At the time, that kind of traffic was not encrypted. It was added in a hurry after these revelations. [1] https://en.wikipedia.org/wiki/MUSCULAR https://en.wikipedia.org/wiki/MUSCULAR [2] https://en.wikipedia.org/wiki/PRISM https://en.wikipedia.org/wiki/PRISM
    • nextts 2y ago
      Oh yeah where I work we run encryption between ec2s. But I don't think it is https. Probably more low level (todo: read up on how it works!)
      • tedd4u 2y ago
        I saw it done with encrypted MPLS at a very large tech company.
• fc417fc802 2y ago
  > Wild West open-source repos There's a deeper issue though. I frequently have difficult getting things to build from source in a network isolated environment. That's after I manually wrangle all the dependencies (and sub-deps, and sub-sub-deps, and ...). Even worse is something like emscripten where you are fully expected to run `npm install`. Any build process that depends on network access is fundamentally broken as far as I'm concerned.
  • no_wizard 2y ago
    Which is nearly all of them, except perhaps C/C++, that I can think of, in terms of languages broadly adopted You can cache and/or emulate the network to go offline but fundamentally a fresh build in most languages will want to hit a network at least by default
    • robinsonb5 2y ago
      In my world (VHDL/Verilog and some C/C++) there's a difference between the "fetch" and "build" steps. It's perfectly reasonable for the fetch step to require network access; the build step should not. The real problem is that some language ecosystems conflate those two steps.
      • fc417fc802 2y ago
        I'm mostly on board with that dichotomy except that I think it's also important that all fetched artifacts either come from a VCS or are similarly cryptographically versioned and all historical versions made available in a reliable manner.
        • robinsonb5 2y ago
          Yes, absolutely - I can't disagree with that.
    • fc417fc802 2y ago
      > at least by default Even in C/C++ after changing the relevant parameters to non-default values things often break. It seems those configurations often go untested. Google managed repos are a nice exception to this. Clearly documented commit hashes for all dependencies for a given release.
• feross 2y ago
  Totally agree — we’re going to look back and wonder how we ever shipped code without knowing what was in our dependencies. Socket is working on exactly this: we analyze the actual code of open source packages to detect supply chain risks, not just known CVEs. We support npm, PyPI, Maven, .NET, Rubygems, and Go. Would love to hear which ecosystems you care about most. (Disclosure: I’m the founder. https://socket.dev https://socket.dev)