11 ms·

The world needs a software bill of materials

135 points by kiyanwang 6y ago

BrianOnHN 6y ago
What's with the images?
- layer8 6y ago
  My first guess was autostereograms, but that doesn’t seem to be the case. Maybe some steganography?
- the_af 6y ago
  I was going to comment the same. I really like them! They look as if displayed on an old CRT with color artifacts.
you_are_naive 6y ago
Blah blah blockchain blah blah have an alternative compliance questionnaire list which companies outsource already mandatory for every software blah blah as a way to check maturity of the company. I sometimes wonder if this is how industries end up not innovating or solving obvious problems for decades because they get strangled with bureaucracy which doesn't solve the original problem highlighted in his own example (vendor choosing to ignore to patch a vulnerability).
anoncow 6y ago
Why does this read like an anti-opensource propaganda piece aimed at big business?
wpietri 6y ago
At least on first reading, I find this unpersuasive. He correctly lists a variety of problems. But he doesn't explain how his proposed solution, listing all the components of a technological product, would make a practical difference. Creating a list is valuable only if people a) read the list, b) recognize problems, and c) do something based on that. And for some of the examples he gives, it seems pretty obvious to me that an SBOM wouldn't help. The Equifax breach, for example. They knew [1] that they needed to upgrade Apache Struts. Somebody was supposed to make the upgrade. They just didn't do it. Who would an SBOM help here? Since it's a consumer-facing website, the only people who weren't informed were consumers. So is he proposing to make public the SBOM for every website? I'm not sure that on balance that helps security. [1] https://www.csoonline.com/article/3444488/equifax-data-breach-faq-what-happened-who-was-affected-what-was-the-impact.html https://www.csoonline.com/article/3444488/equifax-data-breac...
- mikepurvis 6y ago
  Wouldn't having to advertise your out of date dependencies help to shame companies into upgrading on a reasonable schedule? So that upgrades are actually a priority and not just a thing that happens when literally everything else is already done?
  - sokoloff 6y ago
    If that became a problem, companies intending to skirt the disclosure would fork and “maintain” private branches of dependencies such that it couldn’t be determined if they were out of date.
    - spacemanmatt 6y ago
      True story: They already do. I know one company that forked Ruby, and would likely claim every library they run under it is thereby forked for the sake of reporting.
      - mattcwilson 6y ago
        Ok, but there would still be some point of contact where an SBOM would show “company X fork of Ruby, company X fork of package Z” etc, right? And then the choice gets back to how much to trust company X and package Z, weighed against alternative solutions.
        mannykannot 6y ago
        Unless penalties for doing so are legislated, they need not claim that the fork they are actually using is a fork of anything; they could treat it as if part of their proprietary code (though they might be in violation of license agreements if they do this.)
        sokoloff 6y ago
        It seems like it could equally be “internal package UUID1 version X, internal package UUID2 version Y” if malicious compliance and/or industrious laziness was the goal.
pdimitar 6y ago
No SBOM will help you if the people know they have to act but they don't -- out of malice, bureaucratic slowdown, policy restriction and what-have-you. If you don't have hardware and software that can't be tampered with and that automatically apply / enforce the SBOM, then it is essentially worthless.
- jacques_chester 6y ago
  I disagree. SBOMs will help to create useful pressure on upstream providers to show their work. In particular, as another commenter pointed out, failing to provide an SBOM and providing a deliberately inaccurate or incomplete SBOM are quite different. One is presumably mere incompetence, the latter opens the door to consequences for fraud.
ris 6y ago
The solution to this problem is not bureaucracy. The solution is in the reproducible builds project, Guix and Nix.
- Ericson2314 6y ago
  The technical side of the solution is those, yes, but it's equally important that procuring administrators start requiring that level of auditability. That's the social solution. And just making the good technology is no guaranteed that society will raise its standards accordingly. Look no further than the sorry state of programming languages historically if you want proof of that...
- snicker7 6y ago
  There is a big difference between Guix and Nix and this area. Guix only accepts packages that can be fully bootstrapped, unlike Nix. It's a lot more effort, but it means everything you install in the official Guix channel ultimately comes from a ~300 byte, auditable binary seed. Nix is more lenient when it comes to packages vendoring their own dependencies outside of Nix proper.
dlor 6y ago
My big problem with all the SBOM efforts is that any kind of compliance/accuracy will be best effort and most likely wrong, leading to more problems and blame. This is not as simple as writing down your dependencies. Most people don't even know what their full set of transitive dependencies is, or how to even go about finding it. How do you know the SBOM you get is even accurate? You can't just crack open a binary and look at what's inside. If you could, we wouldn't need these giant complicated file formats.
- fulafel 6y ago
  You can, in fact, crack open the binaries and look at what's inside. The field of tooling for it is called SCA (software composition analysis).
  - dlor 6y ago
    Sort of. The quality of the data this tooling generates varies GREATLY among languages, build systems and environments. For packaged software like Solarwinds, sure you can try to run an SCA tool. But is anyone claiming an SBOM or SCA tool could have prevented that attack? The bigger issue is services and hosted software. You can't crack open an API or website that stores your data to see what database they're using. You could ask that they publish an SBOM, but who knows if it's accurate.
    - fulafel 6y ago
      I feel you're moving the goalposts a bit. Perfect is the enemy of good, etc. Also surely the tooling would get a lot of investment and improvement poured into it if the proposal went through. Anyway, if this kind of thing really took off, I could well imagine there being regulation for SaaS products having to do audits involving this, for example.
      - dlor 6y ago
        I sort of see this as a situation where an imperfect SBOM is worse than nothing. It would do nothing but add false confidence. I still haven't seen an example of a single supply-chain attack that an SBOM would have prevented.
playcache 6y ago
The rekor project under sigstore is interesting in this regard: https://github.com/sigstore/rekor https://github.com/sigstore/rekor Its listed as a signature transparency log, but they support some sort of custom manifest system, so you can set your own schema in your prefered format (xml, json, yaml) - the only thing is they require the manifest / material file is signed (I guess as it then brings a level of non-repudation). I am hoping someone works on an SBOM type. I heard some of the in-toto folks are working on the project as well. This is a good step towards a SBOM recorded supply chain.
- dlor 6y ago
  Maintainer here! That's exactly the idea. We're working with intoto and others to get metadata that we can actually verify, directly from build systems. Rekor is a place to put and find that metadata that's globally visible and can't be tampered with. We're hoping to add support for the ITE-6 in-toto link format soon, which I see as kind of like an SBOM that can be produced directly from your build system.
mrweasel 6y ago
Doesn't something like FDA approval of medical systems already require this? I believe you're required to maintain a list, and risk analysis of third party software you incorporate into medical products: https://en.wikipedia.org/wiki/Software_of_unknown_pedigree https://en.wikipedia.org/wiki/Software_of_unknown_pedigree
exo762 6y ago
Solution is different. Decriminalize software errors exploitation / hacking. Reliance on justice system is a fools' errand in a global economy anyway. There are two tiers of software operations. Global (FAANG etc) and local. First are rather good with security - because they pay attention. Local live in a fantasy world where people who should care are afraid to poke. Incentives for local services (government, municipal, local companies) are misaligned. This is why their security is in such a bad shape and this is why they fail spectacularly.
varispeed 6y ago
I find it obscene that there are some important software tools that are often developed by one guy in his rented basement and big corporations make billions on the back of it without sharing a penny. We need a royalty system for open source software, so that these companies will have to start paying fair share to the developers they exploit. This will also ensure the overall safety as developers will have funds to do audits or hire staff to fix security issues or there will be an incentive for people to contribute as successful PRs would be eligible for royalties. Such system could be embedded in GitHub and other VCS. Individuals and companies with a revenue below certain threshold would be exempt from paying. There should be no opt out - in many countries a worker cannot legally work for free, even if they agree to it - they have to be paid at least a minimum wage and in the same vein companies using open source software without payment are circumventing this rule. This will also level the playing field for developers from poor backgrounds - some developers cannot afford to work for free, so they cannot contribute to open source even if they wanted to do, because they have bills to pay. This way open source wouldn't be only reserved for privileged developers who can afford to commit their spare time.
- O5vYtytb 6y ago
  This would be the end of open source software as we know it.
  - varispeed 6y ago
    In a way yes, but currently it is not sustainable. I think we can do much better.
- gonzo41 6y ago
  As usual, https://xkcd.com/2347/ https://xkcd.com/2347/ The contributors to oss are in a tricky spot. Look at what happened with AWS forking Elasticsearch, sure there were reasons, but it seems like there's a gap in the licenses at the moment, that doesn't account for the scale things like ssl play in modern life. Whatever legal terms you'd use, you'd want to aim to not scare small companies in the hope of anchoring a income stream when the scale and find our the oss clause kicks in.
lambda_obrien 6y ago
From experience in medical equipment, a BOM was about as useful as a piece of toilet paper. Trying to keep one up to date when every resistor and nut and bolt is included is a pain and about 20 percent was probably wrong. An SBOM is just more beauracracy, what you need is due companies to actually want to pay for developers to use the right tools for the job. If you have security requirements that don't allow for using as many dependencies or require more updates, then pay developers to write something in house or to keep things up to date with more sprints dedicated to maintenance.
- icegreentea2 6y ago
  From my experience in medical equipment, I wouldn't have guessed 20% wrong... maybe closer to sub 5% wrong. Though I guess we weren't tracking individual components on boards... but we were definitely counting every nut, bolt and screw that we were using in assembly. I definitely hear your second part though. Having cobbled together an SBOM, it's definitely a pain. We got some value of it, since it really did give us a sense of the scale and shape of our dependencies.
tkinom 6y ago
I like to see OS capable of full auditable logs of every app execution in the system (phone, mac, linux, windows) every .so, .dll use by each app and their hash/datetime creation. every files/dirs creation / write/read by which app every socket bind and connect requests. and other privilege operations There should be virus total type check on all app/.so/.dll. There should be allow/forbid LIST for exec,file/dir access/socket, privilege ops access similar to typical firewall software - Not just for net, but also for app execution and files access. "Default allow", "Default forbid - with log/notification" fully under user control. like selinux, but with much better UI/UX (web base, build on top of ebpf?)
- 616c 6y ago
  In Linux, not completely an answer to what you want auditd does a lot of it, but I rarely see it mentioned outside the government and military because of use of the STIG requirements. And to your point: the UI sucks as it just text-based config in its own format and no one likes it or reads the outputted logs in my experience, even the SOC people who should know it.
LockAndLol 6y ago
So basically reproducible builds with a dependency list and the CVE list could provide more certainty about the tools we use? What about websites though? Hash-summed files aren't going to save us, because resources can be loaded dynamically and the client can't know the hash before retrieval. Reproducible builds would be a great first start. Forcing governments to use opensource may be another step.
- dane-pgp 6y ago
  > What about websites though? It is possible for a web page to specify the expected hash of a script file, which the browser will enforce. This is called SRI (Subresource Integrity).[0] Of course that still leaves the bootstrapping problem of how the page itself can be guaranteed to have a specific hash, but fortunately there is a clever hack that can be done with bookmarklets[1], or the page can just be saved and loaded/served locally. While that works technically, the UX isn't great because the address bar won't show the domain of the remote server (although browsers seem to be hiding the address bar from the user more and more). A better solution would be for browsers to support Hashlinks[2], which would allow a bookmark to point to a remote page with fixed contents. [0] https://developer.mozilla.org/en-US/docs/Web/Security/Subresource_Integrity https://developer.mozilla.org/en-US/docs/Web/Security/Subres... [1] https://news.ycombinator.com/item?id=17776456 https://news.ycombinator.com/item?id=17776456 [2] https://github.com/w3c-ccg/hashlink https://github.com/w3c-ccg/hashlink
Roark66 6y ago
If the author was serious about promoting the idea the article would be published in an open manner (not behind a pay wall). The concept is good, but good luck enforcing it with closed source software companies. Anyone that is really interested can already find that info for OS software but where it would really be useful is with closed source software. Where I personally would really love to see it implemented is with embedded devices. I've been recently hacking a not so old IP cam in my spare time. Hardware is great... It has 600mhz 32bit cpu with 64mb ram, hardware h264 (1080p 30fps close to real-time) encoding, bi-directional audio, WiFi, USB host, ptz, free gpio, Ethernet all for around $20 (indoor version) but software is abysmal. It runs Linux Kernel v3 (almost a decade old). Upon startup immediately starts streaming video/audio to a server in China while the mobile app requires you to "register" for an account with a phone number. The only way it can receive the video is from the Chinese server and it displays ads on 20% of its screen. Ridiculous. Thankfully it is pretty easy to hack, but what about all non technical people who buy it?
616c 6y ago
I think SBOM is interesting but it is totally absurd to think that SBOM is a _complete_ solution and/or comprehensive mitigation to Sunburst. It is starting to make me mad people say this. There is money on the line to create or enhance profitable software, so why not sell it that way? I do not believe Solarwinds staff do not manage their dependencies or handle software better or worse than other companies. One attack vector had software that was properly digitally signed (think about _that_) and required cleverly backdooring developer workstation and infecting pipelines, then wiping traces clean. To mimic "if you can dodge a wrench, you can dodge a ball" you can similarly say "if you can hack a build process to build digitally signed software to wipe your traces away, you can hack a SBOM process in a CD pipeline to say whatever the hell you want." I am into rektor and sigstore, but even they must realize, like others, if you think about the event everyone is talking about and the real threat model, we are advocating for some speeds as a solid perimeter defense, these are not fortified walls of security design. Technical people get this nuance, non-technical do not, so this kind of solution advocacy in these articles about Solarwinds as an example, really resonates with the latter, who make purchasing decisions and strategies. That is what worries me. @ris has a key part of it right, imho, but you have to go bigger than that, and not think just open source (even as a FSFer, I say that). > The solution to this problem is not bureaucracy. The solution is in the reproducible builds project, Guix and Nix. My belief for 3 pieces, the third and most difficult is missing. 1. Yes, digitally signed SBOM (in regulation or software contracts, those in USG contracting will know this is coming down the pipe anyway, others will follow). 2. Requirements for reproducible builds and _not_ just open source software doing that (I am thinking a build escrow ecosystem will have to come soon so commercial entities can farm out in some way their pipeline to third parties to build the exact thing they sell, identically match, or huge flairs go up). Again, regulation and contracts will have to push this, but I wonder how crazy I sound when I write this. 3. So if 2 seems hard: we need more appsec competency on just on the dev side, but the build/deploy side. If you have industry security bodies (government, legal, energy, financial) or big employers themselves, they will _need_ to have people set up test labs with realistic deployments over time, watch how their software behaves, build a network of people, resources, and information exchanges. They need to be able to build the skilset, learn to find vulns and most importantly risky default misconfigurations combining multiple software packages individual vendors don't think about. They will need to discuss when software that is 1 month in use or 8 years of use for %80 of my industry sector's employer or 100% of one big employer's network through training and communication to go ask people through these exchanges "hey, these systems are acting weirdly. Is this weird, do others see this or know this mis-configuration could be exploited and people have seen this before?" I mean that kind of knowledge share. If it does not, certainly re 2 and 3, SBOM will change some, but not all.
kodah 6y ago
We already produce a "SBOM", which is just an export of our dependencies. Our dependencies are also scanned for open CVE's, which is non-optimal because it's retroactive. The strongest forms of security operations we implement proactively are fuzzing and linting (in combination with strongly typed languages and language variants). The article references the SolarWinds attack but then doesn't go on to explain how it occurred nor how their SBOM would've defeated it. Instead it quotes Microsoft in saying that it was highly sophisticated. Just a reminder that the origins of the SolarWinds Orion hack are still up in the air [0], which makes the definitive tone of this article all the more confusing. It is speculated that hackers compromised TeamCity and that TeamCity injected code during compile time into Orion. This wouldn't be caught by any kind of dependency inspection and doubly so if the attackers were smart enough to use all standard libraries. Like all hacks, this one had signatures too [1]. Some that stand out to me are the network calls: avsvmcloud[.]com deftsecurity[.]com freescanonline[.]com thedoccloud[.]com websitetheme[.]com highdatabase[.]com incomeupdate[.]com databasegalore[.]com panhardware[.]com zupertech[.]com 13.59.205[.]66 54.193.127[.]66 54.215.192[.]52 34.203.203[.]23 139.99.115[.]204 5.252.177[.]25 5.252.177[.]21 204.188.205[.]176 51.89.125[.]18 167.114.213[.]199 It's impossible to hold definitive allow lists for IP addresses or domains, but knowing characteristics about these calls might at least make finding hacks faster, though this logic is also easily defeated. In what I would call a very sterile software environment you might install fully-configured SE Linux on a system, limit your outbound network call destinations to approved locations, run linting with static compilation, fuzzing on your functions and binary inputs. I can still think of a myriad of ways of compromising a distributed system like that. That's not to say, "do nothing" and more to say, this is a very complex problem and something like an "SBOM" isn't new or going to solve this problem if replicated. [0] https://www.nytimes.com/2021/01/06/us/politics/russia-cyber-hack.html https://www.nytimes.com/2021/01/06/us/politics/russia-cyber-... [1] https://blog.malwarebytes.com/threat-analysis/2020/12/advanced-cyber-attack-hits-private-and-public-sector-via-supply-chain-software-update/ https://blog.malwarebytes.com/threat-analysis/2020/12/advanc...
bryanrasmussen 6y ago
package managers tend to give you something wherein you can read exactly what is in your solution, which I suppose is why people are always reading what their dependencies are and making sure that everything is updated as it should be and security holes plugged. So, given the existence of modern package managers, surely problem solved.
mikewarot 6y ago
Wrong root cause analysis. You'd have to solve the halting problem to get anywhere this way, which is proven impossible, thus it is the wrong path. Real cause: Widespread adoption of Operating Systems that don't default to capability based least privilege.
- dathinab 6y ago
  My dream, a OS with a good proper well designed single capability system with reasonable defaults. Not the mess Linux has which supports capabilities for some privileges but not so much for others, which has bad defaults and which is supper fragmented. I mean for a full cover you need to correctly combine root/sys capabilities with seccomp with bpf with cgroups with polkit with some pctrl settings with linux kernel parameters and even then you probably still need to throw in selinux and I still missed at least file permissions when writing this... Which lets be honest is just ridiculous.
jacques_chester 6y ago
We need SBOMs, but these are not enough. We need supply chain attestations, but these are not enough. What we need is the combination of asset data, process data and to acknowledge that our knowledge of both is always incomplete and subject to change. I call this need a "universal asset graph" and I've been nagging folks for years to get us to it. The sigstore project is the biggest foundation stone of what I'd wish for, at least in terms of creating a robust shared log of observations (a leader of that effort, dlor, is in this discussion). But we still have a very, very long way to go as an industry.
- Ericson2314 6y ago
  Just look at Nix. Here's the thing, having the sellers of unfree software compile the code for is a terrible skeuomorphism from the way traditional products are made. The final integrator should be the one building the code even for propriety0 and unfree software, whose secretiveness should be enforced with contracts not obfuscation and baking in specific dependencies. The fact that the finally compilation graph, and the IP procurement graph have some similarities should just be a coincidence.
  - jacques_chester 6y ago
    I didn't follow your argument. Could you elaborate?
    - Ericson2314 6y ago
      Right now I'd you buy property software, you get a binary of some sort. That's neither composible or auditable. You should get the source code and a reproducible build, that you can modify and integrate with other things. Kinda like licensing closed source game engine parts, or getting hardware design libraries (IP as they say) for making your own system on a chip.
      - jacques_chester 6y ago
        I think I follow now. Source availability is preferable but it will take a while to become normal. It will be even longer bit-for-bit reproducibility is a commercial norm. SBOMs still give us value in the meantime. If I buy product X, which asserts using dependency Y, then when a vulnerability is asserted for Y, I can pester the vendor to show that they have updated. At this point if they claim to upgrade but haven't, that becomes fraud. The economic incentives vs our current anything-goes world are differently weighted.
dzink 6y ago
Server logs are already full with calls to post to different pages, or php scripts of vanilla wordpress installations as attackers try to find vulnerable sites. Wouldn’t a SBOM make the bad guys job easier? If you are a bad actor or a malicious state actor who has just gotten hands on a new exploit the SBOM would give you an instant menu of a available hackable sites. A B2B vendor or Saas vendor can definitely make their stack available to clients upon deal negotiations, but putting it on the open web is also asking for trouble.
- jacques_chester 6y ago
  With an SBOM, attackers gain an advantage, but it is highly concentrated in that attacker. Defenders also gain, the gain is highly dispersed amongst defenders, but each gains ~ the negative value of the attacker's gain. Put another way: there are far more defenders than attackers. When something helps both attackers and defenders, the gains of defenders outweigh gains of attackers.
kohlerm 6y ago
What IMHO really is needed are https://reproducible-builds.org/ https://reproducible-builds.org/ plus some way to verify within a company that only allowed packages are used. One way to solve this is to check all software (including open source one) into a monorepo and run software that checks for copies of open source code.
- dathinab 6y ago
  Reproducible build require source code access. Which is the first point I think is necessary: - Access to source code for at least all entities using the software (including allowing hiring entities to analyze it). Preferable open access to source. Even more preferable open source. - Combine that with reproducible builds and automatic code analysis and you gain additional trust. - Naturally this both requires proper code and artifact signing (which was compromised in the supply chain attack this article refers to). Funny thing is I'm 100% convinced no SBOM, reproducible builds or similar would have prevented this attack. It would just have changed how exactly the attack looks (IMHO). Still it would be an improvement anyway.
  - jacques_chester 6y ago
    Reproducible builds would've had a sporting chance of defeating the solarwinds attack, because the injection point was on particular build servers that were reached through other vectors. If a second party performs a build to verify it, the attacker now has ~2x the cost to conceal their attack.
    - dathinab 6y ago
      True, but this requires the source you build not to have been gone through the build server, which is reasonable. The thing is you can circumvent this by attacking the version control and/or developer systems. And at least the later one are often massively vulnerable to certain kind of supply chain attacks. Ironically the permissions and setups commonly used for a nice development flow are also making systems vulnerable for many kinds of supply chain attacks. I'm currently slowly moving to a more secure dev flow, but it adds overhead. Especially if your dev system is also your laptop. First step is to run any kind of dev tool (especially builds) in a container. Through this often also means running e.g. a language server in the container while running your IDE out of it and making sure nothing will trigger your IDE to do thinks outside of the container...
redleggedfrog 6y ago
For 95% of software will never happen unless SBOMs make the company money.
Jupe 6y ago
Ramblings on these topics... Exposing SBOM on every piece of delivered software will just make a hackers job easier and quicker... Since by design they are machine readable, SBOMs will make querying for specific vulnerabilities trivial. This is not a top-down problem! Any upper layer can be compromised by a lower layer (os, build tool, library, reporting tool, etc.) this problem can only be solved Botton up : from verified OS, to verified (bootstrap) build tools of that OS, to every library installed on that OS, etc. We currently have decades of software resting atop of unverified libraries resting atop of unverified operating systems, all built with unverified tooling. We can't even build verification tools that are, themselves, verified! And if we could, can we even say they verify every potential vulnerability? (mitm, boundaries, race conditions, cpu cache, etc.) I know there is research at some universitys into formally verified OS's, but it's a long way off IMO. This is the problem of our time. But, unfortunately, the industry seems consumed with velocity and cleverness over stability and security.
- silly-silly 6y ago
  > I know there is research at some universitys into formally verified OS's, but it's a long way off IMO. I believe seL4 is verified and used in production ( https://sel4.systems/ https://sel4.systems/ )
l8again 6y ago
The idea of SBOM, or BOM is not a new one. Maven already has one for years now called BOM files [1]. So am I right in assuming that the author's suggestion is to make this BOM file public like the ingredients in food product as some people have suggested? If so, I can see a quick static analysis tool that can spit out the vulnerabilities just by parsing the BOM. So, really there's nothing to do much _technically_ here other than releasing out in the open the BOM. And then displaying ugly warnings about any software's BOM to either shame them or actually hurt their bottom line by lost revenue due to those vulnerabilities. [1]https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#bill-of-materials-bom-poms https://maven.apache.org/guides/introduction/introduction-to...
- l8again 6y ago
  Also, I wonder how can we realistically implement this for SaaS?
  - dathinab 6y ago
    Implement yes, but will it make a relevant difference? IMHO Unlikely. A BOM no one (of relevance) ever reads is as good as no BOM. The main positive effects a BOM can have (outside of SaaS) is to more strongly discourage to use (continue to use) of known to be problematic libraries or services.
dathinab 6y ago
Let me tell you a secret (not really): - supply chain attacks are not new - sophisticated attacks are not new - what made microsoft call it the most sophisticated attack was *not* because of it using a supply chain attack, or waiting or any of the major bullet points listed when speaking about it but the combination of them and all the small details skipped - through it might be the first sophisticated attack using supply chains *which we know of and which got a lot of press/success*. - Neither new are attack which are possible at massive, potential global scale, even without supply chain attacks. So there has been a long time a lot potential for people losing, trust. There is a reason why many tech affine people don't trust governments or large institutions which sensitive data, we know the even the largest institutions will not always manage to keep our data safe. - "does not take the coordinated effort of thousands of engineers in a nation-state" all sophisticated large scale attacks do *not* need thousands of engineers. Most times it's done by teams with noticeable less then 100 people. Nation-scal agressors are not that because they have so many (evil;) programmers but because they have access to nation-scale resources, like access to internet infrastructure, vulnerabilities known by the secret service or just money they can use for probing or obfuscating DDOS attacks. Lastly governments can have an easier job to bring a lot of expertise together, by bringing very qualified people together, not very many. - "load external modules" has nothing to do with supply chain attacks, while it sometimes can make thinks simpler if you can affect the source-code (which is what supply chain attacks are about) you can do everything you need without any "module loading". - "few risks unique to the software domain", I would say more than a few, and they are well known since over a decade - "others are proprietary", which is a problem as it undermines many ways a user can try to detect/catch supply chain attacks. Non of which are perfect, like non would have cough this attack. But it still makes attacks of this kind harder. - "Dependency hell" the only real problem, libraries with non-clear and brittle and to much much changing interfaces, at the same time surprisingly language dependent. - "easier automated than what defenders need to do.", I wouldn't be sure about it, the same way attackers can scan for problem you can do so (and if you find something fix it). Tools like fuzz testing and other kinds automatic software analysis are widely underused. - "96% of all software products included third-party software components and commercial" it tries to make this look like a problem but actually the only way to avoid security vulnerability (and build modern software) is by using external tools. If everyone would implement e.g. their own crypto there would be more vulnerabilities and automatic scanning for them would *still* work as people tend to make similar mistakes when writing similar code. - "and now also by nation state actor", not it should be "and since the beginning". Just look at crypto wars, or how currently the US/EU governments again try to undermine software security for questionable reasons. - "a bill of materials protects best against supply chain vulnerabilities when it is set up as a holistic cross-domain effort"... no it doesn't. There is nothing in a bill of materials which is able to prevent supply chain attacks. I have no idea why the author believes it. At best it can make it easier to catch if a known-to-be-vulnerable dependency is used. But supply chain attacks are about about making a dependency vulnerable without anyone knowing and *by changing it*. Not just by adding code. I guess this is build on the misconception that not having "loading of external modules" would help against supply chain attacks. It doesn't. It just makes it negligible harder. -by the way open accessible source does imply a bill of materials (being derivable form the source) and allows you to automatically run software analysis tools on the source code etc. - "medical industry has been targeted for years" and banking systems, operating systems, mail programs and games have been attacked even longer. The medical industry is interested in this to *avoid legal problems by using absurdities like software being "certified" to be secure (e.g. by using SBOM's) and similar*. A concept which is known to not actually work but tends to work to avoid legal responsibility. If they would care they would push for open accessible source, proper bug bounty programs and similar. - "What needs to be done?" (In my opinion:) Push for at least open accessible source (!= open source). Push for more usage of software analysis tools. Push for proper bug bounty programs (many existing ones are questionable). *Enforce legal liability IF it's clear the software "seller" didn't care for making and keeping the software secure.* (I.e. Liability if negligent). Push for using additional protection which reduce attackers gain even if there are vulnerabilities. E.g. migrations like address layout randomization and shadow stack, sandboxing, dropping privileges, etc. WebASM might play a major role in this. Push for responsible choice of language, libraries and tooling. Push companies to take more responsibility for open source tools they use (!= open accessible source). Lastly sometimes (not seldom) they *way* you use a certain dependency and for what you use it is making a *extreme* difference wrt. security. One example would be alternative hash algorithms in the rust eco system. Use them for certain internal use-case and all is fine. Use them at the wrong place and they enable hash map based DOS attacks. Another example would be openssl which you can use responsible, but also can use in horrible insecure ways. In the end a SBOM is a thing which *should* be cheap to create (potentially automatized) but will only yield a slim improvement in general. It is IMHO *by far* not the most important step to do, but I guess it's one of the most easiest steps to do. So why not, but don't believe it will make any major difference.
euph0ria 6y ago
Are people publishing on Medium in order to earn money now or just that the large migration to a free platform is yet to occur?
dathinab 6y ago
Any supply chain attack can be done in ways where it's externally impossible to (automatically) differentiate a malicious change from a intentional change. Which means BOM's are 100% guaranteed not to prevent supply chain attacks. At best it makes them a small bit harder. Software BOM's can have some small benefits, but preventing supply chain attacks is not one of them. And open access to source always is much more useful then just a BOM. And like a BOM without signature isn't that much trust able (with closed source) accessible source without signing is not trusteable without reproducible builds. Anyway signing source code isn't hard so no reasons not to do so, signing build artifacts can be harder.
- silly-silly 6y ago
  > signing build artifacts can be harder. Is this like signing an RPM package?
  - dathinab 6y ago
    Somewhat. It's not really hard just a bit harder. Mainly on your desktop system you can easily use a hardware security key (like a yubikey) to get a good level of signing security. But your CI server has a good chance to not be your own hardware so you can't plug in a yubikey so to have good key security you either need to find a CI server where you have a HSM or you need to have some form of signing server/service to keep your private key save. You also should at least consider things like "could the artifact be changed in between your CI creating it and your build signing server/IP-based HSM signing it". There are ways to handle it, with different benefits and drawbacks. And considering and understanding this things is maybe the biggest additional amount of work. Or you use a schema-F solution which seems reasonable without understanding it and be okey with it.
fmajid 6y ago
"However, there is an even more worrying effect: if attacks are possible at such a scale that their effect is felt across whole sectors, countries or even globally, as is the case with the “SolarWinds” attack, they have the potential to fundamentally undermine our trust in information technology." How is that a worrying effect? Trust in IT systems is undeserved, because they are built in an environment of economic incentives that make stability, performance and security secondary concerns to features and time to market, and poor engineering practices that derive from those incentives. A software BOM will do nothing to address those problems. Healthy skepticism of IT and adapting with defense in-depth and other measures like data minimization (you can't leak what you don't have) are desirable, or even simply using less software.
hyko 6y ago
A software BOM will not address these issues. Stopping the delusion that any one nation can come out ahead in this game by hoarding vulnerabilities, and working towards establishing and enforcing strict rules of cyber warfare are the first step.
Const-me 6y ago
When I develop software, the source code repo contains a text file with all the third-party stuff I have used, both linked and copy-pasted, along with the URLs where I got the code and their licenses. Not precisely a BOM and I maintain them for different reason, but overall I think pretty close to what’s proposed. Couple examples from my open-source projects: https://github.com/Const-me/vis_avs_dx/blob/master/legal.txt https://github.com/Const-me/vis_avs_dx/blob/master/legal.txt https://github.com/Const-me/Vrmac/blob/master/Pre-existing%20IP.txt https://github.com/Const-me/Vrmac/blob/master/Pre-existing%2...
mybrid 6y ago
Bills of Materials work with hardware because origins can be tracked. Good luck tracking the origin of electrons. I worked in manufacturing for a company the dealt solely with MilSpec (Military Specification). Crates sent to Nuclear Power plants had to X-Rayed on the shipping dock and then again at the receiving dock. If the X-Rays differed in any questionable way the shipment was rejected. However, most supply chains are not that paranoid. In the 1980s there was a grade eight bolt scandal for a satellite that blew up in space because the manufacturer substituted plain steal to make more money. More recently the BOM did nothing to protect the Bay Area Bridge where once again bolts as well as rods were specified of one quality and delivered as another. The bolts and rods are still in the new span because taking them out would require tearing it down again. But the builder assures us things are fine, wink wink, nod nod. https://www.courthousenews.com/34m-settlement-reached-for-defective-work-on-bay-bridge/ https://www.courthousenews.com/34m-settlement-reached-for-de...
- patcon 6y ago
  Aren't you just talking about checksums? Your xray anecdote is neat, but the software equivalent of that paranoia is even simpler and easier to operationalize if there's incentive to do so. e.g., https://reproducible-builds.org/ https://reproducible-builds.org/
  - mybrid 6y ago
    Materials can be traced to point of origin, especially nuclear active material. There is no such tracing with software. A checksum only guarantees that two things have equal software. It does not say the difference of origin between the two.
SergeAx 6y ago
Can we please stop calling SolarWinds hack "the most sophisticated and protracted intrusion attacks of the decade" already? It was nothing more than blunt negligence.
0xbadcafebee 6y ago
> we need to enforce global software supply chain management now. Well, we can't. We don't have the infrastructure for it. It will probably take every industry in the world between 4 and 10 years to have a fully secure supply chain. > The world runs on software. Close: it runs on hardware, and that hardware isn't secure either. It also runs on networks, and authenticated, authorized communications, and those aren't secure either. > However, there is an even more worrying effect: if attacks are possible at such a scale that their effect is felt across whole sectors, countries or even globally, as is the case with the “SolarWinds” attack, they have the potential to fundamentally undermine our trust in information technology. Holy shit. Nobody tell this guy about the NSA hacking Cisco, he might get really upset. > an attack on a low system level may be able to corrupt or circumvent protective measures on higher levels. ..... Unless you design it not to. > Equifax’s 2017 leakage of hundreds of millions of customer records, was possible because the organization failed to update a vulnerability in a low-level web server module No. It was possible because Equifax had no accountability. It was also possible because their network and data access policies were Swiss cheese, and only after that was it the fault of somebody not patching a shitty app. The rest of his argent is that the biggest problem we have is patching and malicious code in our stacks. But even without supply chain attacks you still have 0days which are the same problem: shitty code gets exploited. You can't avoid shitty code. All you can do is mitigate risk. What stops big attacks is not one single fancy idea, but putting enough hoops in place to make big attacks extremely rare and difficult. That, and educating developers on how to not write shitty code and build shitty systems. Most developers I know literally don't even know how to avoid SQLi, or if they do know how, they're too lazy or overworked to do it. They pick random shitty tech and clumsily paste it together to get some MVP working, and then that becomes production. Nobody wants to pay for real components and solutions that were designed the right way and certified. Everybody just wants to write brand new shit code from scratch in fucking JavaScript using a randomly assembled bunch of modules a retarded hamster could write, and then make it so they can't be extended and improved over time, because they're owned by some jackass worse password is Welcome123! Supply chain is just what's trendy. The real fix is to get people to stop trusting morons to make shitty products, and stop paying people to write custom applications that all suck.
b-g-m 6y ago
There also needs to be a process to validate the SBOM matches the compiled software with an air gapped digital twin. ( you have to bypass a few things in the header - like time stamp and a unique I’d) so both versions of your compiled software have the same hash. Then you need to protect that file from interception and still validate every line change in every file. This patent addresses many of these issues with a self-referencing SBOM on the blockchain. https://patents.google.com/patent/EP3687107A1 https://patents.google.com/patent/EP3687107A1
cc_stoic 6y ago
no

135 points by kiyanwang 6y ago

BrianOnHN 6y ago

What's with the images?

layer8 6y ago
My first guess was autostereograms, but that doesn’t seem to be the case. Maybe some steganography?
the_af 6y ago
I was going to comment the same. I really like them! They look as if displayed on an old CRT with color artifacts.

you_are_naive 6y ago

Blah blah blockchain blah blah have an alternative compliance questionnaire list which companies outsource already mandatory for every software blah blah as a way to check maturity of the company. I sometimes wonder if this is how industries end up not innovating or solving obvious problems for decades because they get strangled with bureaucracy which doesn't solve the original problem highlighted in his own example (vendor choosing to ignore to patch a vulnerability).

anoncow 6y ago

Why does this read like an anti-opensource propaganda piece aimed at big business?

wpietri 6y ago

At least on first reading, I find this unpersuasive. He correctly lists a variety of problems. But he doesn't explain how his proposed solution, listing all the components of a technological product, would make a practical difference. Creating a list is valuable only if people a) read the list, b) recognize problems, and c) do something based on that. And for some of the examples he gives, it seems pretty obvious to me that an SBOM wouldn't help. The Equifax breach, for example. They knew [1] that they needed to upgrade Apache Struts. Somebody was supposed to make the upgrade. They just didn't do it. Who would an SBOM help here? Since it's a consumer-facing website, the only people who weren't informed were consumers. So is he proposing to make public the SBOM for every website? I'm not sure that on balance that helps security. [1] https://www.csoonline.com/article/3444488/equifax-data-breach-faq-what-happened-who-was-affected-what-was-the-impact.html https://www.csoonline.com/article/3444488/equifax-data-breac...

mikepurvis 6y ago
Wouldn't having to advertise your out of date dependencies help to shame companies into upgrading on a reasonable schedule? So that upgrades are actually a priority and not just a thing that happens when literally everything else is already done?
- sokoloff 6y ago
  If that became a problem, companies intending to skirt the disclosure would fork and “maintain” private branches of dependencies such that it couldn’t be determined if they were out of date.
  - spacemanmatt 6y ago
    True story: They already do. I know one company that forked Ruby, and would likely claim every library they run under it is thereby forked for the sake of reporting.
    - mattcwilson 6y ago
      Ok, but there would still be some point of contact where an SBOM would show “company X fork of Ruby, company X fork of package Z” etc, right? And then the choice gets back to how much to trust company X and package Z, weighed against alternative solutions.
      - mannykannot 6y ago
        Unless penalties for doing so are legislated, they need not claim that the fork they are actually using is a fork of anything; they could treat it as if part of their proprietary code (though they might be in violation of license agreements if they do this.)
      - sokoloff 6y ago
        It seems like it could equally be “internal package UUID1 version X, internal package UUID2 version Y” if malicious compliance and/or industrious laziness was the goal.

pdimitar 6y ago

No SBOM will help you if the people know they have to act but they don't -- out of malice, bureaucratic slowdown, policy restriction and what-have-you. If you don't have hardware and software that can't be tampered with and that automatically apply / enforce the SBOM, then it is essentially worthless.

jacques_chester 6y ago
I disagree. SBOMs will help to create useful pressure on upstream providers to show their work. In particular, as another commenter pointed out, failing to provide an SBOM and providing a deliberately inaccurate or incomplete SBOM are quite different. One is presumably mere incompetence, the latter opens the door to consequences for fraud.

ris 6y ago

The solution to this problem is not bureaucracy. The solution is in the reproducible builds project, Guix and Nix.

Ericson2314 6y ago
The technical side of the solution is those, yes, but it's equally important that procuring administrators start requiring that level of auditability. That's the social solution. And just making the good technology is no guaranteed that society will raise its standards accordingly. Look no further than the sorry state of programming languages historically if you want proof of that...
snicker7 6y ago
There is a big difference between Guix and Nix and this area. Guix only accepts packages that can be fully bootstrapped, unlike Nix. It's a lot more effort, but it means everything you install in the official Guix channel ultimately comes from a ~300 byte, auditable binary seed. Nix is more lenient when it comes to packages vendoring their own dependencies outside of Nix proper.

dlor 6y ago

My big problem with all the SBOM efforts is that any kind of compliance/accuracy will be best effort and most likely wrong, leading to more problems and blame. This is not as simple as writing down your dependencies. Most people don't even know what their full set of transitive dependencies is, or how to even go about finding it. How do you know the SBOM you get is even accurate? You can't just crack open a binary and look at what's inside. If you could, we wouldn't need these giant complicated file formats.

fulafel 6y ago
You can, in fact, crack open the binaries and look at what's inside. The field of tooling for it is called SCA (software composition analysis).
- dlor 6y ago
  Sort of. The quality of the data this tooling generates varies GREATLY among languages, build systems and environments. For packaged software like Solarwinds, sure you can try to run an SCA tool. But is anyone claiming an SBOM or SCA tool could have prevented that attack? The bigger issue is services and hosted software. You can't crack open an API or website that stores your data to see what database they're using. You could ask that they publish an SBOM, but who knows if it's accurate.
  - fulafel 6y ago
    I feel you're moving the goalposts a bit. Perfect is the enemy of good, etc. Also surely the tooling would get a lot of investment and improvement poured into it if the proposal went through. Anyway, if this kind of thing really took off, I could well imagine there being regulation for SaaS products having to do audits involving this, for example.
    - dlor 6y ago
      I sort of see this as a situation where an imperfect SBOM is worse than nothing. It would do nothing but add false confidence. I still haven't seen an example of a single supply-chain attack that an SBOM would have prevented.

playcache 6y ago

The rekor project under sigstore is interesting in this regard: https://github.com/sigstore/rekor https://github.com/sigstore/rekor Its listed as a signature transparency log, but they support some sort of custom manifest system, so you can set your own schema in your prefered format (xml, json, yaml) - the only thing is they require the manifest / material file is signed (I guess as it then brings a level of non-repudation). I am hoping someone works on an SBOM type. I heard some of the in-toto folks are working on the project as well. This is a good step towards a SBOM recorded supply chain.

dlor 6y ago
Maintainer here! That's exactly the idea. We're working with intoto and others to get metadata that we can actually verify, directly from build systems. Rekor is a place to put and find that metadata that's globally visible and can't be tampered with. We're hoping to add support for the ITE-6 in-toto link format soon, which I see as kind of like an SBOM that can be produced directly from your build system.

mrweasel 6y ago

Doesn't something like FDA approval of medical systems already require this? I believe you're required to maintain a list, and risk analysis of third party software you incorporate into medical products: https://en.wikipedia.org/wiki/Software_of_unknown_pedigree https://en.wikipedia.org/wiki/Software_of_unknown_pedigree

exo762 6y ago

Solution is different. Decriminalize software errors exploitation / hacking. Reliance on justice system is a fools' errand in a global economy anyway. There are two tiers of software operations. Global (FAANG etc) and local. First are rather good with security - because they pay attention. Local live in a fantasy world where people who should care are afraid to poke. Incentives for local services (government, municipal, local companies) are misaligned. This is why their security is in such a bad shape and this is why they fail spectacularly.

varispeed 6y ago

I find it obscene that there are some important software tools that are often developed by one guy in his rented basement and big corporations make billions on the back of it without sharing a penny. We need a royalty system for open source software, so that these companies will have to start paying fair share to the developers they exploit. This will also ensure the overall safety as developers will have funds to do audits or hire staff to fix security issues or there will be an incentive for people to contribute as successful PRs would be eligible for royalties. Such system could be embedded in GitHub and other VCS. Individuals and companies with a revenue below certain threshold would be exempt from paying. There should be no opt out - in many countries a worker cannot legally work for free, even if they agree to it - they have to be paid at least a minimum wage and in the same vein companies using open source software without payment are circumventing this rule. This will also level the playing field for developers from poor backgrounds - some developers cannot afford to work for free, so they cannot contribute to open source even if they wanted to do, because they have bills to pay. This way open source wouldn't be only reserved for privileged developers who can afford to commit their spare time.

O5vYtytb 6y ago
This would be the end of open source software as we know it.
- varispeed 6y ago
  In a way yes, but currently it is not sustainable. I think we can do much better.
gonzo41 6y ago
As usual, https://xkcd.com/2347/ https://xkcd.com/2347/ The contributors to oss are in a tricky spot. Look at what happened with AWS forking Elasticsearch, sure there were reasons, but it seems like there's a gap in the licenses at the moment, that doesn't account for the scale things like ssl play in modern life. Whatever legal terms you'd use, you'd want to aim to not scare small companies in the hope of anchoring a income stream when the scale and find our the oss clause kicks in.

lambda_obrien 6y ago

From experience in medical equipment, a BOM was about as useful as a piece of toilet paper. Trying to keep one up to date when every resistor and nut and bolt is included is a pain and about 20 percent was probably wrong. An SBOM is just more beauracracy, what you need is due companies to actually want to pay for developers to use the right tools for the job. If you have security requirements that don't allow for using as many dependencies or require more updates, then pay developers to write something in house or to keep things up to date with more sprints dedicated to maintenance.

icegreentea2 6y ago
From my experience in medical equipment, I wouldn't have guessed 20% wrong... maybe closer to sub 5% wrong. Though I guess we weren't tracking individual components on boards... but we were definitely counting every nut, bolt and screw that we were using in assembly. I definitely hear your second part though. Having cobbled together an SBOM, it's definitely a pain. We got some value of it, since it really did give us a sense of the scale and shape of our dependencies.

tkinom 6y ago

I like to see OS capable of full auditable logs of every app execution in the system (phone, mac, linux, windows) every .so, .dll use by each app and their hash/datetime creation. every files/dirs creation / write/read by which app every socket bind and connect requests. and other privilege operations There should be virus total type check on all app/.so/.dll. There should be allow/forbid LIST for exec,file/dir access/socket, privilege ops access similar to typical firewall software - Not just for net, but also for app execution and files access. "Default allow", "Default forbid - with log/notification" fully under user control. like selinux, but with much better UI/UX (web base, build on top of ebpf?)

616c 6y ago
In Linux, not completely an answer to what you want auditd does a lot of it, but I rarely see it mentioned outside the government and military because of use of the STIG requirements. And to your point: the UI sucks as it just text-based config in its own format and no one likes it or reads the outputted logs in my experience, even the SOC people who should know it.

LockAndLol 6y ago

So basically reproducible builds with a dependency list and the CVE list could provide more certainty about the tools we use? What about websites though? Hash-summed files aren't going to save us, because resources can be loaded dynamically and the client can't know the hash before retrieval. Reproducible builds would be a great first start. Forcing governments to use opensource may be another step.

dane-pgp 6y ago
> What about websites though? It is possible for a web page to specify the expected hash of a script file, which the browser will enforce. This is called SRI (Subresource Integrity).[0] Of course that still leaves the bootstrapping problem of how the page itself can be guaranteed to have a specific hash, but fortunately there is a clever hack that can be done with bookmarklets[1], or the page can just be saved and loaded/served locally. While that works technically, the UX isn't great because the address bar won't show the domain of the remote server (although browsers seem to be hiding the address bar from the user more and more). A better solution would be for browsers to support Hashlinks[2], which would allow a bookmark to point to a remote page with fixed contents. [0] https://developer.mozilla.org/en-US/docs/Web/Security/Subresource_Integrity https://developer.mozilla.org/en-US/docs/Web/Security/Subres... [1] https://news.ycombinator.com/item?id=17776456 https://news.ycombinator.com/item?id=17776456 [2] https://github.com/w3c-ccg/hashlink https://github.com/w3c-ccg/hashlink

Roark66 6y ago

If the author was serious about promoting the idea the article would be published in an open manner (not behind a pay wall). The concept is good, but good luck enforcing it with closed source software companies. Anyone that is really interested can already find that info for OS software but where it would really be useful is with closed source software. Where I personally would really love to see it implemented is with embedded devices. I've been recently hacking a not so old IP cam in my spare time. Hardware is great... It has 600mhz 32bit cpu with 64mb ram, hardware h264 (1080p 30fps close to real-time) encoding, bi-directional audio, WiFi, USB host, ptz, free gpio, Ethernet all for around $20 (indoor version) but software is abysmal. It runs Linux Kernel v3 (almost a decade old). Upon startup immediately starts streaming video/audio to a server in China while the mobile app requires you to "register" for an account with a phone number. The only way it can receive the video is from the Chinese server and it displays ads on 20% of its screen. Ridiculous. Thankfully it is pretty easy to hack, but what about all non technical people who buy it?

616c 6y ago

I think SBOM is interesting but it is totally absurd to think that SBOM is a _complete_ solution and/or comprehensive mitigation to Sunburst. It is starting to make me mad people say this. There is money on the line to create or enhance profitable software, so why not sell it that way? I do not believe Solarwinds staff do not manage their dependencies or handle software better or worse than other companies. One attack vector had software that was properly digitally signed (think about _that_) and required cleverly backdooring developer workstation and infecting pipelines, then wiping traces clean. To mimic "if you can dodge a wrench, you can dodge a ball" you can similarly say "if you can hack a build process to build digitally signed software to wipe your traces away, you can hack a SBOM process in a CD pipeline to say whatever the hell you want." I am into rektor and sigstore, but even they must realize, like others, if you think about the event everyone is talking about and the real threat model, we are advocating for some speeds as a solid perimeter defense, these are not fortified walls of security design. Technical people get this nuance, non-technical do not, so this kind of solution advocacy in these articles about Solarwinds as an example, really resonates with the latter, who make purchasing decisions and strategies. That is what worries me. @ris has a key part of it right, imho, but you have to go bigger than that, and not think just open source (even as a FSFer, I say that). > The solution to this problem is not bureaucracy. The solution is in the reproducible builds project, Guix and Nix. My belief for 3 pieces, the third and most difficult is missing. 1. Yes, digitally signed SBOM (in regulation or software contracts, those in USG contracting will know this is coming down the pipe anyway, others will follow). 2. Requirements for reproducible builds and _not_ just open source software doing that (I am thinking a build escrow ecosystem will have to come soon so commercial entities can farm out in some way their pipeline to third parties to build the exact thing they sell, identically match, or huge flairs go up). Again, regulation and contracts will have to push this, but I wonder how crazy I sound when I write this. 3. So if 2 seems hard: we need more appsec competency on just on the dev side, but the build/deploy side. If you have industry security bodies (government, legal, energy, financial) or big employers themselves, they will _need_ to have people set up test labs with realistic deployments over time, watch how their software behaves, build a network of people, resources, and information exchanges. They need to be able to build the skilset, learn to find vulns and most importantly risky default misconfigurations combining multiple software packages individual vendors don't think about. They will need to discuss when software that is 1 month in use or 8 years of use for %80 of my industry sector's employer or 100% of one big employer's network through training and communication to go ask people through these exchanges "hey, these systems are acting weirdly. Is this weird, do others see this or know this mis-configuration could be exploited and people have seen this before?" I mean that kind of knowledge share. If it does not, certainly re 2 and 3, SBOM will change some, but not all.

kodah 6y ago

We already produce a "SBOM", which is just an export of our dependencies. Our dependencies are also scanned for open CVE's, which is non-optimal because it's retroactive. The strongest forms of security operations we implement proactively are fuzzing and linting (in combination with strongly typed languages and language variants). The article references the SolarWinds attack but then doesn't go on to explain how it occurred nor how their SBOM would've defeated it. Instead it quotes Microsoft in saying that it was highly sophisticated. Just a reminder that the origins of the SolarWinds Orion hack are still up in the air [0], which makes the definitive tone of this article all the more confusing. It is speculated that hackers compromised TeamCity and that TeamCity injected code during compile time into Orion. This wouldn't be caught by any kind of dependency inspection and doubly so if the attackers were smart enough to use all standard libraries. Like all hacks, this one had signatures too [1]. Some that stand out to me are the network calls: avsvmcloud[.]com deftsecurity[.]com freescanonline[.]com thedoccloud[.]com websitetheme[.]com highdatabase[.]com incomeupdate[.]com databasegalore[.]com panhardware[.]com zupertech[.]com 13.59.205[.]66 54.193.127[.]66 54.215.192[.]52 34.203.203[.]23 139.99.115[.]204 5.252.177[.]25 5.252.177[.]21 204.188.205[.]176 51.89.125[.]18 167.114.213[.]199 It's impossible to hold definitive allow lists for IP addresses or domains, but knowing characteristics about these calls might at least make finding hacks faster, though this logic is also easily defeated. In what I would call a very sterile software environment you might install fully-configured SE Linux on a system, limit your outbound network call destinations to approved locations, run linting with static compilation, fuzzing on your functions and binary inputs. I can still think of a myriad of ways of compromising a distributed system like that. That's not to say, "do nothing" and more to say, this is a very complex problem and something like an "SBOM" isn't new or going to solve this problem if replicated. [0] https://www.nytimes.com/2021/01/06/us/politics/russia-cyber-hack.html https://www.nytimes.com/2021/01/06/us/politics/russia-cyber-... [1] https://blog.malwarebytes.com/threat-analysis/2020/12/advanced-cyber-attack-hits-private-and-public-sector-via-supply-chain-software-update/ https://blog.malwarebytes.com/threat-analysis/2020/12/advanc...

bryanrasmussen 6y ago

package managers tend to give you something wherein you can read exactly what is in your solution, which I suppose is why people are always reading what their dependencies are and making sure that everything is updated as it should be and security holes plugged. So, given the existence of modern package managers, surely problem solved.

mikewarot 6y ago

Wrong root cause analysis. You'd have to solve the halting problem to get anywhere this way, which is proven impossible, thus it is the wrong path. Real cause: Widespread adoption of Operating Systems that don't default to capability based least privilege.

dathinab 6y ago
My dream, a OS with a good proper well designed single capability system with reasonable defaults. Not the mess Linux has which supports capabilities for some privileges but not so much for others, which has bad defaults and which is supper fragmented. I mean for a full cover you need to correctly combine root/sys capabilities with seccomp with bpf with cgroups with polkit with some pctrl settings with linux kernel parameters and even then you probably still need to throw in selinux and I still missed at least file permissions when writing this... Which lets be honest is just ridiculous.

jacques_chester 6y ago

We need SBOMs, but these are not enough. We need supply chain attestations, but these are not enough. What we need is the combination of asset data, process data and to acknowledge that our knowledge of both is always incomplete and subject to change. I call this need a "universal asset graph" and I've been nagging folks for years to get us to it. The sigstore project is the biggest foundation stone of what I'd wish for, at least in terms of creating a robust shared log of observations (a leader of that effort, dlor, is in this discussion). But we still have a very, very long way to go as an industry.

Ericson2314 6y ago
Just look at Nix. Here's the thing, having the sellers of unfree software compile the code for is a terrible skeuomorphism from the way traditional products are made. The final integrator should be the one building the code even for propriety0 and unfree software, whose secretiveness should be enforced with contracts not obfuscation and baking in specific dependencies. The fact that the finally compilation graph, and the IP procurement graph have some similarities should just be a coincidence.
- jacques_chester 6y ago
  I didn't follow your argument. Could you elaborate?
  - Ericson2314 6y ago
    Right now I'd you buy property software, you get a binary of some sort. That's neither composible or auditable. You should get the source code and a reproducible build, that you can modify and integrate with other things. Kinda like licensing closed source game engine parts, or getting hardware design libraries (IP as they say) for making your own system on a chip.
    - jacques_chester 6y ago
      I think I follow now. Source availability is preferable but it will take a while to become normal. It will be even longer bit-for-bit reproducibility is a commercial norm. SBOMs still give us value in the meantime. If I buy product X, which asserts using dependency Y, then when a vulnerability is asserted for Y, I can pester the vendor to show that they have updated. At this point if they claim to upgrade but haven't, that becomes fraud. The economic incentives vs our current anything-goes world are differently weighted.

dzink 6y ago

Server logs are already full with calls to post to different pages, or php scripts of vanilla wordpress installations as attackers try to find vulnerable sites. Wouldn’t a SBOM make the bad guys job easier? If you are a bad actor or a malicious state actor who has just gotten hands on a new exploit the SBOM would give you an instant menu of a available hackable sites. A B2B vendor or Saas vendor can definitely make their stack available to clients upon deal negotiations, but putting it on the open web is also asking for trouble.

jacques_chester 6y ago
With an SBOM, attackers gain an advantage, but it is highly concentrated in that attacker. Defenders also gain, the gain is highly dispersed amongst defenders, but each gains ~ the negative value of the attacker's gain. Put another way: there are far more defenders than attackers. When something helps both attackers and defenders, the gains of defenders outweigh gains of attackers.

kohlerm 6y ago

What IMHO really is needed are https://reproducible-builds.org/ https://reproducible-builds.org/ plus some way to verify within a company that only allowed packages are used. One way to solve this is to check all software (including open source one) into a monorepo and run software that checks for copies of open source code.

dathinab 6y ago
Reproducible build require source code access. Which is the first point I think is necessary: - Access to source code for at least all entities using the software (including allowing hiring entities to analyze it). Preferable open access to source. Even more preferable open source. - Combine that with reproducible builds and automatic code analysis and you gain additional trust. - Naturally this both requires proper code and artifact signing (which was compromised in the supply chain attack this article refers to). Funny thing is I'm 100% convinced no SBOM, reproducible builds or similar would have prevented this attack. It would just have changed how exactly the attack looks (IMHO). Still it would be an improvement anyway.
- jacques_chester 6y ago
  Reproducible builds would've had a sporting chance of defeating the solarwinds attack, because the injection point was on particular build servers that were reached through other vectors. If a second party performs a build to verify it, the attacker now has ~2x the cost to conceal their attack.
  - dathinab 6y ago
    True, but this requires the source you build not to have been gone through the build server, which is reasonable. The thing is you can circumvent this by attacking the version control and/or developer systems. And at least the later one are often massively vulnerable to certain kind of supply chain attacks. Ironically the permissions and setups commonly used for a nice development flow are also making systems vulnerable for many kinds of supply chain attacks. I'm currently slowly moving to a more secure dev flow, but it adds overhead. Especially if your dev system is also your laptop. First step is to run any kind of dev tool (especially builds) in a container. Through this often also means running e.g. a language server in the container while running your IDE out of it and making sure nothing will trigger your IDE to do thinks outside of the container...

redleggedfrog 6y ago

For 95% of software will never happen unless SBOMs make the company money.

Jupe 6y ago

Ramblings on these topics... Exposing SBOM on every piece of delivered software will just make a hackers job easier and quicker... Since by design they are machine readable, SBOMs will make querying for specific vulnerabilities trivial. This is not a top-down problem! Any upper layer can be compromised by a lower layer (os, build tool, library, reporting tool, etc.) this problem can only be solved Botton up : from verified OS, to verified (bootstrap) build tools of that OS, to every library installed on that OS, etc. We currently have decades of software resting atop of unverified libraries resting atop of unverified operating systems, all built with unverified tooling. We can't even build verification tools that are, themselves, verified! And if we could, can we even say they verify every potential vulnerability? (mitm, boundaries, race conditions, cpu cache, etc.) I know there is research at some universitys into formally verified OS's, but it's a long way off IMO. This is the problem of our time. But, unfortunately, the industry seems consumed with velocity and cleverness over stability and security.

silly-silly 6y ago
> I know there is research at some universitys into formally verified OS's, but it's a long way off IMO. I believe seL4 is verified and used in production ( https://sel4.systems/ https://sel4.systems/ )

l8again 6y ago

The idea of SBOM, or BOM is not a new one. Maven already has one for years now called BOM files [1]. So am I right in assuming that the author's suggestion is to make this BOM file public like the ingredients in food product as some people have suggested? If so, I can see a quick static analysis tool that can spit out the vulnerabilities just by parsing the BOM. So, really there's nothing to do much _technically_ here other than releasing out in the open the BOM. And then displaying ugly warnings about any software's BOM to either shame them or actually hurt their bottom line by lost revenue due to those vulnerabilities. [1]https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#bill-of-materials-bom-poms https://maven.apache.org/guides/introduction/introduction-to...

l8again 6y ago
Also, I wonder how can we realistically implement this for SaaS?
- dathinab 6y ago
  Implement yes, but will it make a relevant difference? IMHO Unlikely. A BOM no one (of relevance) ever reads is as good as no BOM. The main positive effects a BOM can have (outside of SaaS) is to more strongly discourage to use (continue to use) of known to be problematic libraries or services.

dathinab 6y ago

Let me tell you a secret (not really): - supply chain attacks are not new - sophisticated attacks are not new - what made microsoft call it the most sophisticated attack was *not* because of it using a supply chain attack, or waiting or any of the major bullet points listed when speaking about it but the combination of them and all the small details skipped - through it might be the first sophisticated attack using supply chains *which we know of and which got a lot of press/success*. - Neither new are attack which are possible at massive, potential global scale, even without supply chain attacks. So there has been a long time a lot potential for people losing, trust. There is a reason why many tech affine people don't trust governments or large institutions which sensitive data, we know the even the largest institutions will not always manage to keep our data safe. - "does not take the coordinated effort of thousands of engineers in a nation-state" all sophisticated large scale attacks do *not* need thousands of engineers. Most times it's done by teams with noticeable less then 100 people. Nation-scal agressors are not that because they have so many (evil;) programmers but because they have access to nation-scale resources, like access to internet infrastructure, vulnerabilities known by the secret service or just money they can use for probing or obfuscating DDOS attacks. Lastly governments can have an easier job to bring a lot of expertise together, by bringing very qualified people together, not very many. - "load external modules" has nothing to do with supply chain attacks, while it sometimes can make thinks simpler if you can affect the source-code (which is what supply chain attacks are about) you can do everything you need without any "module loading". - "few risks unique to the software domain", I would say more than a few, and they are well known since over a decade - "others are proprietary", which is a problem as it undermines many ways a user can try to detect/catch supply chain attacks. Non of which are perfect, like non would have cough this attack. But it still makes attacks of this kind harder. - "Dependency hell" the only real problem, libraries with non-clear and brittle and to much much changing interfaces, at the same time surprisingly language dependent. - "easier automated than what defenders need to do.", I wouldn't be sure about it, the same way attackers can scan for problem you can do so (and if you find something fix it). Tools like fuzz testing and other kinds automatic software analysis are widely underused. - "96% of all software products included third-party software components and commercial" it tries to make this look like a problem but actually the only way to avoid security vulnerability (and build modern software) is by using external tools. If everyone would implement e.g. their own crypto there would be more vulnerabilities and automatic scanning for them would *still* work as people tend to make similar mistakes when writing similar code. - "and now also by nation state actor", not it should be "and since the beginning". Just look at crypto wars, or how currently the US/EU governments again try to undermine software security for questionable reasons. - "a bill of materials protects best against supply chain vulnerabilities when it is set up as a holistic cross-domain effort"... no it doesn't. There is nothing in a bill of materials which is able to prevent supply chain attacks. I have no idea why the author believes it. At best it can make it easier to catch if a known-to-be-vulnerable dependency is used. But supply chain attacks are about about making a dependency vulnerable without anyone knowing and *by changing it*. Not just by adding code. I guess this is build on the misconception that not having "loading of external modules" would help against supply chain attacks. It doesn't. It just makes it negligible harder. -by the way open accessible source does imply a bill of materials (being derivable form the source) and allows you to automatically run software analysis tools on the source code etc. - "medical industry has been targeted for years" and banking systems, operating systems, mail programs and games have been attacked even longer. The medical industry is interested in this to *avoid legal problems by using absurdities like software being "certified" to be secure (e.g. by using SBOM's) and similar*. A concept which is known to not actually work but tends to work to avoid legal responsibility. If they would care they would push for open accessible source, proper bug bounty programs and similar. - "What needs to be done?" (In my opinion:) Push for at least open accessible source (!= open source). Push for more usage of software analysis tools. Push for proper bug bounty programs (many existing ones are questionable). *Enforce legal liability IF it's clear the software "seller" didn't care for making and keeping the software secure.* (I.e. Liability if negligent). Push for using additional protection which reduce attackers gain even if there are vulnerabilities. E.g. migrations like address layout randomization and shadow stack, sandboxing, dropping privileges, etc. WebASM might play a major role in this. Push for responsible choice of language, libraries and tooling. Push companies to take more responsibility for open source tools they use (!= open accessible source). Lastly sometimes (not seldom) they *way* you use a certain dependency and for what you use it is making a *extreme* difference wrt. security. One example would be alternative hash algorithms in the rust eco system. Use them for certain internal use-case and all is fine. Use them at the wrong place and they enable hash map based DOS attacks. Another example would be openssl which you can use responsible, but also can use in horrible insecure ways. In the end a SBOM is a thing which *should* be cheap to create (potentially automatized) but will only yield a slim improvement in general. It is IMHO *by far* not the most important step to do, but I guess it's one of the most easiest steps to do. So why not, but don't believe it will make any major difference.

euph0ria 6y ago

Are people publishing on Medium in order to earn money now or just that the large migration to a free platform is yet to occur?

dathinab 6y ago

Any supply chain attack can be done in ways where it's externally impossible to (automatically) differentiate a malicious change from a intentional change. Which means BOM's are 100% guaranteed not to prevent supply chain attacks. At best it makes them a small bit harder. Software BOM's can have some small benefits, but preventing supply chain attacks is not one of them. And open access to source always is much more useful then just a BOM. And like a BOM without signature isn't that much trust able (with closed source) accessible source without signing is not trusteable without reproducible builds. Anyway signing source code isn't hard so no reasons not to do so, signing build artifacts can be harder.

silly-silly 6y ago
> signing build artifacts can be harder. Is this like signing an RPM package?
- dathinab 6y ago
  Somewhat. It's not really hard just a bit harder. Mainly on your desktop system you can easily use a hardware security key (like a yubikey) to get a good level of signing security. But your CI server has a good chance to not be your own hardware so you can't plug in a yubikey so to have good key security you either need to find a CI server where you have a HSM or you need to have some form of signing server/service to keep your private key save. You also should at least consider things like "could the artifact be changed in between your CI creating it and your build signing server/IP-based HSM signing it". There are ways to handle it, with different benefits and drawbacks. And considering and understanding this things is maybe the biggest additional amount of work. Or you use a schema-F solution which seems reasonable without understanding it and be okey with it.

fmajid 6y ago

"However, there is an even more worrying effect: if attacks are possible at such a scale that their effect is felt across whole sectors, countries or even globally, as is the case with the “SolarWinds” attack, they have the potential to fundamentally undermine our trust in information technology." How is that a worrying effect? Trust in IT systems is undeserved, because they are built in an environment of economic incentives that make stability, performance and security secondary concerns to features and time to market, and poor engineering practices that derive from those incentives. A software BOM will do nothing to address those problems. Healthy skepticism of IT and adapting with defense in-depth and other measures like data minimization (you can't leak what you don't have) are desirable, or even simply using less software.

hyko 6y ago

A software BOM will not address these issues. Stopping the delusion that any one nation can come out ahead in this game by hoarding vulnerabilities, and working towards establishing and enforcing strict rules of cyber warfare are the first step.

Const-me 6y ago

When I develop software, the source code repo contains a text file with all the third-party stuff I have used, both linked and copy-pasted, along with the URLs where I got the code and their licenses. Not precisely a BOM and I maintain them for different reason, but overall I think pretty close to what’s proposed. Couple examples from my open-source projects: https://github.com/Const-me/vis_avs_dx/blob/master/legal.txt https://github.com/Const-me/vis_avs_dx/blob/master/legal.txt https://github.com/Const-me/Vrmac/blob/master/Pre-existing%20IP.txt https://github.com/Const-me/Vrmac/blob/master/Pre-existing%2...

mybrid 6y ago

Bills of Materials work with hardware because origins can be tracked. Good luck tracking the origin of electrons. I worked in manufacturing for a company the dealt solely with MilSpec (Military Specification). Crates sent to Nuclear Power plants had to X-Rayed on the shipping dock and then again at the receiving dock. If the X-Rays differed in any questionable way the shipment was rejected. However, most supply chains are not that paranoid. In the 1980s there was a grade eight bolt scandal for a satellite that blew up in space because the manufacturer substituted plain steal to make more money. More recently the BOM did nothing to protect the Bay Area Bridge where once again bolts as well as rods were specified of one quality and delivered as another. The bolts and rods are still in the new span because taking them out would require tearing it down again. But the builder assures us things are fine, wink wink, nod nod. https://www.courthousenews.com/34m-settlement-reached-for-defective-work-on-bay-bridge/ https://www.courthousenews.com/34m-settlement-reached-for-de...

patcon 6y ago
Aren't you just talking about checksums? Your xray anecdote is neat, but the software equivalent of that paranoia is even simpler and easier to operationalize if there's incentive to do so. e.g., https://reproducible-builds.org/ https://reproducible-builds.org/
- mybrid 6y ago
  Materials can be traced to point of origin, especially nuclear active material. There is no such tracing with software. A checksum only guarantees that two things have equal software. It does not say the difference of origin between the two.

SergeAx 6y ago

Can we please stop calling SolarWinds hack "the most sophisticated and protracted intrusion attacks of the decade" already? It was nothing more than blunt negligence.

0xbadcafebee 6y ago

> we need to enforce global software supply chain management now. Well, we can't. We don't have the infrastructure for it. It will probably take every industry in the world between 4 and 10 years to have a fully secure supply chain. > The world runs on software. Close: it runs on hardware, and that hardware isn't secure either. It also runs on networks, and authenticated, authorized communications, and those aren't secure either. > However, there is an even more worrying effect: if attacks are possible at such a scale that their effect is felt across whole sectors, countries or even globally, as is the case with the “SolarWinds” attack, they have the potential to fundamentally undermine our trust in information technology. Holy shit. Nobody tell this guy about the NSA hacking Cisco, he might get really upset. > an attack on a low system level may be able to corrupt or circumvent protective measures on higher levels. ..... Unless you design it not to. > Equifax’s 2017 leakage of hundreds of millions of customer records, was possible because the organization failed to update a vulnerability in a low-level web server module No. It was possible because Equifax had no accountability. It was also possible because their network and data access policies were Swiss cheese, and only after that was it the fault of somebody not patching a shitty app. The rest of his argent is that the biggest problem we have is patching and malicious code in our stacks. But even without supply chain attacks you still have 0days which are the same problem: shitty code gets exploited. You can't avoid shitty code. All you can do is mitigate risk. What stops big attacks is not one single fancy idea, but putting enough hoops in place to make big attacks extremely rare and difficult. That, and educating developers on how to not write shitty code and build shitty systems. Most developers I know literally don't even know how to avoid SQLi, or if they do know how, they're too lazy or overworked to do it. They pick random shitty tech and clumsily paste it together to get some MVP working, and then that becomes production. Nobody wants to pay for real components and solutions that were designed the right way and certified. Everybody just wants to write brand new shit code from scratch in fucking JavaScript using a randomly assembled bunch of modules a retarded hamster could write, and then make it so they can't be extended and improved over time, because they're owned by some jackass worse password is Welcome123! Supply chain is just what's trendy. The real fix is to get people to stop trusting morons to make shitty products, and stop paying people to write custom applications that all suck.

b-g-m 6y ago

There also needs to be a process to validate the SBOM matches the compiled software with an air gapped digital twin. ( you have to bypass a few things in the header - like time stamp and a unique I’d) so both versions of your compiled software have the same hash. Then you need to protect that file from interception and still validate every line change in every file. This patent addresses many of these issues with a self-referencing SBOM on the blockchain. https://patents.google.com/patent/EP3687107A1 https://patents.google.com/patent/EP3687107A1

cc_stoic 6y ago