9 ms·
We've learned nothing from the SolarWinds hack
- LtWorf 3y ago> "We should sign and verify all our dependencies" pypi: "YOLO, let us deprecate signatures!"
- esafak 3y agoIf pypi won't let you add signatures, can you generate one of your own (for internal use) using a repository manager like jfrog, operating as a pypi proxy?
- hiatus 3y agoNew signatures cannot be uploaded to pypi https://blog.pypi.org/posts/2023-05-23-removing-pgp/ https://blog.pypi.org/posts/2023-05-23-removing-pgp/
- LtWorf 3y agoThey email me at every upload telling me to stop sending them my archaic signature. It seems the focus now is more to keep the secrets on github and authenticate with those. https://discuss.python.org/t/2fa-usability-on-pypi-and-with-packaging-tools/38167/29 https://discuss.python.org/t/2fa-usability-on-pypi-and-with-...
- api 3y agoThe sheer amount of eggs in the GitHub basket is absolutely terrifying to me. If something happens to that thing, the entire computing world will halt. It reminds me a bit of what the journalism world did by becoming so reliant on Twitter for sourcing and communication, but in our case I think it's deeper and worse. We've hard coded GitHub into countless systems. I'd venture to say that most CI systems, package managers, build farms, etc. would break.
- LtWorf 3y agoDebian would keep working :D And at work my team uses a mirror on LAN. The other teams on the other hand…
- LtWorf 3y agoI use .deb packages mostly rather than pip. Signatures are checked if present, at least. Also packages that stop working because python removes stdlib modules that they use, tend to get found and patched in distributions. In pypi if they are abandoned, they will be unusable for all eternity.
- chha 3y agoNot sure about jfrog, but Sonatype does something similar. They basically hash all components/packages from a bunch of different repositories, and then tag the hash with various metadata you can use to create policies. I started using this in my org a couple years back, and we've ended up using it to check commercial software as well just to get an overview of known components, vulnerabilities and things to watch out for. I really wish the big repositories would invest more in useful mechanisms; when we looked into this before making our decision, the only repository with any kind of checking was Maven Central. Nuget had support for author signing and repository signing, and Pypi (at the time) author signing. As far as I remember none of the other repositories had any verification of anything including the git repo, so you couldn't even determine what commit hash the code was based on or who was behind it.
- lijok 3y ago> There is absolutely no way we can perform a full binary analysis of every new version of every binary blob that powers modern IT I wonder if there's enough demand for a service like this to be viable. Bundled with a universal package manager, signed and verified binaries, caching mechanisms, etc.
- throwawayapples 3y agoI'd settle for a source analysis and reproducible builds for just our myriad open source dependencies. All it takes is a single developer to be compromised in the thousands throughout a typical stack..
- hotnfresh 3y agoMaking and maintaining a universal package manager then getting it deployed completely enough at any large org to make a difference strikes me as on the order of “let’s build a perpetual motion machine that’s also a fusion reactor” as far as feasibility level. … now, that doesn’t mean one couldn’t make money while utterly failing to ultimately deliver the promised value.
- ryukafalz 3y agoI'm sure there's enough demand for someone to sell a service that attempts this, but for several of the reasons mentioned in the post I expect it would be ineffective. When you run all your code and all its dependencies with full authority, it only takes one tiny piece of malicious code to blow the whole system wide open. I think scanning will always be a losing battle.
- lrvick 3y agoWorking on this right now.
- debatem1 3y agoOh?
- ashishbijlani 3y agoPlug: we've been building Packj [1] to detect malicious Python/NPM/Ruby/Rust/Java/PHP packages. It carries out static/dynamic/metadata analysis to look for "suspicious” attributes such as spawning of shell, use of files, network communication, use of decode+eval, mismatch of GitHub code vs packaged code, and several more. 1. https://github.com/ossillate-inc/packj https://github.com/ossillate-inc/packj
- neilv 3y agoA deeper root cause arguably of this and countless other security incidents involving software: https://xkcd.com/2030/ https://xkcd.com/2030/ > I don't quite know how to put this, but our entire field is bad at what we do, and if you rely on us, everyone will die.
- nuc1e0n 3y agoPutting all your eggs in one basket sure saves money as you don't need to buy extra baskets :P
- 1970-01-01 3y ago>The Inconvenient Truth about how to actually fix this The security game is just too fast paced for a profitable business. Until we reach the point where it makes financial sense to move slowly and take extra time to ensure critical business systems are secure, nothing will be fixed.
- LtWorf 3y agoYou mean we should make companies pay hefty fines
- drivebycomment 3y agoThe inconvenient truth is that people say they want more, better security but they don't want to pay for it. So to survive you need "just enough" security to get by, and that "just enough" turns out to be quite low. It's also not clear if a significantly better proactive security is necessarily better - humanity is still collectively figuring out how to balance the security/privacy vs the convenience, and we do risk fossilized information technology stacks if we jump on the security too hard. That said, it's often easy things that get the most benefit and we are collectively a long way away from getting the basic security things done properly across the board.
- LtWorf 3y ago> The inconvenient truth is that people say they want more, better security but they don't want to pay for it. They are already paying for it, and not getting the security.
- Loughla 3y agoHow do you differentiate between honest to goodness mistakes and a willful/malicious attitude toward just getting (product) done to make money at all costs? It seems like a fine structure would have to take that into account? Or am I way off base?
- ReactiveJelly 3y ago
- spaceribs 3y ago> If we could agree on a good, standardized capabilities model for software and everyone starts using it, we will have reached security Nirvana. I want to get there, but I also don't trust anyone except (maybe) myself to determine what capabilities are "good" versus "bad". I literally just spent most of today messing with my 2018 Samsung TV trying to get rid of the bloatware they pre-installed. I had to do that because the apps took up 96% of the storage within. These apps cannot be deleted normally. If we're heading towards a place where vendors are the ones deciding what users are allowed to do, as well as what the software is allowed to do, and at what level of granularity, I have no reason to trust them any more than I trust a russian state-sponsored hacker.
- jk_i_am_a_robot 3y ago"I literally just spent most of today messing with my 2018 Samsung TV trying to get rid of the bloatware they pre-installed removed." Can you elaborate?
- bimguy 3y agoHe did elaborate. "I had to do that because the apps took up 96% of the storage within." Literally the line right after the one you quoted...
- c420 3y agoI second this, please. I can't even update the preinstalled apps I use on my 2018 Samsung TV because they keep adding more default apps and there's literally zero free space with nothing but their default apps installed.
- spaceribs 3y agoI combined a bunch of awful "tech tips" websites into a working approach: 1. Reset my TV to factory settings. 2. Go into the "apps" item while it desperately tries to update itself 3. Go into "Developer Mode" by pressing 12345 on the remote. 4. Restart the TV by holding down power button for 2 seconds. 5. Open "Apps" again 6. Open the cog. 7. Under the app you want to remove, press "Deep Link Testing" and then hit cancel. 8. If the "Apps" app hasn't been updated yet, your delete button should magically reactivate. I was able to delete the product manual which is like 90mb, all others were reloaded after another restart though but it gave me enough wiggle room to update and install other apps.
- gjsman-1000 3y ago> If we could agree on a good, standardized capabilities model for software and everyone starts using it, we will have reached security Nirvana. That's about as likely as everyone agreeing on a good, standardized operating system architecture, API, and driver model that also happens to be Written In Rust™ for good measure.
- dheera 3y ago> Some people carry around headphones, smartwatches, etc - but some don't carry any devices at all (or keep Bluetooth off on their phone). I wonder if one could do better by using a HackerRF and spy on LTE+WiFi communications. You wouldn't need to decrypt them, just identify how many different devices there are.
- kevindamm 3y agoa lot of devices, especially phones, will randomize the hardware MAC address when on an unknown network or doing open pings, but if a known SSID is found it will use a consistent, and often correct, MAC
- Terretta 3y agoOver time (particularly since iOS 6), less and less permissions have been granted to iOS apps by default, instead requiring apps to request those permissions from users explicitly. It's still not perfect (like access to contacts still being a binary "yes/no"), but every permission clawed back from the default set required breaking backwards compatibility, a phrase rarely uttered in regard to the Linux and Windows kernels. If you have been an iOS developer since 2012, I'm sorry you had to go through that, but your extra work has been profoundly important to the privacy and security of mobile OSes. I'd like to see that same principled energy brought to desktop and server OSes. Or, demand the EU let you side-load so you don't have to bother.
- orf 3y agoSide-loaded apps still have the same permission model?
- nine_k 3y agoYes. There's no way for an app to sidestep that. AFAICT the only way to sidestep things on Android is to root the phone, then explicitly install some native-code binaries that do direct access to things normally controlled by permissions. This is not something that can easily happen by mistake, let alone by normal operation.
- deleted 3y ago[deleted]
- Terretta 3y agoPermissions I've read side-loaders insisting are crucial to them: - permission to ignore my subscription intents - permission to bypass my private relay prefs - permission to share my PII and purchase profile to third parties - permission to share my browsing with adtech - permission to dynamically load unvetted code - permission to bully the user into letting the app out of the sandbox model - permission to bully users into granting elevated privs There are any number of permissions devs think side-loading will give them, and any number of these actively misused by app devs today on MacOS side. A majority of the side-loading table thumpers build business models on the user as the product, the user's data as the thing of value. These firms are not on the user's side. Of course not you, gentle reader. You only want side-loading for good. The government only wants encryption backdoors for the children. If we think back and side doors weaken security posture or tempt legislatures, we should maintain that stance. Instead of building holes we pinky swear are only for the good guys, imagine there were no holes.
- marcosdumay 3y agoWe are still installing obscure binary-only datacenter-wide software with full system permissions for security reasons, and powerful organizations still require that from third parties. If we can't change that, what hope we have for a sane small-kernel capability-based system?
- zx8080 3y ago> We are still installing obscure binary-only datacenter-wide software with full system permissions for security reasons, and powerful organizations still require that from third parties. Do you mean macos? It falls under this definition from what I can see.
- lamontcg 3y agoChef and Puppet and other "control plane" software would satisfy that definition.
- abbeyj 3y agoChef and Puppet are binary-only?
- lamontcg 3y agoChef's omnibus installation is a full ruby interpreter along with associated support libraries like openssl and everything more or less above glibc necessary to run the client. It may not be binary only but there's ~100MB of so of binaries in it. Pretty sure Puppet has something similar with pre-built binaries.
- dvfjsdhgfv 3y agoI believe most orgs switched to Ansible already, not because of binary blobs or their size, but because of the necessity of installing anything more than an SSH server.
- ChrisArchitect 3y agodid this whole site/blog used to be on a different domain? cyrnel.net?
- thadt 3y agoKnowing your supply chain is still going to be an important aspect of fixing this. The author is absolutely right that our build systems can be compromised, but securing them is another part of securing the supply chain. Managing updates carefully is still yet another aspect of securing the supply chain. Auditing (or 'deploy another agent') - is still going to be an important aspect of verifying that various parties are still operating as we would expect. Making these better are all pretty important toward hardening our systems. The proposed solution - 'run software in a way where we don't really care if it has a vulnerability, because it will happen' - is close to reality, but not quite there. The reality is that no matter how good our technical solutions are, trust ends up being a people problem, which means that compromises of that trust are going to happen. No matter how fine grained your capability based system is - if a system has the capability to do anything, then you're going to suffer loss when it is compromised. If a system gets compromised or corrupted and you don't care, then it's not a system you were using for anything useful in the first place. While capability based systems are another important part of the solution, I would counter-propose that what we're actually looking for is compartmentalization. Acknowledging the reality that parts of our system are going to be compromised, and we are going to suffer some loss when that happens, carefully controlled capabilities can help scope that blast radius of compromises.
- _8j50 3y agoI partially disagree to this post. What the author wants is unrealistic and their understanding of detections isn't ideal. I assume they know that a SIEM doesn't need agents. But an EDR does. EDR can be a well configured auditd/sysmon and a response agent. Or you pay up and get a "nobody gets fired for buying crowdstrike". One reality is that APT hacks are out of scope for most orgs. Another reality is that for at least a decade and a half the security industry has accepted that prevention is not a good strategy against APTs. This is why you need that other EDR agent and SIEM. You focus your detection on what the threar actors do after compromise and around your important data. For most APTs this is effective but it is far from perfect. The biggest problem is people like the author who are not seeing actual compromises happen all the time. Security costs money and it can only cost as much as the risk tolerance of the company. It can't cost so much that the company's profit margins take a beating for example. That's why NIST recommendations are important, so execs can say we are spending enough. Another side of prevention is that it needs to be as invisible as possible. If users notice it or it gets in their way, by default it is bad, you have to do things to compensate for the deteriorated user experience. You can add MFA for example but better make up for it by making it a yubikey and doing SSO. Keep in mind that corporate networks are a hodge-podge patchwork of random things pieced together over time. Threat actors need one mistake to abuse. And the P in APTs mean they'll try until they find the mistake, it's their day job quite literally lol.
- colek42 3y agoI think if we can sufficiently isolate the build process we can solve this problem. Lot's of opportunity with our project Witness to add extra isolation. It is something we are working on. However, the real supply chain security "business problem" is just tracking everything in a standardized way. This is what the in-toto project helps with. I wrote about it here: https://www.cncf.io/blog/2023/08/17/unleashing-in-toto-the-api-of-devsecops/ https://www.cncf.io/blog/2023/08/17/unleashing-in-toto-the-a.... we also wrote Witness and Archivista to help solve this problem.. We have lots of work to do. https://github.com/in-toto/witness https://github.com/in-toto/witness Full disclosure, I am a member of the steering committee for in-toto and the CEO of TestifySec which is the main contributor to Witness.
- fyokdrigd 3y agohow much experience you have with embed? from small iot white label like small business alarm systems to behemoths like Samsung... the only constant is they ship whatever and the lowest interns handle build
- colek42 3y agoStep one is actually wanting to improve security. Those IoT companies have no motivator. Most of our business is with Federal/Defense and Finance. Those companies will only change if liability changes or the regulatory environment forces them to.
- etothepii 3y agoI'm currently trying to get through a InfoSec questionnaire for a large company, one of their requirements has been that we purchase over $1m of cyber limit. The questions that we have to answer are heavily focused on the assumptions that we: 1. Have a network. 2. Have a physical premises. 3. Don't use API keys. I'd happily attest to using AWS IAM properly. I'd happily be told that we aren't using it properly and asked to make changes.
- bradfier 3y agoThese repetitive one-size-fits-all InfoSec questionnaires are a big part of the reason we started Platformed (https://platformed.com https://platformed.com) to automate the process for vendors. As a smaller vendor selling into a larger organisation you end up spending a lot of time rephrasing the same answer, which often boils down to "We don't have this very specific control you're asking for, but we do have an equivalent more appropriate for our size or business which is ...".
- etothepii 3y agoI had a look at the site. Putting the answers in the questions isn't where we struggle it's trying to answer questions sensibly at all that assume a 1990s network topology.
- dang 3y agoRelated ongoing thread: Discouraging the use of web application firewalls - https://news.ycombinator.com/item?id=38255004 https://news.ycombinator.com/item?id=38255004 - Nov 2023 (125 comments)
- mike_hearn 3y agoThere's actually another possible approach to blocking SolarWinds style hacks that doesn't require pervasive sandboxing. Sandboxing is very hard with the current set of primitives operating systems expose, e.g. macOS doesn't expose low level sandboxing primitives as a documented API at all, Windows barely has a sandboxing API, Linux has several but they're all extremely low level and complicated. That other approach is to rely on confidential computing technologies, in particular, the remote attestation capabilities of Intel SGX and AMD SEV. These technologies are often associated with DRM or multi-party computation, but they can be used in a different way. In this alternative use case you statically attach a remote attestation to a piece of data as a signature of computation. A signature of computation is a proof that a piece of data was derived by running a specific program with a specific input. If that program is deterministic, and has been verified by yourself or a third party, then it means the correctness of the transform can be mechanically verified in such a way that it's much harder to hack. In particular if you use SGX then root exploits on the machine doing the computation won't help the attacker due to the hardware level protections. If you use SEV it's harder but can AFAIK still be done with some specialized OS hacking. Concretely this means that a shipped software artifact could come with a signature proving not only the identity of the developers, as is the case today, but that those files were produced using a known third party compiler and linker applied to e.g. a source tree at a specific git commit hash. Because those files can be verified by anyone including the developers themselves, it means that a compromised CI system is very limited in what it can do: every file has a cryptographic proof tracing it backwards to the input source tree, which in turn is replicated independently on developer's workstations and laptops so it can be checked by many semi-independent actors. And for extra value the company can hire a third party auditor who verifies that the source code at hash X does in fact meet the vendor's description of what it does. I've experimented with this sort of approach in the past, but unfortunately CC as a tech gets sort of repeatedly stuck in the domain gap between low level systems programmers and the kind of wideband ecosystem building you need to deliver genuine business value. People aren't aware of what it can do or how to use it. Also, this approach has a subtle side effect: it means the compiler is now a valid target for attack. For example if source code can corrupt compiler internals sufficiently to inject code, then you could potentially cause a miscompile that invalidates the proof. One solution for that is to use memory-safe compilers but the only one I know of is Graal.
- MilStdJunkie 3y agoAh, but I've heard from Reliable Sources that CMMC will fix everything.
- maaryy 3y ago[flagged]