4 ms·

Every time these issues come up for discussion, my mind always returns to some sort of imagined utopia where our software is much more modular, composable and c

by parley 13y ago

Every time these issues come up for discussion, my mind always returns to some sort of imagined utopia where our software is much more modular, composable and compartmentalizable (grammar?) than todays "mainstream" platforms. It could enable several different things, like running systems where most modules have access to much less resources of different kinds, allowing less mischief (microkernels, hierarchical resource mgmt frameworks like GenodeOS, etc). Greater modularity would also mean that basic or common modules would need less changes, requiring new audits less regularly, thus decreasing that load on the community. Of course many vulnerabilities are the result of unexpected interactions between modules, but then a certain composition of certain modules could be audited and hopefully not require any changes for some time. Being a software engineer I'm not kidding myself with regards to the huge software engineering problems inherent in achieving that, and of course the open source community already performs a lot of code reuse. Many will argue that an argument for more reuse is an argument against fragmentation and thus an argument against experimenting and forking. I guess that's true, but I still feel that more could be split and shared while still experimenting on many other things. There will also always be politics, personalities and the will to reinvent wheels for many reasons, like self-education or implementation pet peeves. Also, different languages and/or runtimes/VMs and their differing suitability for different environments affect fragmentation greatly. We probably won't ever end up with a single "winner", no matter how much many C/C++/JS/Go/Rust/Haskell/ATS/theorem provers we go through, and for reasons (only some of which are mentioned above) we probably don't want to. Dreaming is nice, though.

chubot 13y ago
Definitely. One thing I've noticed is that constant code churn causes a huge amount of instability in many software domains (cloud, desktop, phone). The problem is only getting worse and will continue to get worse as software runs more of our lives and the world. Apple is sending down updates all the time; Ubuntu is; Google is, etc. Nobody can keep track of what the hell they're running. It's a security nightmare. There are so many components to systems now -- and not all of them need to change all the time. What I imagine, and what you seem to be hinting it, is that we have to do is factor software into processes which vary by the amount of code churn. Instead of having 1000 modules being patched every week, have 500 modules which change every 5 years, 300 which change every year, ... and maybe 2 or 3 which change every day. This would let us maintain the pace of software innovation. Unix does this to some extent -- think about when you would have a REAL need to upgrade "grep"? Almost never. It's basically hardware at this point. Likewise, with the architecture of Chrome and the Quark browser yesterday on HN, you could just update the rendering engine, or the JS engine, which run in restricted processes. Browser kernel updates could be separate, and verified separately. Related paper I found interesting: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.180.6737 http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.180.... We are moving closer and closer to "ubiquitous computing", and I agree with that paper in that software upgrade has to be treated like a first class problem. Right now even the best vendors are too sloppy about upgrades and modularity. I am constantly being nagged for updates on every device, and at work I am constantly dealing with shifting sands underneath my code.
- parley 13y ago
  Thanks for the paper link, it looks interesting. Yes, factoring software into different processes and enforcing proper sandboxing and resource restrictions is definitely one aspect of it. Hierarchies of such processess where resource allocation can be delegated makes it even more interesting. Another aspect is composing any one process of - where possible - shared, audited modules that haven't changed for a good while and that you're able to verify individually before building your binary. It requires proper code signing and delegates the problem to the domains of who to trust (and of course issues like Thompson's famous Trusting Trust) and once again I'm not kidding myself about the difficulties of that either. But that shouldn't stop us from working towards such a goal if we find it desireable. This of course helps closed source software production, but even more important is for any individual or FOSS organisation to be able to build their own system of (a myriad of) components that can be cryptographically verified to not have changed since some specific audit event, provided that these signatures of single modules and signatures of compositions of modules can be kept by a public and decently trustworthy actor or preferrably actors, like Linux distributions or FSF or whoever. In a world where the loyalties of individuals are easily purchased we need to keep each other honest. This extends from Joe building his own doubly-linked lists and red-black trees to crypto modules that take side-channel attacks into account to kernels and drivers and everything. Many will argue that layers and abstractions can kill software with complexity. I certainly agree that they can if used poorly (or, quite frankly, if not used with great skill) but they are also one of the only things that can make complex problems manageable, and - as you say - our systems are now hilariously complex.

by parley 13y ago

chubot 13y ago

Definitely. One thing I've noticed is that constant code churn causes a huge amount of instability in many software domains (cloud, desktop, phone). The problem is only getting worse and will continue to get worse as software runs more of our lives and the world. Apple is sending down updates all the time; Ubuntu is; Google is, etc. Nobody can keep track of what the hell they're running. It's a security nightmare. There are so many components to systems now -- and not all of them need to change all the time. What I imagine, and what you seem to be hinting it, is that we have to do is factor software into processes which vary by the amount of code churn. Instead of having 1000 modules being patched every week, have 500 modules which change every 5 years, 300 which change every year, ... and maybe 2 or 3 which change every day. This would let us maintain the pace of software innovation. Unix does this to some extent -- think about when you would have a REAL need to upgrade "grep"? Almost never. It's basically hardware at this point. Likewise, with the architecture of Chrome and the Quark browser yesterday on HN, you could just update the rendering engine, or the JS engine, which run in restricted processes. Browser kernel updates could be separate, and verified separately. Related paper I found interesting: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.180.6737 http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.180.... We are moving closer and closer to "ubiquitous computing", and I agree with that paper in that software upgrade has to be treated like a first class problem. Right now even the best vendors are too sloppy about upgrades and modularity. I am constantly being nagged for updates on every device, and at work I am constantly dealing with shifting sands underneath my code.

parley 13y ago
Thanks for the paper link, it looks interesting. Yes, factoring software into different processes and enforcing proper sandboxing and resource restrictions is definitely one aspect of it. Hierarchies of such processess where resource allocation can be delegated makes it even more interesting. Another aspect is composing any one process of - where possible - shared, audited modules that haven't changed for a good while and that you're able to verify individually before building your binary. It requires proper code signing and delegates the problem to the domains of who to trust (and of course issues like Thompson's famous Trusting Trust) and once again I'm not kidding myself about the difficulties of that either. But that shouldn't stop us from working towards such a goal if we find it desireable. This of course helps closed source software production, but even more important is for any individual or FOSS organisation to be able to build their own system of (a myriad of) components that can be cryptographically verified to not have changed since some specific audit event, provided that these signatures of single modules and signatures of compositions of modules can be kept by a public and decently trustworthy actor or preferrably actors, like Linux distributions or FSF or whoever. In a world where the loyalties of individuals are easily purchased we need to keep each other honest. This extends from Joe building his own doubly-linked lists and red-black trees to crypto modules that take side-channel attacks into account to kernels and drivers and everything. Many will argue that layers and abstractions can kill software with complexity. I certainly agree that they can if used poorly (or, quite frankly, if not used with great skill) but they are also one of the only things that can make complex problems manageable, and - as you say - our systems are now hilariously complex.