4 ms·
At the risk of stating the obvious, (b) doesn't progress because it's more expensive and often unnecessary. In the cases where it would be useful, when TCO inco
by contingencies 11y ago
At the risk of stating the obvious, (b) doesn't progress because it's more expensive and often unnecessary. In the cases where it would be useful, when TCO incorporating whole lifecycle costs such as maintenance and/or rebuilds are considered, (b) is still often a failure because it requires higher lead time and up front investment. That's not to say there's no space for very carefully built systems, just that they're a niche and likely always will be, particularly as hardware is now basically free and improved tooling (eg. popularization of CI) helps to reduce the difference between one-off hacks and "robust, maintainable" systems built with knowledge of prior work and theory.
- nickpsecurity 11y agoRegarding expensive, empirical tests found the medium assurance methods to cost either the same or cheaper due to huge reductions in debugging or maintenance issues. This was the case for Fagan Inspections, Cleanroom especially, TSP, and iterative methods with safer languages. Far as high assurance, the level of assurance is usually applied to a high level design or some small, critical component. The cost reported for LOCK was about 30% extra on top of a solid lifecycle. Same percentage for security costs of one Special Access Program. A small team did an EAL7 IPsec VPN in relatively short time. I'd love to know cost overhead of Praxis Correct by Construction. seL4 cost a few million for highest extreme. So, applied correctly, even high assurance methods result in a reasonable premium for certain industries. Niche as you said. Time to market is another story with high assurance killing that. Yet, medium assurance is nearly as fast and around same price as solid lifecycle while paying dividends later. So, the cost and lead time objections are a misconception if we're aiming for medium level which is merely high quality. It benefits long-term better than alternatives. Only exception is bottom-dollar deals and one offs where it barely working is acceptable.
- contingencies 11y agoThe study likely assumed that people already familiar with working in these methodologies were equally available, equally happy to do so, and equally priced as regular developers, which is unlikely to be case. A certain minimum team size is probably required to make many of these processes viable; many real world systems are written by solo coders. Also, in reality it is relatively trivial to scavenge work like seL4, Linux kernel security toolkits, software HA clustering and/or proxy implementations, hardware watchdogs, ssss and/or distribute infrastructure across disparate locations to provide what probably amounts to magnitude higher redundancy and resilience against real world attacks or component failures without paying those costs. Indeed, the implementation of such is becoming near free many cases due to convergence on SoA/microservices. To put this more plainly, "I can get another if I break it, so a clay cup trumps a grail." - Mirza Asadullah Khan Ghalib, classical Urdu and Persian poet from the Mughal Empire ... quote #1 in my fortune db @ https://github.com/globalcitizen/taoup https://github.com/globalcitizen/taoup
- nickpsecurity 11y ago"The study likely assumed that people already familiar with working in these methodologies were equally available, equally happy to do so, and equally priced as regular developers, which is unlikely to be case." For things like Cleanroom and PSP/TSP, they were first-time users of the process. Not long-time. Just decent developers shown a better way of getting results. In high assurance, it varied from first-time to experienced users. First time users usually gained the benefits of the method but with difficulty. There'd be at least one specialist aiding the team. Best results came from people who had done it a while. So, for high that's partially true and for stuff I push on commercial sector it's the opposite. "A certain minimum team size is probably required to make many of these processes viable; many real world systems are written by solo coders. " That's a legit critique. They can still benefit as I experienced myself and some others. It requires one to divide the work into periods where you wear various hats. It takes discipline to externalize one's own work and attack it objectively. Good news is that inspection, testing, static analysis, model-checking, design-by-contract... these all catch problems that are objective more than subjective. Still get plenty of results. There's been one or two threads on here where even solo people do QA on their own stuff with good results. Still open-ended problem worth some experiments to see how much can be done at what effectiveness for a solo coder juggling work. So, good point. "in reality it is relatively trivial to scavenge " It's so trivial we had today yet another HN thread about distributed locks showing how people were copying one implementation and screwing it up. Then there was disagreement in the comments. There's threads like that on various critical things at least once a day or so on HN. Shows it's trivial to do and hard to do right for many things. Better to make re-usable, strong solutions to the various problems we keep encountering where possible. Can always mix high, medium, and average assurance components as a compromise where we have X goals but only Y resources. Pieces can improve over time. "Indeed, the implementation of such is becoming near free many cases due to convergence on SoA/microservices." I've yet to see one go 5 years under heavy load without downtime much less the 17 years VMS clusters did or 30+ years for some IBM mainframe installations. For security, things are more abysmal. I'll agree many solutions are popping up that can do amazing things. Robust, scalable, fairly secure, and near-zero-downtime systems don't seem to be among them in this nearly-free, mainstream market you're talking about. A rarity in the market in general. So, there's still room for both medium and high assurance software. To be clear, I'm only pushing medium because even first-timers did a better job than most do today by eliminating the problem areas & doing great detection. Similar cost, too. Only hoping for high-assurance to be selectively applied to high-impact stuff that doesn't change often. A recent example was a formal verification of SCOOP: Eiffel's race-free, easy-to-analyze, concurrency model. It found issues in the model that were fixed. Three other analysis, ranging from specific to general, eliminated deadlocks, livelocks, and performance-hit respectively. That's the kind of situation where a significant investment of resources can keep providing benefits with almost no effort. Otherwise, medium assurance aka commercial + sane extras. ""I can get another if I break it, so a clay cup trumps a grail."" Nice quote and good principle in much of current economy. Not so much my personal data or business I.P. when a byzantine failure takes out it and my hot standby. I think the comparison mainly applies to non-critical services and hardware. ;)