3 ms·

Normally EVERYTHING has both economies and diseconomies of scale. You model the price per unit as the sum of different curves. Complexity not only increases o

Normally EVERYTHING has both economies and diseconomies of scale. You model the price per unit as the sum of different curves. Complexity not only increases on Software, but if you design a thermal engine, or a plane, or a car. Working making something as simple as fiberglass, we had something like 100 components, like tensioactives. Most of them we had no idea what they were for, as they were added like decades ago by someone who new. Nobody wanted to remove a given component and be responsible for the fiber breaking and stopping the line, incurring on tens of thousands of dollars in penalties, so new complexity was added, but not removed. In my experience, software is the thing in which YOU CAN get the most economies of scale possible, because you do not depend of the physics of the world. But you need to control complexity as you develop. In the real world, you create a box because it is the only way of doing something, and the box automatically encloses everything that is inside. You can't see inside, nor want to. It is a black box that abstracts your problems away. In software you have to create the boxes. Most people don't do it, with nefarious consequences.

markokrajnc 11y ago
"But you need to control complexity as you develop." One way to control the complexity is by consistent development of automatic regression tests which cover >90% of functionality. Then you are much less afraid if you remove something or make a big change in production grade software used by many customers.
- iofj 11y ago
  This is one of those seductive statements that is true, but it's also such incredibly bad advice that just this one piece of theory has the potential to completely destroy your company if you follow it. But it's seductive for a reason : it's true. Doing regression tests on this scale will protect you from quite a sizable class of bugs. BUT there is no way to write lots of tests without locking down the implementation of the program. If you limit yourself to black box testing (meaning either no unit tests, or breaking and replacing unit tests is encouraged) you can retain some flexibility for a while. You want to be in the situation that if I go into an application and replace one algorithm with an equivalent one at any point in the app, none of the tests fail. I have never once seen any testsuite do that correctly. In most cases tests complicate an already complex codebase. This leads to having to choose between 2 approaches : either you ignore tests results (which of course does result in bugs), or you always fix tests (which of course leads to the tests becoming tautological over time as different people "fix" them and results in both more effort AND bugs). I don't disagree that for some software locking the functionality down is appropriate. But not for line of business software, apps with large UIs, essentially anything consumer facing ... It is only appropriate for standardized infrastructure software, things that manage accounts, process payments, regulate a power plant, avionics, ...
  - akkartik 11y ago
    I've been thinking about this a lot, and I think a way out might be to drop the phrase 'black box' in your comment. I've been exploring white-box testing by making assertions on the log emitted by a program. Not only does it allow greater flexibility in large-scale reorganizations without needing to mess with tests, it also allows us to write tests for things that we don't currently write tests for, like performance (make sure the number of swaps in this sort function don't exceed this threshold), fault tolerance, race conditions, and so on. More details: http://akkartik.name/about http://akkartik.name/about. An early write-up on white-box testing: http://akkartik.name/post/tracing-tests http://akkartik.name/post/tracing-tests
jlcx 11y ago
Your experience with fiberglass reminds me of http://c2.com/cgi/wiki?OnionInTheVarnish http://c2.com/cgi/wiki?OnionInTheVarnish
EvanPlaice 11y ago
Following your fiberglass manufacturing example. It sounds like you've identified technical debt as it applies to physical engineering. In all likelihood, somebody on the 'business end' made some very bad decisions that were penny-wise/pound-foolish. Long-term sustainability for mechanical systems require paying extra to maintain the design. A lot of business decisions involving engineering are completely reactionary. Changes are usually required when some unforeseen event happens (ex something breaks, market changes, etc). Poor budgeting will lack a contingency plan to cover these costs. Even with a full-time staff of engineers, their role is usually limited to daily operations. Future updates usually require contracting outside support, at a premium cost. When it comes time to do the work, it's in the interest of the business decision makers to reduce the impact on the 'bottom line' as much as physically possible. The work will need to be done on a short schedule to reduce the impact on production. As soon as the changes are complete, it's in the business' best interest to send the outside support home as soon as possible. As a result, design/documentation updates are left undone. Technical Debt: Over time, physical changes accumulate whereas the design stays the same. When the design/documentation is unreliable, the cost of future changes grows dramatically with each additional update. Technical Bankruptcy: At some point it gets so bad that the design/documentation isn't worth the paper it's printed on, nobody knows how the system works, the implementation becomes fragile. Nobody is willing to accept the risk/cost of a significant engineering redesign. The system runs until it fails and the business writes it off as a loss. There are some parallels that can be drawn. Complexity, is inherently difficult to maintain in the long-term. You could choose to favor: - Traditional Design: Monolithic architecture w/ deep-coupling in software vs monolithic engineering designed for production not maintainability. Everything is delivered as one big ball of mud. Even with good test coverage, the sheer complexity makes it difficult to make changes. - Black Boxes: External libraries in software vs COTS components in hardware. The internals are opaque, if it breaks the only option is to replace it. If it becomes obsolete and/or long-term support is cut, good luck. - White Boxes: Internal well-tested libraries/modules in software vs modular/pluggable well-tested designs in hardware. White boxes solve the long-term support issue but require significant resources to maintain/develop. - Gray Boxes: Open source libraries/modules in software; open hardware designs. May not be feature-complete or include sufficient test coverage. The upfront cost is dramatically reduced but there may be license/communication barriers that prevent changes from being made. There are trade-offs and risks that come with each approach. Monolithic designs require better language-level architecture and internal separation of concerns. Modular design require better system-level architecture and external separation of concerns. The only constant is, the greater complexity the greater the cost to maintain.

markokrajnc 11y ago

"But you need to control complexity as you develop." One way to control the complexity is by consistent development of automatic regression tests which cover >90% of functionality. Then you are much less afraid if you remove something or make a big change in production grade software used by many customers.

iofj 11y ago
This is one of those seductive statements that is true, but it's also such incredibly bad advice that just this one piece of theory has the potential to completely destroy your company if you follow it. But it's seductive for a reason : it's true. Doing regression tests on this scale will protect you from quite a sizable class of bugs. BUT there is no way to write lots of tests without locking down the implementation of the program. If you limit yourself to black box testing (meaning either no unit tests, or breaking and replacing unit tests is encouraged) you can retain some flexibility for a while. You want to be in the situation that if I go into an application and replace one algorithm with an equivalent one at any point in the app, none of the tests fail. I have never once seen any testsuite do that correctly. In most cases tests complicate an already complex codebase. This leads to having to choose between 2 approaches : either you ignore tests results (which of course does result in bugs), or you always fix tests (which of course leads to the tests becoming tautological over time as different people "fix" them and results in both more effort AND bugs). I don't disagree that for some software locking the functionality down is appropriate. But not for line of business software, apps with large UIs, essentially anything consumer facing ... It is only appropriate for standardized infrastructure software, things that manage accounts, process payments, regulate a power plant, avionics, ...
- akkartik 11y ago
  I've been thinking about this a lot, and I think a way out might be to drop the phrase 'black box' in your comment. I've been exploring white-box testing by making assertions on the log emitted by a program. Not only does it allow greater flexibility in large-scale reorganizations without needing to mess with tests, it also allows us to write tests for things that we don't currently write tests for, like performance (make sure the number of swaps in this sort function don't exceed this threshold), fault tolerance, race conditions, and so on. More details: http://akkartik.name/about http://akkartik.name/about. An early write-up on white-box testing: http://akkartik.name/post/tracing-tests http://akkartik.name/post/tracing-tests

jlcx 11y ago

Your experience with fiberglass reminds me of http://c2.com/cgi/wiki?OnionInTheVarnish http://c2.com/cgi/wiki?OnionInTheVarnish

EvanPlaice 11y ago

Following your fiberglass manufacturing example. It sounds like you've identified technical debt as it applies to physical engineering. In all likelihood, somebody on the 'business end' made some very bad decisions that were penny-wise/pound-foolish. Long-term sustainability for mechanical systems require paying extra to maintain the design. A lot of business decisions involving engineering are completely reactionary. Changes are usually required when some unforeseen event happens (ex something breaks, market changes, etc). Poor budgeting will lack a contingency plan to cover these costs. Even with a full-time staff of engineers, their role is usually limited to daily operations. Future updates usually require contracting outside support, at a premium cost. When it comes time to do the work, it's in the interest of the business decision makers to reduce the impact on the 'bottom line' as much as physically possible. The work will need to be done on a short schedule to reduce the impact on production. As soon as the changes are complete, it's in the business' best interest to send the outside support home as soon as possible. As a result, design/documentation updates are left undone. Technical Debt: Over time, physical changes accumulate whereas the design stays the same. When the design/documentation is unreliable, the cost of future changes grows dramatically with each additional update. Technical Bankruptcy: At some point it gets so bad that the design/documentation isn't worth the paper it's printed on, nobody knows how the system works, the implementation becomes fragile. Nobody is willing to accept the risk/cost of a significant engineering redesign. The system runs until it fails and the business writes it off as a loss. There are some parallels that can be drawn. Complexity, is inherently difficult to maintain in the long-term. You could choose to favor: - Traditional Design: Monolithic architecture w/ deep-coupling in software vs monolithic engineering designed for production not maintainability. Everything is delivered as one big ball of mud. Even with good test coverage, the sheer complexity makes it difficult to make changes. - Black Boxes: External libraries in software vs COTS components in hardware. The internals are opaque, if it breaks the only option is to replace it. If it becomes obsolete and/or long-term support is cut, good luck. - White Boxes: Internal well-tested libraries/modules in software vs modular/pluggable well-tested designs in hardware. White boxes solve the long-term support issue but require significant resources to maintain/develop. - Gray Boxes: Open source libraries/modules in software; open hardware designs. May not be feature-complete or include sufficient test coverage. The upfront cost is dramatically reduced but there may be license/communication barriers that prevent changes from being made. There are trade-offs and risks that come with each approach. Monolithic designs require better language-level architecture and internal separation of concerns. Modular design require better system-level architecture and external separation of concerns. The only constant is, the greater complexity the greater the cost to maintain.