6 ms·

It is easy to see that the total complexity has been severely reduced in the last years when you look at how many small companies are doing things that required

It is easy to see that the total complexity has been severely reduced in the last years when you look at how many small companies are doing things that required an army of people a few decades ago. For example, running the technology for a real-time banking system nowadays can be done by perhaps even a single developer (with enough experience) thanks to open source, cloud computing and advances in software and hardware. Now we even have distributed relational databases that are relatively straightforward to operate, like CockroachDB.

pron 6y ago
First, Brooks wasn't talking about that. He was talking about productivity boosts due to programming language design and programming methodology. More specifically, he did not exclude improvements to both kinds of complexity overall, but only those that drastically change their relative proportion. Second, in the early 90s I had a summer job at a company whose old ERP, which had been written in a language called Business Basic and ran on a minicomputer, was maintained and operated by a single person.
- jasode 6y ago
  >, Brooks wasn't talking about that. He was talking about productivity boosts due to programming language design and programming methodology. Brooks was also talking about non-programming language advancements as a possible "silver bullets". See the pdf and look for the following sections that are not about programming syntax: Hopes for the Silver Artificial Intelligence Environments and Tools Promising Attacks on the Conceptual Essence Buy versus Build He underestimated AI machine learning. With hindsight, we see that the deep-layer neural net combined with GPU beat human rule-based programming for automatic language translations, DeepMind AlphaZero beats expert programmers hand-tweaking IBM DeepBlue, etc. EDIT reply to : ", nor have they reduced the overall cost of programming by 10x even after more than three decades." I'm not convinced of that. The issue with your conclusion is that it omits the increasing expectations of more complex systems. I'd argue we did get 10x improvement -- if -- we hold the size & complexity of the system constant. E.g. write 1970s style text-based code to summarize sales revenue by region and output it to green bar dot matrix printers. This was tedious work in old COBOL but much easier today with Python Pandas or even no code tools like MS Excel. The invisible factor we often overlook is that our demand for software to do more complex things will always outpace the smaller productivity increases of the tools. This makes it look like our newer programming tools never gave us 10x improvement when it actually did.
  - pron 6y ago
    How did he underestimate statistical machine learning? Whatever achievements were made, they did not take place within a decade, nor have they reduced the overall cost of programming by 10x even after more than three decades. And, indeed, the one thing that Brooks presented as being the most promising direction, i.e. buy vs. build. He said that unlike changes to programming languages and methodology that wouldn't give a huge boost, buy vs. build might (although he wasn't sure about that). So he was exactly right about that. Also, 1986 wasn't 1966. In 1986 people didn't write simple software in COBOL. They used things like Magic and Business Basic and even Smalltalk, and, shortly after, Visual Basic and Access. Excel was released in 1987, and we had spreadsheets in the early 80s, too (and Brooks explicitly mentions them in No Silver Bullet). RAD tools and "no code" was very much in vogue in the late 80s and early 90s. That was Brooks's present, not future. He even says that this is the right direction: I believe the single most powerful software-productivity strategy for many organizations today is to equip the computer-naive intellectual workers who are on the firing line with personal computers and good generalized writing, drawing, file, and spreadsheet programs and then to turn them loose. The same strategy, carried out with generalized mathematical and statistical packages and some simple programming capabilities, will also work for hundreds of laboratory scientists. When generalised, Brooks's prediction amounts to expecting diminishing returns due to reduction of accidental complexity, and we're seeing exactly that. > I'd argue we did get 10x improvement -- if -- we hold the size & complexity of the system constant. Only if we do the one thing Brooks says would work: Buy vs. Build. When we write from scratch -- no way. Even then, I think that while we may see a 10x reduction for specific simple task, we won't see it for large, complex software, which is where most of the effort in software is invested.
    - jasode 6y ago
      >, Visual Basic and Access. Excel was released in 1987, and we had spreadsheets in the early '80s, too. When generalised, Brooks's prediction amounts to diminishing returns due to reduction of accidental complexity, and we're seeing exactly that. The "diminishing returns" of _what_ exactly? That's what I'm trying to make clear. Let me try and restate another way: (1) 10x improvement in programming tasks vs (2) 10x improvement in completing business projects I'm emphasizing that (1) has been achieved many times in multiple areas but it's overshadowed by not seeing (2) happen. I previously mentioned some things that I'm more than 10x faster on now: https://news.ycombinator.com/item?id=23758199 https://news.ycombinator.com/item?id=23758199 Visual Basic is another good example. When I first used VB Winforms in 1990s, it was more than 10x faster than hand-coding the raw C "Wndproc()" message loop. But that legitimate productivity gain is dwarfed by the business wanting new complexity (the app needs to connect to the internet, it needs to be new-fangled web app for browsers instead of a desktop exe, it needs to work on mobile phones, etc, etc). Our new desires for new business functionality multiply faster than the time-savings progress in tools. And "accidental complexity" isn't fixed either. And new deployment environments, new features also add a new set of accidental complexity. E.g. if next generation of apps need to interface to virtual reality (headsets, etc), the programming code will have logic that doesn't have direct business value. So we'll then get new 10x programming tool/library to manage that accidental complexity in the VR environment but then.... we're on to the neural implants SDK and we have no silver bullets for that new thing which means we revisit this topic again. >while we may see a 10x reduction for specific simple task, we won't see it for large, complex software, which is where most of the effort in software is invested. I agree. But again to be clear, today's expectation of "large, complex software" -- has also changed. EDIT reply to: "I'm saying that (1) has not been achieved even within a period of time that's 3x Brooks's prediction, " Raw Windows SDK C language WndProc() was late 1980s and by 1993, I was using Visual Basic 3 drag & drop buttons on to Winforms. Just that one example was 10x improvement within a decade. For line-of-business apps, VB was 10x+ more productive because of the paradigm shift (in addition to things like not worrying about mental bookkeeping of malloc()/free() etc.) >But most tasks cannot be achieved today 10x faster than in 1986 For discussion purposes, I don't know why we have to constantly refer to 1986 even though the paper has that date. It's repeated submission for discussion makes it seem like people consider it an evergreen topic that transcends Brook's days of the IBM 360 mainframe. As another example, the productivity improvement is the writing and deploying complex apps using Ruby on Rails or Javascript frameworks and deployed on AWS. That's more than is more 10x faster than the 1990s CGI days of having C Language code writing to stdout to output HTML. Those early web apps were simpler and yet they were so utterly tedious and slow to code.
paulryanrogers 6y ago
Total complexity must then exclude the layers of hardware, cloud infrastructure, and FOSS contributions that involved many other people.

pron 6y ago

First, Brooks wasn't talking about that. He was talking about productivity boosts due to programming language design and programming methodology. More specifically, he did not exclude improvements to both kinds of complexity overall, but only those that drastically change their relative proportion. Second, in the early 90s I had a summer job at a company whose old ERP, which had been written in a language called Business Basic and ran on a minicomputer, was maintained and operated by a single person.

jasode 6y ago
>, Brooks wasn't talking about that. He was talking about productivity boosts due to programming language design and programming methodology. Brooks was also talking about non-programming language advancements as a possible "silver bullets". See the pdf and look for the following sections that are not about programming syntax: Hopes for the Silver Artificial Intelligence Environments and Tools Promising Attacks on the Conceptual Essence Buy versus Build He underestimated AI machine learning. With hindsight, we see that the deep-layer neural net combined with GPU beat human rule-based programming for automatic language translations, DeepMind AlphaZero beats expert programmers hand-tweaking IBM DeepBlue, etc. EDIT reply to : ", nor have they reduced the overall cost of programming by 10x even after more than three decades." I'm not convinced of that. The issue with your conclusion is that it omits the increasing expectations of more complex systems. I'd argue we did get 10x improvement -- if -- we hold the size & complexity of the system constant. E.g. write 1970s style text-based code to summarize sales revenue by region and output it to green bar dot matrix printers. This was tedious work in old COBOL but much easier today with Python Pandas or even no code tools like MS Excel. The invisible factor we often overlook is that our demand for software to do more complex things will always outpace the smaller productivity increases of the tools. This makes it look like our newer programming tools never gave us 10x improvement when it actually did.
- pron 6y ago
  How did he underestimate statistical machine learning? Whatever achievements were made, they did not take place within a decade, nor have they reduced the overall cost of programming by 10x even after more than three decades. And, indeed, the one thing that Brooks presented as being the most promising direction, i.e. buy vs. build. He said that unlike changes to programming languages and methodology that wouldn't give a huge boost, buy vs. build might (although he wasn't sure about that). So he was exactly right about that. Also, 1986 wasn't 1966. In 1986 people didn't write simple software in COBOL. They used things like Magic and Business Basic and even Smalltalk, and, shortly after, Visual Basic and Access. Excel was released in 1987, and we had spreadsheets in the early 80s, too (and Brooks explicitly mentions them in No Silver Bullet). RAD tools and "no code" was very much in vogue in the late 80s and early 90s. That was Brooks's present, not future. He even says that this is the right direction: I believe the single most powerful software-productivity strategy for many organizations today is to equip the computer-naive intellectual workers who are on the firing line with personal computers and good generalized writing, drawing, file, and spreadsheet programs and then to turn them loose. The same strategy, carried out with generalized mathematical and statistical packages and some simple programming capabilities, will also work for hundreds of laboratory scientists. When generalised, Brooks's prediction amounts to expecting diminishing returns due to reduction of accidental complexity, and we're seeing exactly that. > I'd argue we did get 10x improvement -- if -- we hold the size & complexity of the system constant. Only if we do the one thing Brooks says would work: Buy vs. Build. When we write from scratch -- no way. Even then, I think that while we may see a 10x reduction for specific simple task, we won't see it for large, complex software, which is where most of the effort in software is invested.
  - jasode 6y ago
    >, Visual Basic and Access. Excel was released in 1987, and we had spreadsheets in the early '80s, too. When generalised, Brooks's prediction amounts to diminishing returns due to reduction of accidental complexity, and we're seeing exactly that. The "diminishing returns" of _what_ exactly? That's what I'm trying to make clear. Let me try and restate another way: (1) 10x improvement in programming tasks vs (2) 10x improvement in completing business projects I'm emphasizing that (1) has been achieved many times in multiple areas but it's overshadowed by not seeing (2) happen. I previously mentioned some things that I'm more than 10x faster on now: https://news.ycombinator.com/item?id=23758199 https://news.ycombinator.com/item?id=23758199 Visual Basic is another good example. When I first used VB Winforms in 1990s, it was more than 10x faster than hand-coding the raw C "Wndproc()" message loop. But that legitimate productivity gain is dwarfed by the business wanting new complexity (the app needs to connect to the internet, it needs to be new-fangled web app for browsers instead of a desktop exe, it needs to work on mobile phones, etc, etc). Our new desires for new business functionality multiply faster than the time-savings progress in tools. And "accidental complexity" isn't fixed either. And new deployment environments, new features also add a new set of accidental complexity. E.g. if next generation of apps need to interface to virtual reality (headsets, etc), the programming code will have logic that doesn't have direct business value. So we'll then get new 10x programming tool/library to manage that accidental complexity in the VR environment but then.... we're on to the neural implants SDK and we have no silver bullets for that new thing which means we revisit this topic again. >while we may see a 10x reduction for specific simple task, we won't see it for large, complex software, which is where most of the effort in software is invested. I agree. But again to be clear, today's expectation of "large, complex software" -- has also changed. EDIT reply to: "I'm saying that (1) has not been achieved even within a period of time that's 3x Brooks's prediction, " Raw Windows SDK C language WndProc() was late 1980s and by 1993, I was using Visual Basic 3 drag & drop buttons on to Winforms. Just that one example was 10x improvement within a decade. For line-of-business apps, VB was 10x+ more productive because of the paradigm shift (in addition to things like not worrying about mental bookkeeping of malloc()/free() etc.) >But most tasks cannot be achieved today 10x faster than in 1986 For discussion purposes, I don't know why we have to constantly refer to 1986 even though the paper has that date. It's repeated submission for discussion makes it seem like people consider it an evergreen topic that transcends Brook's days of the IBM 360 mainframe. As another example, the productivity improvement is the writing and deploying complex apps using Ruby on Rails or Javascript frameworks and deployed on AWS. That's more than is more 10x faster than the 1990s CGI days of having C Language code writing to stdout to output HTML. Those early web apps were simpler and yet they were so utterly tedious and slow to code.

paulryanrogers 6y ago

Total complexity must then exclude the layers of hardware, cloud infrastructure, and FOSS contributions that involved many other people.