6 ms·

Software takes a long time to build because it's always new. That is, software is trivially copy-able, so there is no reason to spend effort duplicating any so

Software takes a long time to build because it's always new. That is, software is trivially copy-able, so there is no reason to spend effort duplicating any software that already exists. (Legal reasons and "not-invented-here" syndrome notwithstanding.) This is a huge different from how the "real" world works, where almost all of the work involved in building, say, a car, is actually just the work of assembling identical copies of that product so that it can be physically sold to more than one person. The insane amounts of repeated effort involved in building any physical product at scale have a silver lining, in that the repetition enables a very good and stable estimate of how long another repetition will take. Software is the opposite situation --- we only need to build anything once, but not having built that thing before, we don't really know how long it will take.

TuringTest 6y ago
If that were true, it wouldn't take long to build from scratch a functionally identical clone of an existing application. Yet it does. The conclusion is that the tools to build software from a known specification to working code are sub-optimal.
- xmprt 6y ago
  That's like saying a Ford F150 is functionally identical to a Ford Model A because they are both cars. The specific implementation under the hood requires a lot of work. And very rarely do people ever make functionally identical clones without massive refactoring to allow for extensibility.
  - TuringTest 6y ago
    "implementation under the hood requires a lot of work" "people don't make functionally identical clones" "massive refactoring to allow for extensibility" You're enumerating the factors that make my point true. Those are some of the reasons why building software takes a lot even if you know exactly what it is supposed to do. You can compound that to the problem of figuring out what the software should do, which classic IDEs are also bad at. Online collaboration and prototyping tools that put the program in front of stakeholders early; they're like car makers' focus groups that were used to design cars tailored to a target demographic group.
  - lloeki 6y ago
    Building the first F150 is an act of creation and design. Assembling the second one is finding the same parts and doing the same assembly steps, the thinking is already sorted out. Building the first Model A means going back to the drawing board because it's not the same. Even at a smaller scope building a F150 with an updated exhaust means going back to the drawing board for that part, and making sure it keeps working correctly with all the other parts. It turns out that with software, once you have one fully assembled "F150", a second identical one can be created out of thin air (cp / git clone / run compiler / whatevs). So the effort is ~100% on thinking and design. Now why would you want 2..n F150s? Because you want to have them in the wild doing something useful. That's making them available and distributing them and having more of these as needs increase. That's cp'ing your code onto servers and deploying and running it and making it reachable. You do that with DigitalOcean, Chef, K8S, or whatever AWS service du jour, and hopefully you automate as much of it as possible just like we have car factories with robots to not build these with human hands. Deploying and spinning up more of that same code so that it keeps up with the scale of whatever it needs to do... that's ops, and that's basically the only part of software that remotely resembles industrial work. The part before that is development, as in "research & development", and that's 100% not "industrial" by any stretch of the imagination. Thinking that developing software is like building a house or a car and "industrialising" things with waterfall or Kanban on SCRUM and pulling bogus time estimates to completion is the biggest lie ever, just like pulling out the next vaccine or devising a new maths theorem cannot have any reliable time to completion because you're constantly up against unknown unknowns. This field just flat out doesn't work that way.
- toast0 6y ago
  > The conclusion is that the tools to build software from a known specification to working code are sub-optimal. That may be true, but I don't know that your example supports it. Cloning an app doesn't mean you had a specification for the app. The tools to build a known specification from an existing app are sub-optimal, as are the tools to build a known specification from scratch. I've only rarely in my career had the benefit of a written specification.
  - TuringTest 6y ago
    > The tools to build a known specification from an existing app are sub-optimal, as are the tools to build a known specification from scratch. That actually reinforces my point ;-) All the steps in the toolchain could benefit from more agile interactions, allowing the programmer to spend less time fiddling with syntax errors and recalling which functions need to be applied in what order, and more time evaluating and fixing errors in the current logic as written. That would expose what the build program is doing and how it differs from the expected intent. The online notebooks used in data analysis (Jupyter, Apache Zeppelin) are a step in the right direction; IMHO their approach of 'data is always readable besides the code processing it' makes for a better introspection infrastructure than the REPL loops of old.
    - mstipetic 6y ago
      Basically we're going back to what delphi had 30 years ago. I understand that scalability concerns warrant a different approach, but a lot of the software we write will not deal with google or netflix scale. I keep going back to the example of elixir/erlang (mostly because I'm getting into it now) but the tooling that is available in erlang is much more powerful than any newrelic or what ever monitoring/introspection tool we use now. And we've had it for years, but we've just ignored it.
- krisoft 6y ago
  I don’t want to sound smartass but building an identical clone of an existing application is very fast and often totally automated. That’s what your compiler chain does.
  - sriku 6y ago
    At the risk of sounding like another, it can be simpler than that - just install the already compiled software. In a sense, the thing about composing software using "microservices" is about "install it and get going".
  - TeMPOraL 6y ago
    Parent wrote "functionally identical", not "identical".
  - TuringTest 6y ago
    What does the compiler do if you start "from scratch", i.e., without any source code?
  - username90 6y ago
    Change the criteria that you should rewrite it in another framework and the point still stands. You already have a perfect specification in the existing code, no unknowns at all, just implement it. Yet it still takes months or years.
    - PeterisP 6y ago
      A good physical analogy is the historical Soviet recreation of USA B-29 Superfortress as Tu-4, based on copying all the parts (based on 4 captured aircraft), but needing to rework to the metric standards (i.e. so wherever you have, for example, 1/4 inch i.e. 6.35mm sheet metal or fasteners or whatever, you have to pick 6 or 7mm instead (because your "different framework" doesn't supply 1/4 inch stuff), which changes the weight distribution and structural integrity, which requires additional tweaks). You're making a functional copy, still took more a year just for the designs. The same with other physical objects - as long as the underlying framework or supply chain changes, it tends to require re-engineering.
- etripe 6y ago
  While your second sentence is definitely true, there are some subtleties that also matter. Car specs are (certainly as opposed to software specs) very exact and constrained by dimensions, chemistry and physics. None of those have version numbers. Creating a particular model of a car is decidedly single-paradigm. The same cannot be said for a functionally identical clone. You could make a functionally identical clone of an app in OOP or FP, nodejs, Rust or C++, MVC, MVVM or MVU, as a polyglot SPA + backend combination... All this to say: software as an industry also lacks standardisation, process documentation and documented best practices. Software "best practices" are more like "accepted dogma at the time" or "least likely to bite us later", not "will work 99.999% of the time". Software as a product is immaterial to start with.
sriku 6y ago
> Software takes a long time to build because it's always new. And yet it feels like authentication and authorization code need to be rewritten for every application.
- danielheath 6y ago
  I don’t rewrite bcrypt, and authorisation is intrinsically linked to your data model.
- Cthulhu_ 6y ago
  Not in my experience; basic session auth is easy enough, and oauth has been around for a while now. What keeps giving though is CRUD. Naming things, forms, APIs, persistence, every time, rinse and repeat.
- threatofrain 6y ago
  Only vendor-specific code, so it's not that bad. What business do most people have rewriting auth stuff?
- xorcist 6y ago
  That's because it is mostly integration that is done, not so much construction. AA needs to permeate the whole project, from the details of a function call to high level design like making sure an actor can call another part of the software stack to perform an otherwise unauthorized operation. "Building" software can mean any number of things. When you get down to the specifics, there is no getting around using specific language.
TeMPOraL 6y ago
I think focusing on this comparison to the real world assemblies and constructions doesn't yield much insight. The equivalent of a car being assembled is a compiler building an executable from source. You can consider factories to be just very, very long compiles, and forget about it. The question of why software takes a long time to build pertains to the coding phase, to which real life's analogue is drafting/design phase. It is my impression that the drafting/design phase in "hard" industries can be just as messy and hard to schedule as software development is - we just don't pay attention, because for physical products, drafting/design is nearly free (capex & opex of the assembly phase dominates), whereas in software, the construction is nearly free (compiler time is too cheap to meter). It's true that, on the surface, software is "always new". But after doing it for a while, I have this sinking feeling that a lot of it is old and repetitive work that we hadn't yet automated - there are deep similarities between pieces of code we're writing, but we don't have a language to fully capture them and abstract them away.
- BerislavLopac 6y ago
  > The equivalent of a car being assembled is a compiler building an executable from source I disagree. The equivalent of a car being assembled is copying the binary executable from one storage medium to another; building an executable from source would be the equivalent of building the production process for a single car model, while coding is the equivalent of designing and testing that model.
  - TeMPOraL 6y ago
    This is where the analogy breaks down somewhat - so I propose you imagine a universe in which `cp` doesn't exist and you have to build each copy of an executable from scratch :). (Perhaps you're signing each copy with a different key, or something.) I insist on connecting compilation to construction of individual cars, because the source code corresponds to the engineering plans of a car, and not to the plans of the production process.
    - pas 6y ago
      Sure, every car has a unique VIN and every running process is a different constellation of electrons (or just the one, if one-electron universe is assumed). But looking at getting functionality to users, looking at the business problem, the time/effort/risk is spent on car design and software development, and much less time and risk is in the copying business. (Yes, Tesla recently had car copying problems, but they brute forced their problem, and now they are pretty good at factory copying too.)
lazysheepherd 6y ago
Best comparison (car/machine manufacturing vs software development) I've ever heard on this topic is as following; While producing a machine, there are two steps; 1) designing the blueprint (takes a lot of time, unpredictable) 2) mass production (takes a lot of time, predictable) Those steps in software development; 1) building the software (takes a lot of time, unpredictable) 2) deploying the code (takes virtually no time, predictable) So the real difference is this: software is almost all about doing something new, each and every time. It's all flesh and has no bone: e.g. devoid of a long, very stable and predictable mass production stage. It's almost all about R&D, all of it's length. Hence 98% of the process is novel work therefore takes time and hard to predict.
fendy3002 6y ago
And add to that the number of custom software are many times more than most things in real life. Nobody will asks for their BMW to transport a cow, or for their BMW to connect / communicate with other's Mercedes. But we often asked for accounting app to able to handle employee's payroll and paid leaves. Then with that many demand for custom software, we also got many times software manufacturer than car manufacturer. Inexperienced manufacturer (programmer) will make development even slower. Not to mention that the client even don't know what they want.
- Izkata 6y ago
  Every once in a while when I see an example of an absurdity that wouldn't happen, I head off to Google to see what I can find. > Nobody will asks for their BMW to transport a cow How about a cow transporting a BMW? https://carnewschina.com/2013/02/22/bmw-owner-in-china-is-angry-hires-cow/ https://carnewschina.com/2013/02/22/bmw-owner-in-china-is-an...
Justsignedup 6y ago
To note. All engineering has this problem. Yes, building cars in a factory yields repetitive results, but what about all the different engineering projects. The truth is that this is not a new problem, we just lack discipline of fields that have been refined over thousands of years. Because of the quick idea to prototype rate, product research and refinement takes a back seat. All engineering projects have massive research before the first stone is placed. Nobody mid way through building a sky scraper decides "oh, do we need to redo the foundation?" I would also like to note, nobody is gonna give a self-taught architect who's 19 the ability to build that skyscraper foundation. Each building technique is developed, tested, and then once determined good, is actually used by others, and supervised by an industry veteran with underlings who learn and eventually themselves supervise such projects. Meanwhile in software it is the opposite. Not to say that there isn't a reason software grows so fast. Things change daily. If software worked like architecture we'd be licensing even the right to use ruby on rails-style modeling. But software works differently, thus faster iteration, thus more loosey-goosey, thus harder to estimate.
- pintxo 6y ago
  > I would also like to note, nobody is gonna give a self-taught architect who's 19 the ability to build that skyscraper foundation. Each building technique is developed, tested, and then once determined good, is actually used by others, and supervised by an industry veteran with underlings who learn and eventually themselves supervise such projects. Meanwhile in software it is the opposite. It's a bit less black and white: Any 6+ year old might build his own tree house, while in reality no one will let a self-taught kid write the control software for a nuclear power plant.
deleted 6y ago
[deleted]

TuringTest 6y ago

If that were true, it wouldn't take long to build from scratch a functionally identical clone of an existing application. Yet it does. The conclusion is that the tools to build software from a known specification to working code are sub-optimal.

xmprt 6y ago
That's like saying a Ford F150 is functionally identical to a Ford Model A because they are both cars. The specific implementation under the hood requires a lot of work. And very rarely do people ever make functionally identical clones without massive refactoring to allow for extensibility.
- TuringTest 6y ago
  "implementation under the hood requires a lot of work" "people don't make functionally identical clones" "massive refactoring to allow for extensibility" You're enumerating the factors that make my point true. Those are some of the reasons why building software takes a lot even if you know exactly what it is supposed to do. You can compound that to the problem of figuring out what the software should do, which classic IDEs are also bad at. Online collaboration and prototyping tools that put the program in front of stakeholders early; they're like car makers' focus groups that were used to design cars tailored to a target demographic group.
- lloeki 6y ago
  Building the first F150 is an act of creation and design. Assembling the second one is finding the same parts and doing the same assembly steps, the thinking is already sorted out. Building the first Model A means going back to the drawing board because it's not the same. Even at a smaller scope building a F150 with an updated exhaust means going back to the drawing board for that part, and making sure it keeps working correctly with all the other parts. It turns out that with software, once you have one fully assembled "F150", a second identical one can be created out of thin air (cp / git clone / run compiler / whatevs). So the effort is ~100% on thinking and design. Now why would you want 2..n F150s? Because you want to have them in the wild doing something useful. That's making them available and distributing them and having more of these as needs increase. That's cp'ing your code onto servers and deploying and running it and making it reachable. You do that with DigitalOcean, Chef, K8S, or whatever AWS service du jour, and hopefully you automate as much of it as possible just like we have car factories with robots to not build these with human hands. Deploying and spinning up more of that same code so that it keeps up with the scale of whatever it needs to do... that's ops, and that's basically the only part of software that remotely resembles industrial work. The part before that is development, as in "research & development", and that's 100% not "industrial" by any stretch of the imagination. Thinking that developing software is like building a house or a car and "industrialising" things with waterfall or Kanban on SCRUM and pulling bogus time estimates to completion is the biggest lie ever, just like pulling out the next vaccine or devising a new maths theorem cannot have any reliable time to completion because you're constantly up against unknown unknowns. This field just flat out doesn't work that way.
toast0 6y ago
> The conclusion is that the tools to build software from a known specification to working code are sub-optimal. That may be true, but I don't know that your example supports it. Cloning an app doesn't mean you had a specification for the app. The tools to build a known specification from an existing app are sub-optimal, as are the tools to build a known specification from scratch. I've only rarely in my career had the benefit of a written specification.
- TuringTest 6y ago
  > The tools to build a known specification from an existing app are sub-optimal, as are the tools to build a known specification from scratch. That actually reinforces my point ;-) All the steps in the toolchain could benefit from more agile interactions, allowing the programmer to spend less time fiddling with syntax errors and recalling which functions need to be applied in what order, and more time evaluating and fixing errors in the current logic as written. That would expose what the build program is doing and how it differs from the expected intent. The online notebooks used in data analysis (Jupyter, Apache Zeppelin) are a step in the right direction; IMHO their approach of 'data is always readable besides the code processing it' makes for a better introspection infrastructure than the REPL loops of old.
  - mstipetic 6y ago
    Basically we're going back to what delphi had 30 years ago. I understand that scalability concerns warrant a different approach, but a lot of the software we write will not deal with google or netflix scale. I keep going back to the example of elixir/erlang (mostly because I'm getting into it now) but the tooling that is available in erlang is much more powerful than any newrelic or what ever monitoring/introspection tool we use now. And we've had it for years, but we've just ignored it.
krisoft 6y ago
I don’t want to sound smartass but building an identical clone of an existing application is very fast and often totally automated. That’s what your compiler chain does.
- sriku 6y ago
  At the risk of sounding like another, it can be simpler than that - just install the already compiled software. In a sense, the thing about composing software using "microservices" is about "install it and get going".
- TeMPOraL 6y ago
  Parent wrote "functionally identical", not "identical".
- TuringTest 6y ago
  What does the compiler do if you start "from scratch", i.e., without any source code?
- username90 6y ago
  Change the criteria that you should rewrite it in another framework and the point still stands. You already have a perfect specification in the existing code, no unknowns at all, just implement it. Yet it still takes months or years.
  - PeterisP 6y ago
    A good physical analogy is the historical Soviet recreation of USA B-29 Superfortress as Tu-4, based on copying all the parts (based on 4 captured aircraft), but needing to rework to the metric standards (i.e. so wherever you have, for example, 1/4 inch i.e. 6.35mm sheet metal or fasteners or whatever, you have to pick 6 or 7mm instead (because your "different framework" doesn't supply 1/4 inch stuff), which changes the weight distribution and structural integrity, which requires additional tweaks). You're making a functional copy, still took more a year just for the designs. The same with other physical objects - as long as the underlying framework or supply chain changes, it tends to require re-engineering.
etripe 6y ago
While your second sentence is definitely true, there are some subtleties that also matter. Car specs are (certainly as opposed to software specs) very exact and constrained by dimensions, chemistry and physics. None of those have version numbers. Creating a particular model of a car is decidedly single-paradigm. The same cannot be said for a functionally identical clone. You could make a functionally identical clone of an app in OOP or FP, nodejs, Rust or C++, MVC, MVVM or MVU, as a polyglot SPA + backend combination... All this to say: software as an industry also lacks standardisation, process documentation and documented best practices. Software "best practices" are more like "accepted dogma at the time" or "least likely to bite us later", not "will work 99.999% of the time". Software as a product is immaterial to start with.

sriku 6y ago

> Software takes a long time to build because it's always new. And yet it feels like authentication and authorization code need to be rewritten for every application.

danielheath 6y ago
I don’t rewrite bcrypt, and authorisation is intrinsically linked to your data model.
Cthulhu_ 6y ago
Not in my experience; basic session auth is easy enough, and oauth has been around for a while now. What keeps giving though is CRUD. Naming things, forms, APIs, persistence, every time, rinse and repeat.
threatofrain 6y ago
Only vendor-specific code, so it's not that bad. What business do most people have rewriting auth stuff?
xorcist 6y ago
That's because it is mostly integration that is done, not so much construction. AA needs to permeate the whole project, from the details of a function call to high level design like making sure an actor can call another part of the software stack to perform an otherwise unauthorized operation. "Building" software can mean any number of things. When you get down to the specifics, there is no getting around using specific language.

TeMPOraL 6y ago

I think focusing on this comparison to the real world assemblies and constructions doesn't yield much insight. The equivalent of a car being assembled is a compiler building an executable from source. You can consider factories to be just very, very long compiles, and forget about it. The question of why software takes a long time to build pertains to the coding phase, to which real life's analogue is drafting/design phase. It is my impression that the drafting/design phase in "hard" industries can be just as messy and hard to schedule as software development is - we just don't pay attention, because for physical products, drafting/design is nearly free (capex & opex of the assembly phase dominates), whereas in software, the construction is nearly free (compiler time is too cheap to meter). It's true that, on the surface, software is "always new". But after doing it for a while, I have this sinking feeling that a lot of it is old and repetitive work that we hadn't yet automated - there are deep similarities between pieces of code we're writing, but we don't have a language to fully capture them and abstract them away.

BerislavLopac 6y ago
> The equivalent of a car being assembled is a compiler building an executable from source I disagree. The equivalent of a car being assembled is copying the binary executable from one storage medium to another; building an executable from source would be the equivalent of building the production process for a single car model, while coding is the equivalent of designing and testing that model.
- TeMPOraL 6y ago
  This is where the analogy breaks down somewhat - so I propose you imagine a universe in which `cp` doesn't exist and you have to build each copy of an executable from scratch :). (Perhaps you're signing each copy with a different key, or something.) I insist on connecting compilation to construction of individual cars, because the source code corresponds to the engineering plans of a car, and not to the plans of the production process.
  - pas 6y ago
    Sure, every car has a unique VIN and every running process is a different constellation of electrons (or just the one, if one-electron universe is assumed). But looking at getting functionality to users, looking at the business problem, the time/effort/risk is spent on car design and software development, and much less time and risk is in the copying business. (Yes, Tesla recently had car copying problems, but they brute forced their problem, and now they are pretty good at factory copying too.)

lazysheepherd 6y ago

Best comparison (car/machine manufacturing vs software development) I've ever heard on this topic is as following; While producing a machine, there are two steps; 1) designing the blueprint (takes a lot of time, unpredictable) 2) mass production (takes a lot of time, predictable) Those steps in software development; 1) building the software (takes a lot of time, unpredictable) 2) deploying the code (takes virtually no time, predictable) So the real difference is this: software is almost all about doing something new, each and every time. It's all flesh and has no bone: e.g. devoid of a long, very stable and predictable mass production stage. It's almost all about R&D, all of it's length. Hence 98% of the process is novel work therefore takes time and hard to predict.

fendy3002 6y ago

And add to that the number of custom software are many times more than most things in real life. Nobody will asks for their BMW to transport a cow, or for their BMW to connect / communicate with other's Mercedes. But we often asked for accounting app to able to handle employee's payroll and paid leaves. Then with that many demand for custom software, we also got many times software manufacturer than car manufacturer. Inexperienced manufacturer (programmer) will make development even slower. Not to mention that the client even don't know what they want.

Izkata 6y ago
Every once in a while when I see an example of an absurdity that wouldn't happen, I head off to Google to see what I can find. > Nobody will asks for their BMW to transport a cow How about a cow transporting a BMW? https://carnewschina.com/2013/02/22/bmw-owner-in-china-is-angry-hires-cow/ https://carnewschina.com/2013/02/22/bmw-owner-in-china-is-an...

Justsignedup 6y ago

To note. All engineering has this problem. Yes, building cars in a factory yields repetitive results, but what about all the different engineering projects. The truth is that this is not a new problem, we just lack discipline of fields that have been refined over thousands of years. Because of the quick idea to prototype rate, product research and refinement takes a back seat. All engineering projects have massive research before the first stone is placed. Nobody mid way through building a sky scraper decides "oh, do we need to redo the foundation?" I would also like to note, nobody is gonna give a self-taught architect who's 19 the ability to build that skyscraper foundation. Each building technique is developed, tested, and then once determined good, is actually used by others, and supervised by an industry veteran with underlings who learn and eventually themselves supervise such projects. Meanwhile in software it is the opposite. Not to say that there isn't a reason software grows so fast. Things change daily. If software worked like architecture we'd be licensing even the right to use ruby on rails-style modeling. But software works differently, thus faster iteration, thus more loosey-goosey, thus harder to estimate.

pintxo 6y ago
> I would also like to note, nobody is gonna give a self-taught architect who's 19 the ability to build that skyscraper foundation. Each building technique is developed, tested, and then once determined good, is actually used by others, and supervised by an industry veteran with underlings who learn and eventually themselves supervise such projects. Meanwhile in software it is the opposite. It's a bit less black and white: Any 6+ year old might build his own tree house, while in reality no one will let a self-taught kid write the control software for a nuclear power plant.

deleted 6y ago

[deleted]