5 ms·

Something that's been concerning me about the current approach to web development in particular is the accumulation of unacknowledged -- and increasingly unmeas

Something that's been concerning me about the current approach to web development in particular is the accumulation of unacknowledged -- and increasingly unmeasurable -- technical debt. I'm not sure that's precisely the right phrase, but here's what I mean: we all (should) know about technical debt in our own projects, but we also know that every project accumulates technical debt. When we build our projects on top of other projects, our work now has the potential to be affected by the technical debt -- bugs, poor optimizations, whatever -- in the underlying projects. Obviously, this has always been true to varying degrees. But we've reached a point where modern web applications are pulling in dozens of dependencies in both production and development toolchains, and increasingly those dependencies are themselves built on top of other dependencies. I don't want to suggest we should be writing everything from scratch and constantly re-inventing wheels, but when even modest applications end up with hundreds of subdirectories in their local "node_modules" directory, it's hard not to wonder whether we're making things a little...fragile, even taking into account that many of those modules are related and form part of the same pseudo-modular project. Is this a completely ridiculous concern? (The answer may well be "yes," but I'd like to see a good argument why.)

Silhouette 11y ago
I think if anything you're being too kind here. The way dependencies are typically handled in the Node/NPM ecosystem is horrendous, and this has serious negative implications for the stability and longevity of almost any project built within that ecosystem today. Unfortunately, modern web development is led, at least in publicity terms, by people who think phrases like "move fast and break things" and "living standard" are cool. The trouble is, what actually happens if you follow those people is that you break lots of things very quickly and you have no stable standards the help everyone do better.
- r2dnb 11y ago
  >Move fast and break things You made my day, I never heard of that one... unbelievable !
  - andrewingram 11y ago
    It was Facebook's motto for a long time.
- swk2015 11y ago
  The Node / NPM system is fantastic. Even my design colleagues use it on a daily basis, and for them, 'npm install' just works. Interesting that Microsoft have adopted a very NPM style with their new .NET Core project.json files.
Touche 11y ago
More simply put, stability is not valued in the front-end world. It's interesting to compare the dominance of Microsoft in the enterprise world where IT runs the show was established largely due to stability. IT is run by managers who can get fired if instability costs the company money. In the front-end world managers can't keep up with the changing landscape and therefore cannot make stability based decisions (in terms of software stability). Instead, in my experience, they make decisions based on how easy it is to hire (as cheap as possible) developers. So, as a proxy, popularity drives the front-end because developers who have the time to keep up with the latest trends (IE mostly young 20-somethings) because they're the ones who get hired.
- ethbro 11y ago
  I think that's an interesting observation about Microsoft re: the period where they had an enterprise OS (NT) and a consumer OS (3.1x/9x). I think those two tensions and philosophies came through clearly.
- soft_dev_person 11y ago
  It's a bit ironic to praise Microsoft for stability in this context, when they probably have had a major role of causing instability in the web front-end world for more than a decade.
  - chc 11y ago
    I thought most people's complaint was that Microsoft caused stagnation, not instability. Explorer was too stable for most developers' tastes.
    - d215 11y ago
      Most people's complaint was Microsoft was trying to push their own proprietary standards (Activex anyone?) upon the www at large. Also their adherence to Web standards at the time IE6 came out was so bad, that it caused almost unsurmountable instability.
gotchange 11y ago
How's web development any different from other "branches" of software development when it comes to technical debt or deep dependency trees?! It's been the norm in the field of software development since the early days to rely on other people's work for productivity reasons and many other objectives and consequently the trade-off of convenience with technical debt whether in implicit or explicit terms. I don't think that web development is more guilty of this "sin" than any other branch of the field.
- chipotle_coyote 11y ago
  You might be right. From exploring other open source applications my impression is that there tends to be many times more external libraries, tools and plugins pulled in by modern web applications when compared to, say, a Cocoa/Objective-C application of moderate size, though.
- colllectorof 11y ago
  The difference is that we've seen that the other kind of web development (without grapevines of libraries and complex toolchains) is both possible and capable of producing usable results. We're regressing.
hassy 11y ago
> But we've reached a point where modern web applications are pulling in dozens of dependencies in both production and development toolchains, and increasingly those dependencies are themselves built on top of other dependencies. Um, take ANY non-trivial application, written in any language, and then follow the chain of its dependencies all the way down to the kernel, and shriek in horror. JS is not unique here, the only unique part is perhaps how obvious npm makes it just how much stuff your code depends on.
- RaleyField 11y ago
  > all the way down to the kernel There was a bug that allowed you to go to SMM. The fun doesn't stop with the kernel.
- iheartmemcache 11y ago
  Sort of kind of. I deal as a consultant for large corporations where down-time is a no-no. Think secondary market-makers, institutional investors, healthcare insurers. Not quite rocket control systems or firmware for dialysis machines but five-nines is a given. Bugs happen everywhere. That's an axiom. You can decrease the risk of it with formally verifying the system (see: SPARK for Ada), but they'll occur. The difference is the what in response to those bugs. ASP.NET on Oracle 11g might cost a few hundred thousand a year in support contracts alone, but when the bugs occur there is an insurance policy (literally-- at that level, there's an SLA and everything), professional liability insurance, and legal recourse. Those ecosystems have had 1: time to mature so those previous bugs that customers have identified and reported have been resolved, and 2: the option to pay (a lot of money, I'll admit) to have any new bugs resolved almost immediately. I was working with a large healthcare provider and some how a bug made it through 3 rounds of QA into production. Within 6 hours 3 engineers who worked on that specific component had flown across the country, landed, got in my car and was driven to our office, while another team was simultaneously Remote Desktop'ing to see if they could resolve it in the process. Microsoft still offers LTS for a 13 year old operating system. I'm not ragging on node.js - its endemic of any new technology with a lot of fervor. In fact, it very much reminds me of the CL community in the late 1990s where everyone was writing a ton of solutions, submitting their packages which worked very well for their own purposes, but broke the second you tried to do something it wasn't meant to. It would break and you'd be on a mailing list back-and-forth'ing for two or three weeks. A large amount of code is in use which hasn't had to stand the test of time (which flushes out a) the poorly designed solutions and b) the bugs in "good" solutions). To make matters worse, and no one knows which is going to be around in 5 years (i.e. what is "good"?) so it's all a gamble where no one quite what to put their money on (should I pick Knockout? Angular? oh wait now it's all about isomorphic code..should I use Redux or Flux?).
lr4444lr 11y ago
I think you make some good points, as do a number of your repliers. We're pulled in different directions by a number of business and technical priorities. What I see as a common denominator is a development life-cycle that expects increasingly nimble articulation of break-off projects and shorter iteration times into production, at least in SaaS. Concerns about code stability and optimization are not entirely lost by any means, but they don't seem to have the same priority when the business "has to" keep moving, and the tech "has to" keep evolving at this pace.
skewart 11y ago
As others have pointed out, the number of lines of code that the typical 2016 webapp depends on isn't all that different from other ecosystems. The dependency footprint just looks a lot bigger because its broken up into lots and lots of little modules. I'd argue this actually makes things a little less fragile, because - at least theoretically - its easier to manage versions and do rollbacks of the different various pieces if something goes wrong. You can surgically switch the version of a dependency without necessarily changing a lot of other functionality. Of course, things aren't always that simple. And lots of little modules might effectively mean less integration testing and higher chances of bugs. So, maybe it's a toss-up.
Outdoorsman 11y ago
Your point is one well worth making..."technical debt" is not an inept choice of terms in IMHO... I've been in the trade for decades now...I'm more inclined to apply the term "technical baggage" to the feeling I often have these days when sitting down to solve a problem for a client... Layers upon layers of dependencies to sort to do a proper job of putting systems back on sound footing...multiple patches to be examined, some to be discarded...with the clock ticking...non-tech managers love to hear you say, "OK, you're back up"...increasingly, that's what we settle for... All to keep the wheels of commerce turning with as little down time as possible... I have to admit that my discomfort is at least somewhat influenced by comparing the current environment to the environment of the early days...essentially stand-alone programs, elegantly written, performing the same basic function that now require five, ten... Forgive the reminiscence...times are different...

Silhouette 11y ago

I think if anything you're being too kind here. The way dependencies are typically handled in the Node/NPM ecosystem is horrendous, and this has serious negative implications for the stability and longevity of almost any project built within that ecosystem today. Unfortunately, modern web development is led, at least in publicity terms, by people who think phrases like "move fast and break things" and "living standard" are cool. The trouble is, what actually happens if you follow those people is that you break lots of things very quickly and you have no stable standards the help everyone do better.

r2dnb 11y ago
>Move fast and break things You made my day, I never heard of that one... unbelievable !
- andrewingram 11y ago
  It was Facebook's motto for a long time.
swk2015 11y ago
The Node / NPM system is fantastic. Even my design colleagues use it on a daily basis, and for them, 'npm install' just works. Interesting that Microsoft have adopted a very NPM style with their new .NET Core project.json files.

Touche 11y ago

More simply put, stability is not valued in the front-end world. It's interesting to compare the dominance of Microsoft in the enterprise world where IT runs the show was established largely due to stability. IT is run by managers who can get fired if instability costs the company money. In the front-end world managers can't keep up with the changing landscape and therefore cannot make stability based decisions (in terms of software stability). Instead, in my experience, they make decisions based on how easy it is to hire (as cheap as possible) developers. So, as a proxy, popularity drives the front-end because developers who have the time to keep up with the latest trends (IE mostly young 20-somethings) because they're the ones who get hired.

ethbro 11y ago
I think that's an interesting observation about Microsoft re: the period where they had an enterprise OS (NT) and a consumer OS (3.1x/9x). I think those two tensions and philosophies came through clearly.
soft_dev_person 11y ago
It's a bit ironic to praise Microsoft for stability in this context, when they probably have had a major role of causing instability in the web front-end world for more than a decade.
- chc 11y ago
  I thought most people's complaint was that Microsoft caused stagnation, not instability. Explorer was too stable for most developers' tastes.
  - d215 11y ago
    Most people's complaint was Microsoft was trying to push their own proprietary standards (Activex anyone?) upon the www at large. Also their adherence to Web standards at the time IE6 came out was so bad, that it caused almost unsurmountable instability.

gotchange 11y ago

How's web development any different from other "branches" of software development when it comes to technical debt or deep dependency trees?! It's been the norm in the field of software development since the early days to rely on other people's work for productivity reasons and many other objectives and consequently the trade-off of convenience with technical debt whether in implicit or explicit terms. I don't think that web development is more guilty of this "sin" than any other branch of the field.

chipotle_coyote 11y ago
You might be right. From exploring other open source applications my impression is that there tends to be many times more external libraries, tools and plugins pulled in by modern web applications when compared to, say, a Cocoa/Objective-C application of moderate size, though.
colllectorof 11y ago
The difference is that we've seen that the other kind of web development (without grapevines of libraries and complex toolchains) is both possible and capable of producing usable results. We're regressing.

hassy 11y ago

> But we've reached a point where modern web applications are pulling in dozens of dependencies in both production and development toolchains, and increasingly those dependencies are themselves built on top of other dependencies. Um, take ANY non-trivial application, written in any language, and then follow the chain of its dependencies all the way down to the kernel, and shriek in horror. JS is not unique here, the only unique part is perhaps how obvious npm makes it just how much stuff your code depends on.

RaleyField 11y ago
> all the way down to the kernel There was a bug that allowed you to go to SMM. The fun doesn't stop with the kernel.
iheartmemcache 11y ago
Sort of kind of. I deal as a consultant for large corporations where down-time is a no-no. Think secondary market-makers, institutional investors, healthcare insurers. Not quite rocket control systems or firmware for dialysis machines but five-nines is a given. Bugs happen everywhere. That's an axiom. You can decrease the risk of it with formally verifying the system (see: SPARK for Ada), but they'll occur. The difference is the what in response to those bugs. ASP.NET on Oracle 11g might cost a few hundred thousand a year in support contracts alone, but when the bugs occur there is an insurance policy (literally-- at that level, there's an SLA and everything), professional liability insurance, and legal recourse. Those ecosystems have had 1: time to mature so those previous bugs that customers have identified and reported have been resolved, and 2: the option to pay (a lot of money, I'll admit) to have any new bugs resolved almost immediately. I was working with a large healthcare provider and some how a bug made it through 3 rounds of QA into production. Within 6 hours 3 engineers who worked on that specific component had flown across the country, landed, got in my car and was driven to our office, while another team was simultaneously Remote Desktop'ing to see if they could resolve it in the process. Microsoft still offers LTS for a 13 year old operating system. I'm not ragging on node.js - its endemic of any new technology with a lot of fervor. In fact, it very much reminds me of the CL community in the late 1990s where everyone was writing a ton of solutions, submitting their packages which worked very well for their own purposes, but broke the second you tried to do something it wasn't meant to. It would break and you'd be on a mailing list back-and-forth'ing for two or three weeks. A large amount of code is in use which hasn't had to stand the test of time (which flushes out a) the poorly designed solutions and b) the bugs in "good" solutions). To make matters worse, and no one knows which is going to be around in 5 years (i.e. what is "good"?) so it's all a gamble where no one quite what to put their money on (should I pick Knockout? Angular? oh wait now it's all about isomorphic code..should I use Redux or Flux?).

lr4444lr 11y ago

I think you make some good points, as do a number of your repliers. We're pulled in different directions by a number of business and technical priorities. What I see as a common denominator is a development life-cycle that expects increasingly nimble articulation of break-off projects and shorter iteration times into production, at least in SaaS. Concerns about code stability and optimization are not entirely lost by any means, but they don't seem to have the same priority when the business "has to" keep moving, and the tech "has to" keep evolving at this pace.

skewart 11y ago

As others have pointed out, the number of lines of code that the typical 2016 webapp depends on isn't all that different from other ecosystems. The dependency footprint just looks a lot bigger because its broken up into lots and lots of little modules. I'd argue this actually makes things a little less fragile, because - at least theoretically - its easier to manage versions and do rollbacks of the different various pieces if something goes wrong. You can surgically switch the version of a dependency without necessarily changing a lot of other functionality. Of course, things aren't always that simple. And lots of little modules might effectively mean less integration testing and higher chances of bugs. So, maybe it's a toss-up.

Outdoorsman 11y ago

Your point is one well worth making..."technical debt" is not an inept choice of terms in IMHO... I've been in the trade for decades now...I'm more inclined to apply the term "technical baggage" to the feeling I often have these days when sitting down to solve a problem for a client... Layers upon layers of dependencies to sort to do a proper job of putting systems back on sound footing...multiple patches to be examined, some to be discarded...with the clock ticking...non-tech managers love to hear you say, "OK, you're back up"...increasingly, that's what we settle for... All to keep the wheels of commerce turning with as little down time as possible... I have to admit that my discomfort is at least somewhat influenced by comparing the current environment to the environment of the early days...essentially stand-alone programs, elegantly written, performing the same basic function that now require five, ten... Forgive the reminiscence...times are different...