11 ms·

Web development is in truly a sad state. Having worked on a larger Node project I'll never do it again. The only thing you can do is npm shrinkwrap and never

• bahmutov 11y ago
  Well, consider that you have not paid anything for these packages, can you complain that authors do not follow the semver? If you want to protect yourself and upgrade reliably - there are tools, like my https://github.com/bahmutov/next-update https://github.com/bahmutov/next-update - runs your tests and only keeps upgrades that don't break stuff.
  • dham 11y ago
    Nope, you are a 100% correct. That's why I made sure I praised/thanked the author/contributors for their work on the project and suggested to them how fix the issue. Thanks for the library.
  • tracker1 11y ago
    Seems like the author's coworkers didn't follow semver, a feature/method was added in a minor release, they should have bumped the minor/minimum when bumping... locking to x.x.* is the new npm default iirc.
• phpnode 11y ago
  > I'm not sure it's possible to move a project using Babel 5, async/await, generators and react to upgrade to Babel 6. I've spent hours on it. Well, you're actually right. Babel 6 has had a lot of problems, particularly with async functions, but they are slowly being worked out. I think it's pretty clear at this point that it was released too early, and they really need smoke tests to prevent this kind of problem in future. If you've already built a significant project using Babel 5, definitely consider holding off the upgrade for another month or two.
  • Swizec 11y ago
    The funniest problem I ran into is that in Babel 6 using `typeof` does not work. Had to rewrite all our code to use Lodash type helpers because it is an external dependency and doesn't go through our build stack. We need our shit to work in production and don't have time to wait for forever. Bug hasn't been fixed in almost a month: https://phabricator.babeljs.io/T6644 https://phabricator.babeljs.io/T6644 You'd think something like a basic language feature wouldn't break when you update your compiler. I did. I was wrong. Thank the gods of software gardening that we were only using typeof in five or so places.
    • wdmeldon 11y ago
      It's kinda funny actually. I remember the oft quoted maxim regarding very mature static languages: "It's never the compilers fault". This was obviously a hyperbole, but it held true most of the time. These days, the definition of "compiler" has gotten extremely loose on the web side, and it seems like correctness has gone by the wayside.
      • nostrademons 11y ago
        That's always been the case with fast-growing technologies. I remember finding a disturbing number of bugs in Java 1.3 c. 2000; if you ask Swift programmers today about the stability of the compiler, they could tell you all about compiler crashes & bugs. If you want stability, use a technology that was popular 10 years ago. If you want to work on the bleeding edge, you will be bleeding. Decide what's appropriate for your project. The reason people worked in Java in 2000 or Swift/Babel today is so they can take advantage of temporary market opportunities where other people get deterred by the pain of capitalizing on them. This can be very financially lucrative if you yourself are willing to put up with that pain.
• iends 11y ago
  I too have switched from Node.js to Go in my own projects. However, Node.js (and Angular 1.x) still pay the bills. I tried to convince my previous company to write some of their microservices in Go but it was a hard sell. In fact, I introduced a minor piece of go code that would only run once and politics exploded. I was asked to rewrite it in Node.js. Obviously, I'm not longer with that company, but still writing Node in the day job. As you mention, packages have minor updates with breaking changes, so if you don't shrinkwrap things explode. There is a huge number of JS packages that do a lot of interesting things but the average quality leaves a lot to be desired. I've found people get something working very quickly with Node.js but then after a year...or three it's very hard to maintain the Node.js system. Especially when the original author is no longer on the team. This is the problem TypeScript tries to solve, and TypeScript is really nice in my opinion...but its 2015 and my computer has four cores and 4 virtual cores. Languages that do not enable developers to easily take advantage of these cores are not interesting and not something I want to invest too much time in for fear they are out of favor in the next few years when cores increase even more... Go has this solved, but people don't quite trust Google yet. For one, they seem to have a history abandoning things (or completely rewriting from scratch them like Angular 2). Go is missing a few needed features that are preventing more wide spread adoption (things like better package management), but these features are not things that the core team is prioritizing at this time.
  • politician 11y ago
    > This is the problem TypeScript tries to solve. If your development methodology collapses when an original developer leaves, then the problem was that they were carrying tech debt by force of will rather than building a robust architecture. This isn't something TypeScript can solve. > Languages that do not enable developers to easily take advantage of these cores... Single-threaded runtimes, like Node, aren't intrinsically problematic, and multi-threaded solutions aren't a panacea. If you have multiple cores and a single-threaded application that you'd like to scale, then assuming the other resources are plentiful, just spawn multiple instances and pin them to those extra cores. > Go ... I like Go, and its goroutines, but working with them still requires an understanding of locking and concurrency to get it right. If the plan is to throw a random front-end developer at Go and expect magic, I think that's wildly optimistic.
• Scarbutt 11y ago
  I don't think most people run into this because most backend node programmers have moved to Go at this point, so they're not aware of the upgrade problems. How does Go solve this?
  • phillmv 11y ago
    Last time I checked Go didn't really have a package manager, so I think the joke here is that most web apps are semi disposable and people haven't hit the upgrade cycles for their Go apps either.
  • dham 11y ago
    It was just a joke. Just saying that most of the Node developers have probably long since moved to Go.
  • lobster_johnson 11y ago
    Go's static typing goes a long way in helping catch breakage due to things like adding or remove symbols. (However, it does have some warts where incompatible versions may silently compile, like initializers.) While Go doesn't have package management, it does have a framework (go get) for automatically fetching and compiling dependencies. However, people quickly realized that there's no way to pin dependencies; it will always fetch the master branch's HEAD, so if you "go get" a project, it will fetch some arbitary version depending what time of day it is. The community has devised various ways around it, and recent versions of Go have a way of resolving packages to a local "vendor" directory, but it's not a solved problem. There's no solution that stands out from the rest as particularly good or feature-complete when you look at the historical alternatives (Ruby's Bundler in particular).
    • paulddraper 11y ago
      Always getting HEAD seems like a rookie mistake that rejects decades of lessons from mature systems and is a real deal breaker. But that may just be me. I think the same thing about generics, and others clearly don't.
• matt2000 11y ago
  I could not agree more. The rapid versioning and huge quantity of dependencies in Node and javascript development in general is just a recipe for disaster. This will become more obvious as large legacy node projects start transitioning to new teams and the true scope of the downsides will become obvious. However, I'm not sure it will be obvious here because the focus is almost exclusively on green field new development with the latest tools. The legacy of node will be nothing but sadness. (Sort of a joke, but also kind of true)
  • nostrademons 11y ago
    Isn't that the legacy of every programming system in existence? I've never known a programmer to take over maintenance of another programmer's code and say "Wow, this is a joy to work with." You always end up with a big ball of mud that grew really rapidly under the pressures of customer demands (because the ones that don't grow really rapidly and respond to customers don't survive to attract maintenance programmers), that you then have to untangle and understand. It seems like you have two choices if you'd like to work in the software industry: you can put up with shitty code and boring maintenance work in exchange for a steady, risk-free paycheck, or you can take on the risk that your code will fail outright and the certainty that it will be grumbled about and ultimately replaced in exchange for the intellectual thrill and potential financial rewards of working on a green-field project.
    • ry_ry 11y ago
      Every language has it, but because JavaScript is evolving so rapidly the problem is amplified. We've seen more change in the last couple of years than some more mature languages have seen in the last decade - js isn't finished, and I don't think it ever should be. On the plus side, it keeps us employed as long as we are prepared to keep learning the new stuff, or earn a spectacular exit.
      • tracker1 11y ago
        Even without JS there's been a lot of changes over the years... in ASP.Net from 1.0 was a big mess, by 3.x it was nice, then came MVC, paradigm shift... that coupled with additions to C# and VS. There's been similar changes to Java, Python and others over the years too. Let alone Ruby which would never have gained ground if it weren't for Rails. I will say for a 20 year old language, JavaScript has grown more in the past 6 years than it's first 14, and a lot of that is thanks to Node/npm.
• pjmlp 11y ago
  Which is why I never cared about Node and rather work with native UIs instead of web ones when given the option.
• sotojuan 11y ago
  It doesn't have to be this way. At least stuff like Elm forces semver on your projects. It detects if your new version has breaking changes and won't publish it unless you bump it up a version. That said, Elm is super niche and will probably never be popular. But my point is, there's people doing it right. I too am confused about Babel 6... but to be fair I haven't looked too much into it. I just think it's weird that something as "simple" as doing ES2015 -> ES5 now has plugins and a whole ecosystem around it. I understand the creator's ambitions, but I'd prefer to just install it and plug it to my build system. I don't know much about Go, but I am personally looking into Elixir as another backend choice.
  • tracker1 11y ago
    It's not just ES6/7 transpilation though... there are a lot of ES6 features that are in node, and modern browsers, so not everything needs to be translated all the time... let alone more advanced options like flow and jsx (more than just for react, but mercury and others). That's why the change, I do think they pulled the gun a bit soon though, but many would have jumped to it just the same.
• deleted 11y ago
  [deleted]
• codeshaman 11y ago
  Yes. I loved node when it came out. I did a lot of experimental projects with it. I also liked npm and the ability to use modules... But at some point, this all turned into dependency insanity. It didn't bite me as much, because i've never used node for production apps, but I imagine it has the potential to be a serious issue, when your bread-and-butter app depends on 694 other libraries.. who knows what lurks in them... What did bite me was the apparent "infinity" of possibilities at each step of the development process. Whenever I need a feature - there's 20 different libs there to help you. Which one should I pick ?
  • krisdol 11y ago
    >It didn't bite me as much, because i've never used node for production apps, but I imagine it has the potential to be a serious issue, when your bread-and-butter app depends on 694 other libraries.. who knows what lurks in them... The problem is this approach that somehow finds a monolith with 694 dependencies acceptable, yet turns around and praises node.js modules for being so "bite-sized". >What did bite me was the apparent "infinity" of possibilities at each step of the development process. Whenever I need a feature - there's 20 different libs there to help you. Which one should I pick ? This kind of choice paralysis used to bite me too. What totally fixed it for me was really weighing the consequences of adding a dependency every time I'd consider one. Most of my production services are a couple hundred lines and have fewer than 5 dependencies (of which, half are boilerplate that every other service has). Rails is great at big CRUD MVC applications. If you want a Rails-like app in Node.js, you should start with an opinion "skeleton" MEAN/MVC framework rather than re-implementing rails and all its dependencies with every application.
• bkurtz13 11y ago
  > I don't think most people run into this because most backend node programmers have moved to Go at this point Comments like this are what make it difficult for me to take HN seriously anymore. It's just an example of the hivemind at work, assume that an entire industry has the same ideas as you about what technology to use. The Node/JS ecosystems are experiencing a Cambrian explosion in technology, with all that implies. If you don't like it, don't worry. It probably won't take 20 million years this time to settle in on the "best" ways of doing things.
  • deleted 11y ago
    [deleted]
  • n0w 11y ago
    On the contrary, I got a kick out of the quoted line because it was exactly what I did.
    • deleted 11y ago
      [deleted]
    • poorman 11y ago
      Same here.
• rpedela 11y ago
  Cant speak for Babel, but the rest of it does not mirror my experience. I have had either no trouble or minor trouble upgrading Node versions. I always hard-code versions for dependencies, and then periodically upgrade which usually takes a few minutes. Once in awhile an API changes, and I have to spend a couple hours (gasp!). You make it sound like upgrading in Node is completely broken, but that doesnt appear to be the case in my experience.
  • cozuya 11y ago
    I use Node on both OSX and Windows and its definitely broken in Windows or maybe just not a priority to work well. Getting a full stack modern application working on localhost on a Windows machine is really painful and typical requires hours of googling obscure npm i errors.
    • rpedela 11y ago
      I bet. Even though Node claims to work on Windows, most dependencies are built and tested for Linux (or OSX). Just curious, what are the use cases for Node on Windows?
      • alxeder 11y ago
        I think the main use case is developer workstations which run on Windows because of corporate guidlines
        • cozuya 11y ago
          I'm probably the rare developer who prefers the UI and feel of Windows to OSX.. though I am probably a prototypical use case to switch to Mint, but haven't got around to it.
          • krisdol 11y ago
            I feel comfortable with all the UIs, but essentially I want to be as close to my application's production environment as possible, while still being productive. I've never worked on a team where that environment was windows, and setting up node/ruby/docker/vim/bash/nginx in a windows environment has always been a total pain compared to either linux or OS X (still kind of a pain for docker, but slightly less so). For some of these core tools, authors explicitly state that windows is a second class citizen. It's just not worth the expense to support something that has so few comparative users, in some cases (ruby). It's a lot of work for something that will never feel "native".
• tolmasky 11y ago
  There are a number of really good (separate) points here: 1. Shrink-wrapping: yes, of course. Semver ranges never made sense to me. All changes are breaking changes. The idea that we could safely auto-upgrade a project is insane. A semver range is an open invitation to allow people not on your team to arbitrarily change your code at any time in the future. If it was up to me, npm install -save would just shrink-wrap. The idea behind semver ranges is fundamentally flawed since it relies on a human making a determination of whether the change is patch or major or what have you. Even if you automate it and tie it to tests you can't know for sure: Even in a magical world where a "patch" upgrade could be guaranteed to not actually break behavior, it could still have dire perf consequences (which in an async world could absolutely change the behavior of your running program by surfacing new race conditions or what have you). That's why this is the behavior on tonicdev.com: when you require a package we magically shrink-wrap it at that point in time. Reproducibility should be the default. All changes are breaking changes. 2. The quickness of change is definitely also an issue. This may just be a price of progress thing though. I certainly wouldn't want to give up amazing rewrite the world changes like the development of React for a "stable" environment. I can also see the flip side to this though.
  • cjhveal 11y ago
    For what it's worth, w.r.t. your first point you can get part of the way there by creating a .npmrc in your project with "save-exact=true". Even if it doesn't protect you from higher order dependencies that violate semver, it will peg all saved dependencies to an exact version (even by other engineers). But alas, even that sometimes doesn't give you any guarantees with the (albeit rare) force publish. Better than nothing, though.
• tjholowaychuk 11y ago
  Yep haha, it's easy to be blind to all this if you haven't switched to something else. I knew it was bad but I didn't realize how much better platforms like Go were until really making the move. Too many people have grown used to transpiling etc, IMO it's not acceptable that we transpile pretty much everything server and client for usability.
  • Touche 11y ago
    I think this is a temporary problem. What is now called ES2015 (or ES6) was largely part of ES4 that got abandoned. Once most of ES2015 is in browsers and current versions of Node, especially modules, I think the need to transpile will be much less. Eventually JavaScript will syntactically be close to "done" and we won't need transpilers any more.
    • tjholowaychuk 11y ago
      Agreed, it'll get there eventually. It's just unfortunate that this critical stuff wasn't done earlier, could have been huge for the community. Things like async/await should be top priority IMO. I'd almost rather just start from scratch so there's not a ton of weird legacy baggage.
  • fidget 11y ago
    A lot of the GP's concerns are around libraries breaking compatability, and upgrading deps being hard, but Go doesn't exactly have all the answers in that regard.
    • tjholowaychuk 11y ago
      Yeah certainly. I've found that in practice it's not a big problem with Go, you can easily vendor your app's deps, of which there are typically fewer (not 500mb+). Keeping them updated for security reasons is the only drawback of that approach really. I vastly prefer Go's technique compared to a central registry.
• lobster_johnson 11y ago
  I've also had a bunch of cases where old versions of packages (looking at you, esprima!) suddenly disappear from the official npmjs.com repository. We have apps that are stable and have been running for many months, until suddenly we need to add a small feature. With a missing package deep in the tree — not one we were depending on directly — we have to waste several hours being forced to upgrade packages to newer versions, just to work around the missing version. It's insane. (It's possible to unpublish NPM packages, which is what happened here. That in itself is terrible, especially since all you get is a 404 from NPM — no reason for the unpublish provided.) The NPM tool itself is a mess: * It suffers from sporadic bugs (the dreaded "npm ERR cb() never called") and isn't resilient against network issues (e.g., timeouts). * It can't use its own cache for packages that have compiled artifacts. * npm-shrinkwrap.json isn't stable. Adding a package tends to rewrite the whole file depending on whether it has resolved packages from the network or locally. * Lots of surprising disconnect between shrinkwrap and node_modules. For example, "npm i package" does nothing if it's in the shrinkwrap file, but "npm i package@latest" upgrades (but doesn't, obviously, modify the shrinkwrap file). * It uses a lot of files. We've had inode issues on some boxes because the sheer size of the NPM cache + deploys use so many files. Production deploys includes all sorts of artifacts (tests, documentation, readme, etc.) that you can't exclude. * The nested tree (fixed, I hear, in NPM 3.0) is really unmanagable. The microscopic, mutating nature of NPM packages is chaotic. There's no practical way to do upgrades in a controlled manner. (My wish list item: A command that prints out all upgradeable packages and a complete changelog for each version bump.) NPM could learn a lot from Ruby's Bundler.
  • funthree 11y ago
    https://github.com/tblobaum/git-rev https://github.com/tblobaum/git-rev
• sebcat 11y ago
  > Web development is in truly a sad state. Has it ever been better though? Technically, you could argue that webdev has never been as good as it is now. Not that I like the current situation, I just think that things in webdevland are better now than ever before.
• abritinthebay 11y ago
  Ladies and gentleman - the Dunning–Kruger effect in action.
• Keats 11y ago
  I agree with pretty much everything you said minus the Go one: if you are doing a CRUD app and you don't expect a huge traffic something like Python works well. Go shines for microservices though. I can't talk about Node on the server since I don't use that - already doing way too much javascript for the frontends - but the frontend side is hard to keep up with as well. Between all the Babel and Webpack plugins it's hard just to get a working setup and the `do one thing and do it well` should not be a bigger thing than user experience. TypeScript makes writing JavaScript less painful but doesn't solve the ecosystem for me: I do rm -rf node_modules/ && npm install semi regularly to make sure everything works (not using shrinkwrap yet) on a fresh install and I get install errors way too often (peer dependencies etc). To save you time, don't bother with npm3 for now as it is extremely slow.
• chippy 11y ago
  Job security through dependency insecurity.
• jessaustin 11y ago
  You might want more tests?
• eecks 11y ago
  Why use node on a large project instead of something like java/spring? Serious question btw
• bootload 11y ago
  "The only thing you can do is npm shrinkwrap and never update your dependencies ever again. If you update you are f'ed. It's a nightmare." Node as npm, Perl has CPAN. Why are modules hard in Node, when modules in Perl work?
• 20years 11y ago
  "On the front end, people are probably re writing the front end every 3 or 4 months. Moving from javascript framework to javascript framework." That seems crazy! Maybe I am out of touch but how is that even viable or smart? Seems like a time & money suck both of which can be spent on so many other valuable areas. Does the consumer or whoever the site services really care about what front-end framework is being used? No, they just want a decent user experience so why not focus on that rather than the latest shiny framework. I just don't get it. I feel like some devs have lost site on what's really important with developing software. It is not about the latest shiny thing. It is about the target customer.
• dalore 11y ago
  To paraphrase: "If you don't freeze your dependencies, you're going to have a bad time". This is the case in any environment. This is the case for your OS, for python requirements, for java jars, everything. Your versions should be part of the source control (not necessarily the dependencies in your source, but at least a project file that has the versions listed). Python has a requirements.txt. It's best practice to pin your versions in it. I've been trying out jspm recently. It's quite nice and teaches you this out of the box. You pull down a npm package, it will put the version into your package.json so everyone can use the same version.