4 ms·

I am not a JavaScript developer but was asked to help the JS team just the other week with an npm build issue. The npm modules built fine on OS X dev boxes, bu

by meshko 11y ago

I am not a JavaScript developer but was asked to help the JS team just the other week with an npm build issue. The npm modules built fine on OS X dev boxes, but CI server running Linux was failing because older version of gcc was installed. While troubleshooting this, I came to learn a bit about node.js and npm ecosystem and I surprised slightly more than I expected. Apparently, it is considered normal now to have build process download dozens of these npm modules. Many of them are build time dependencies. Things that are good old native binaries (e.g. for changing compression levels of graphics files) are downloaded by npm and built from source. Then when module runs, it tries to execute them by pulling in one of half a dozen possible modules for executing native binaries. There is no logging. All code is written in crazy continuation style, because the framework is inherently asynchronous -- nevermind that it only matters in a handful of hot IO cases requiring optimization -- all code is like that, undecipherable. There is no logging or error handling. A 3 months old project setup by smart consultants who do this kind of thing 24x7 already has a significant number of deprecated dependencies.

pachydermic 11y ago
It's maddening. >The real painful part of the process is caused by everything around React, including the build tools. Can't agree with that more. I really like the way react works. It just makes sense and doesn't get in your way very often. BUT it's incredibly frustrating dealing with npm dependency hell. Especially if you really are concerned about security and don't just trust that "someone else has probably made sure this code is safe, right?" I just want to use something for longer than 6 months :(
- meshko 11y ago
  Right. React itself appears somewhat reasonable, except I am not entirely sure that putting entire app state into a huge clusterfuck of json is going to scale, but I am trying to reserve my judgement until I try it myself. And shadow DOM... sigh. It's a cool hack, but it feels like surrender to me.
  - Silhouette 11y ago
    FWIW, as an experienced developer whose instincts are probably quite similar to yours, I do recommend keeping an open mind with tools like React. They're different to a lot of more traditional tools, but like optimisations and profiling, they're best judged on their real world results rather than against dogma. I'm not sure where you're coming from with the "huge clusterfuck of json" comment, but where the source data comes from is quite cleanly separated from how it's rendered using React, with one major caveat I'll come back to in a minute. React itself is essentially a tool for rendering templates but with a declarative programming style, plus one fundamental performance optimisation that makes that declarative style fast enough to use in practice, plus a small number of hooks to integrate quite flexibly with other parts of your code and to allow for further performance optimisation if you need it. Of course, there is no magic and there are always trade-offs. Most visibly, like any declarative system, sometimes the costs are significant and you do need to give it a hand to get acceptable performance. This is where the caveat I mentioned comes in, because you'll find a lot of projects using React also wind up using immutable data structures of one kind or another to store their model state. That lets you do fast comparisons based on identity rather than deep equality to see whether relevant parts of your underlying model data have changed, and thus lets you bypass the whole re-rendering process in areas of your UI where nothing interesting has happened. There are a handful of knock-on effects in practice that you have to deal with sometimes as well. The upside, of course, is that you get much the same advantages as any other declarative programming style: your rendering logic tends to be simpler, more concise, more amenable to analysis, and less subject to surprising interactions and edge cases than more traditional, imperative rendering code tends to be, and all of these become disproportionately greater advantages as the complexity of the rendering and/or the underlying data model grow.
    - meshko 11y ago
      Yeah, our consultants set us up using immutable.js. Of course then it turned out that some components don't work with it, causing more craziness. By "huge clusterfuck of json" I meant that it appears that entire app state is combined into a single huge data structure, which has its positive sides (like ability to save and load state easily), but also sounds scary on an intuitive level.
- kccqzy 11y ago
  You can perfectly use React without npm. Just download the prebuilt development and production versions of react and a simple <script src="..."> would do.
  - scribu 11y ago
    Ok, so what about JSX? Yes, you can write React templates without JSX, but not happily.
Silhouette 11y ago
This is "normal" mostly to projects where the development team has little general development experience and in particular little exposure to how software is developed away from the JS ecosystem. In fairness, plenty of people have long warned that recent trends in the JS ecosystem were unhealthy. It's always been possible to run useful modern tools like Browserify and SASS directly, without all the build systems and task runners and other such tools, and plenty of development teams do, and those teams don't run into NPM hell to the same extent.
voltagex_ 11y ago
Yep, just came out of a 3 month project. I was lucky that there weren't so many dependencies, but I still couldn't run the stylesheet build stuff because compass couldn't build a C extension or something. Count 'em, that's C, Ruby and Node, just to compile SASS to CSS.
- johnny22 11y ago
  at least you can keep it to just C and node if libsass (via node-sass) is good enough for you nowadays. libsass still doesn't support quite everything though :(
AgentME 11y ago
> Apparently, it is considered normal now to have build process download dozens of these npm modules. Tons of NPM modules are small single functions with tests. NPM requires very little boilerplate to publish modules, so you get lots of small and focused (and hopefully composable) modules. Not every module is a megabyte+ one-stop kitchen-sink-containing toolkit that's mostly redundant with your other dependencies. Though the linked article is specifically complaining about unmaintained tools in the sbt ecosystem. It doesn't appear he's using a build system using NPM modules.
ignoramous 11y ago
Node has had problems with async-programming model and error-handling since the very beginning, but the ecosystem has evolved, and has a wide array of solutions to both these problems. It isn't anyone's fault that best-practices aren't followed. If the code is open-source, then its easier to fix because node-modules, more often than not, are "unix-ey" in that they do one thing, and hopefully, you expect that it does that well. I think you'd find the real problem is with the tendency by the maintainers to abandon a lot of these modules...
deleted 11y ago
[deleted]

by meshko 11y ago

pachydermic 11y ago

It's maddening. >The real painful part of the process is caused by everything around React, including the build tools. Can't agree with that more. I really like the way react works. It just makes sense and doesn't get in your way very often. BUT it's incredibly frustrating dealing with npm dependency hell. Especially if you really are concerned about security and don't just trust that "someone else has probably made sure this code is safe, right?" I just want to use something for longer than 6 months :(

meshko 11y ago
Right. React itself appears somewhat reasonable, except I am not entirely sure that putting entire app state into a huge clusterfuck of json is going to scale, but I am trying to reserve my judgement until I try it myself. And shadow DOM... sigh. It's a cool hack, but it feels like surrender to me.
- Silhouette 11y ago
  FWIW, as an experienced developer whose instincts are probably quite similar to yours, I do recommend keeping an open mind with tools like React. They're different to a lot of more traditional tools, but like optimisations and profiling, they're best judged on their real world results rather than against dogma. I'm not sure where you're coming from with the "huge clusterfuck of json" comment, but where the source data comes from is quite cleanly separated from how it's rendered using React, with one major caveat I'll come back to in a minute. React itself is essentially a tool for rendering templates but with a declarative programming style, plus one fundamental performance optimisation that makes that declarative style fast enough to use in practice, plus a small number of hooks to integrate quite flexibly with other parts of your code and to allow for further performance optimisation if you need it. Of course, there is no magic and there are always trade-offs. Most visibly, like any declarative system, sometimes the costs are significant and you do need to give it a hand to get acceptable performance. This is where the caveat I mentioned comes in, because you'll find a lot of projects using React also wind up using immutable data structures of one kind or another to store their model state. That lets you do fast comparisons based on identity rather than deep equality to see whether relevant parts of your underlying model data have changed, and thus lets you bypass the whole re-rendering process in areas of your UI where nothing interesting has happened. There are a handful of knock-on effects in practice that you have to deal with sometimes as well. The upside, of course, is that you get much the same advantages as any other declarative programming style: your rendering logic tends to be simpler, more concise, more amenable to analysis, and less subject to surprising interactions and edge cases than more traditional, imperative rendering code tends to be, and all of these become disproportionately greater advantages as the complexity of the rendering and/or the underlying data model grow.
  - meshko 11y ago
    Yeah, our consultants set us up using immutable.js. Of course then it turned out that some components don't work with it, causing more craziness. By "huge clusterfuck of json" I meant that it appears that entire app state is combined into a single huge data structure, which has its positive sides (like ability to save and load state easily), but also sounds scary on an intuitive level.
kccqzy 11y ago
You can perfectly use React without npm. Just download the prebuilt development and production versions of react and a simple <script src="..."> would do.
- scribu 11y ago
  Ok, so what about JSX? Yes, you can write React templates without JSX, but not happily.

Silhouette 11y ago

This is "normal" mostly to projects where the development team has little general development experience and in particular little exposure to how software is developed away from the JS ecosystem. In fairness, plenty of people have long warned that recent trends in the JS ecosystem were unhealthy. It's always been possible to run useful modern tools like Browserify and SASS directly, without all the build systems and task runners and other such tools, and plenty of development teams do, and those teams don't run into NPM hell to the same extent.

voltagex_ 11y ago

Yep, just came out of a 3 month project. I was lucky that there weren't so many dependencies, but I still couldn't run the stylesheet build stuff because compass couldn't build a C extension or something. Count 'em, that's C, Ruby and Node, just to compile SASS to CSS.

johnny22 11y ago
at least you can keep it to just C and node if libsass (via node-sass) is good enough for you nowadays. libsass still doesn't support quite everything though :(

AgentME 11y ago

> Apparently, it is considered normal now to have build process download dozens of these npm modules. Tons of NPM modules are small single functions with tests. NPM requires very little boilerplate to publish modules, so you get lots of small and focused (and hopefully composable) modules. Not every module is a megabyte+ one-stop kitchen-sink-containing toolkit that's mostly redundant with your other dependencies. Though the linked article is specifically complaining about unmaintained tools in the sbt ecosystem. It doesn't appear he's using a build system using NPM modules.

ignoramous 11y ago

Node has had problems with async-programming model and error-handling since the very beginning, but the ecosystem has evolved, and has a wide array of solutions to both these problems. It isn't anyone's fault that best-practices aren't followed. If the code is open-source, then its easier to fix because node-modules, more often than not, are "unix-ey" in that they do one thing, and hopefully, you expect that it does that well. I think you'd find the real problem is with the tendency by the maintainers to abandon a lot of these modules...

deleted 11y ago

[deleted]