6 ms·

Just out of curiosity as I've never personally experienced any bugs of the sort, but what is an example of a really nasty bug that someone has experienced that

Just out of curiosity as I've never personally experienced any bugs of the sort, but what is an example of a really nasty bug that someone has experienced that this sort of thing prevents?

viraptor 9y ago
Asterisk discovered licensing issues with their sound files at some point. Instead of updating old packages with .1 at the end of the version, they replaced existing ones in the repo. That caused failures in build systems which verified checksums and suddenly-different builds on systems which didn't.
joshribakoff 9y ago
Inherited project running in production, sources in git did not build. Project depended on packages which were present when it was built, but never added to package.json. Code in production was minified. Had to track down a ton of missing npm dependencies. Another example is someone depends on a library, then that library's author deletes it. This is why it may be worth committing your dependency's sources, despite "best practices" dictating otherwise. When someone needs to add a feature 10yrs from now and they only have an incomplete package.json & a copy of minified code, they're in for a world of hell. I even had to recover a project which built non deterministically. Sometimes it would build, sometimes it would error. Running the build several times, it would eventually succeed. The project's author had written their build system against an unstable version of Webpack & Typescript, before they were popular (both of which are now perfectly stable).
- taneq 9y ago
  > This is why it may be worth committing your dependency's sources, despite "best practices" dictating otherwise. Maybe I'm not a hip young web developer but I've always considered "best practice" to be keeping the currently-in-use version of all of your product's dependencies in a repo (either your main one or a separate 'dependencies' repo). If you update the dependency, you test against the new version and check it in if successful. The idea of letting your build process depend on a third-party library that's downloaded ad-hoc (and may or may not be trustworthy) is terrifying.
- viraptor 9y ago
  > This is why it may be worth committing your dependency's sources, despite "best practices" dictating otherwise I think you may have misunderstood the best practice here. That best practice is you shouldn't just copy the dep code into your repo. The maintenance/release best practice however is that you should always have a copy of all your dependencies available locally and permanently store them. They're two completely separate rules.
  - joshribakoff 9y ago
    So are you saying you store them, but not in your version control? What do you mean, like backups of production? What about when production only contains minified / compiled artifacts from the build? Where should I be storing my dependencies? Got any articles to share that elaborate? Or links to open source projects on GitHub that employ the "best practice"? Everyone seems to be just committing bower.json package.json composer.json etc.. & calling it a day.
    - viraptor 9y ago
      Basically don't rely on the external sources. Keep your own repository with the packages you need. Whether that's controlled with git, backups, prepared S3 bucket or other ways is less relevant. The reason stuffing the dependencies with the normal source is bad is you have a huge repo after a while, where most of the size comes from old dep versions you're unlikely to ever use. (But have to clone anyway)
      - joshribakoff 9y ago
        I guess it's a tradeoff. A bloated history sucks for sure. So does maintaining brittle legacy code that has to change often. It's a tradeoff between a bloated repo and being able to easily bisect issues from arbitrary points in the projects history. I was hoping you knew of some tool that maintains the dependencies separately while correlating them with specific commits in the main repo. It's nice being able to just checkout arbitrary commits and have it just work. In one project there was dozens of package.json files. I committed the node_modules as a short term hack to get a one step build. It makes git very slow but it was worth the tradeoff for this particular project. Another issue is node gyp since its platform specific, you don't want to be committing mac specific stuff when some team members use Linux. Managing dependencies can really suck sometimes. Another consideration is deployment. In my experience deploying using Capistrano and git is often faster than rsyncing a bunch of files or copying from s3 or wherever. My conclusion for this project is yes it sucks that our repo is now bloated. But maybe that should suck. Maybe it makes us more cognizant of our dependency issues. Maybe having tons of deps should cause you pain, as motivation to more carefully consider your dependencies

Just out of curiosity as I've never personally experienced any bugs of the sort, but what is an example of a really nasty bug that someone has experienced that this sort of thing prevents?

viraptor 9y ago

Asterisk discovered licensing issues with their sound files at some point. Instead of updating old packages with .1 at the end of the version, they replaced existing ones in the repo. That caused failures in build systems which verified checksums and suddenly-different builds on systems which didn't.

joshribakoff 9y ago

Inherited project running in production, sources in git did not build. Project depended on packages which were present when it was built, but never added to package.json. Code in production was minified. Had to track down a ton of missing npm dependencies. Another example is someone depends on a library, then that library's author deletes it. This is why it may be worth committing your dependency's sources, despite "best practices" dictating otherwise. When someone needs to add a feature 10yrs from now and they only have an incomplete package.json & a copy of minified code, they're in for a world of hell. I even had to recover a project which built non deterministically. Sometimes it would build, sometimes it would error. Running the build several times, it would eventually succeed. The project's author had written their build system against an unstable version of Webpack & Typescript, before they were popular (both of which are now perfectly stable).

taneq 9y ago
> This is why it may be worth committing your dependency's sources, despite "best practices" dictating otherwise. Maybe I'm not a hip young web developer but I've always considered "best practice" to be keeping the currently-in-use version of all of your product's dependencies in a repo (either your main one or a separate 'dependencies' repo). If you update the dependency, you test against the new version and check it in if successful. The idea of letting your build process depend on a third-party library that's downloaded ad-hoc (and may or may not be trustworthy) is terrifying.
viraptor 9y ago
> This is why it may be worth committing your dependency's sources, despite "best practices" dictating otherwise I think you may have misunderstood the best practice here. That best practice is you shouldn't just copy the dep code into your repo. The maintenance/release best practice however is that you should always have a copy of all your dependencies available locally and permanently store them. They're two completely separate rules.
- joshribakoff 9y ago
  So are you saying you store them, but not in your version control? What do you mean, like backups of production? What about when production only contains minified / compiled artifacts from the build? Where should I be storing my dependencies? Got any articles to share that elaborate? Or links to open source projects on GitHub that employ the "best practice"? Everyone seems to be just committing bower.json package.json composer.json etc.. & calling it a day.
  - viraptor 9y ago
    Basically don't rely on the external sources. Keep your own repository with the packages you need. Whether that's controlled with git, backups, prepared S3 bucket or other ways is less relevant. The reason stuffing the dependencies with the normal source is bad is you have a huge repo after a while, where most of the size comes from old dep versions you're unlikely to ever use. (But have to clone anyway)
    - joshribakoff 9y ago
      I guess it's a tradeoff. A bloated history sucks for sure. So does maintaining brittle legacy code that has to change often. It's a tradeoff between a bloated repo and being able to easily bisect issues from arbitrary points in the projects history. I was hoping you knew of some tool that maintains the dependencies separately while correlating them with specific commits in the main repo. It's nice being able to just checkout arbitrary commits and have it just work. In one project there was dozens of package.json files. I committed the node_modules as a short term hack to get a one step build. It makes git very slow but it was worth the tradeoff for this particular project. Another issue is node gyp since its platform specific, you don't want to be committing mac specific stuff when some team members use Linux. Managing dependencies can really suck sometimes. Another consideration is deployment. In my experience deploying using Capistrano and git is often faster than rsyncing a bunch of files or copying from s3 or wherever. My conclusion for this project is yes it sucks that our repo is now bloated. But maybe that should suck. Maybe it makes us more cognizant of our dependency issues. Maybe having tons of deps should cause you pain, as motivation to more carefully consider your dependencies