6 ms·

So are you saying you store them, but not in your version control? What do you mean, like backups of production? What about when production only contains minifi

So are you saying you store them, but not in your version control? What do you mean, like backups of production? What about when production only contains minified / compiled artifacts from the build? Where should I be storing my dependencies? Got any articles to share that elaborate? Or links to open source projects on GitHub that employ the "best practice"? Everyone seems to be just committing bower.json package.json composer.json etc.. & calling it a day.

viraptor 9y ago
Basically don't rely on the external sources. Keep your own repository with the packages you need. Whether that's controlled with git, backups, prepared S3 bucket or other ways is less relevant. The reason stuffing the dependencies with the normal source is bad is you have a huge repo after a while, where most of the size comes from old dep versions you're unlikely to ever use. (But have to clone anyway)
- joshribakoff 9y ago
  I guess it's a tradeoff. A bloated history sucks for sure. So does maintaining brittle legacy code that has to change often. It's a tradeoff between a bloated repo and being able to easily bisect issues from arbitrary points in the projects history. I was hoping you knew of some tool that maintains the dependencies separately while correlating them with specific commits in the main repo. It's nice being able to just checkout arbitrary commits and have it just work. In one project there was dozens of package.json files. I committed the node_modules as a short term hack to get a one step build. It makes git very slow but it was worth the tradeoff for this particular project. Another issue is node gyp since its platform specific, you don't want to be committing mac specific stuff when some team members use Linux. Managing dependencies can really suck sometimes. Another consideration is deployment. In my experience deploying using Capistrano and git is often faster than rsyncing a bunch of files or copying from s3 or wherever. My conclusion for this project is yes it sucks that our repo is now bloated. But maybe that should suck. Maybe it makes us more cognizant of our dependency issues. Maybe having tons of deps should cause you pain, as motivation to more carefully consider your dependencies
dozzie 9y ago
> So are you saying you store them, but not in your version control? What do you mean, like backups of production? Ask your nearby Linux sysadmins about binary packages and source packages.
- joshribakoff 9y ago
  can you elaborate? How does this help with npm packages
  - dozzie 9y ago
    Do you think websites are somehow different than all the rest of the software that they cannot be put into an RPM nor have all their dependencies bundled in a SRPM?
    - joshribakoff 9y ago
      Can you please try to communicate in some other way besides asking rhetorical questions, it comes off as rude & elitist. AFAIK most package managers are essentially a .tar with a checksum. Is there some specific toolset or methodology you follow? Eg. how does each project declare which RPMs? How are these RPMs getting created, by hand and one per package? Or one per project, via some kind of tool? I've literally never heard of anyone using rpms for nodeJS dependencies, and I am intrigued. Let's take an example project with 500 sub-folders under node_modules, are you suggesting I bundle the whole project into an rpm including its dependencies? Because I already have server backups & such, which accomplishes the same thing. Before I spent time making an RPM to backup the project, I could just take container backups or something. My actual issue is when you have only minified code, or compiled executable -- and your boss wants you to add new features, but you do not have the sources, only a package.json that references projects that have been deleted. Or worse, no package.json -- not much can be done at this point, the onus was on the original author. Or are you suggesting each dependency gets wrapped into an rpm? Because that sounds time consuming & doesn't seem like it provides any benefit [it'd be just swapping npm for rpm, it sounds like?]. Plus in my parent comment, the example I used, I inherited a project where 10% of the node_modules were simply missing, the project's author built the code, then deleted his node_modules & left the company. I had to derive what the dependencies were by weeding through minified code. Not sure how RPMs help. I'm also not sure how using RPMs helps when you have developers who may be on different distros or mac, now they have to run docker or vagrant to get the frontend deps? ew. But perhaps I'm not understanding what you're suggesting to do with RPMs, you're not communicating very clearly or nicely IMO. Perhaps linking to an example repo employing your technique or an article that explains it in more detail would be a better way to get your point across. AFAIK, RPM & NPM were made to solve totally different problems. Trying to use NPM to manage linux packages or using RPM to manage javascript packages honestly sounds like a nightmare, going against the grain. RPM was made for linux packages, not frontend web development. I'm trying to keep an open mind but you're not explaining yourself.

viraptor 9y ago

Basically don't rely on the external sources. Keep your own repository with the packages you need. Whether that's controlled with git, backups, prepared S3 bucket or other ways is less relevant. The reason stuffing the dependencies with the normal source is bad is you have a huge repo after a while, where most of the size comes from old dep versions you're unlikely to ever use. (But have to clone anyway)

joshribakoff 9y ago
I guess it's a tradeoff. A bloated history sucks for sure. So does maintaining brittle legacy code that has to change often. It's a tradeoff between a bloated repo and being able to easily bisect issues from arbitrary points in the projects history. I was hoping you knew of some tool that maintains the dependencies separately while correlating them with specific commits in the main repo. It's nice being able to just checkout arbitrary commits and have it just work. In one project there was dozens of package.json files. I committed the node_modules as a short term hack to get a one step build. It makes git very slow but it was worth the tradeoff for this particular project. Another issue is node gyp since its platform specific, you don't want to be committing mac specific stuff when some team members use Linux. Managing dependencies can really suck sometimes. Another consideration is deployment. In my experience deploying using Capistrano and git is often faster than rsyncing a bunch of files or copying from s3 or wherever. My conclusion for this project is yes it sucks that our repo is now bloated. But maybe that should suck. Maybe it makes us more cognizant of our dependency issues. Maybe having tons of deps should cause you pain, as motivation to more carefully consider your dependencies

dozzie 9y ago

> So are you saying you store them, but not in your version control? What do you mean, like backups of production? Ask your nearby Linux sysadmins about binary packages and source packages.

joshribakoff 9y ago
can you elaborate? How does this help with npm packages
- dozzie 9y ago
  Do you think websites are somehow different than all the rest of the software that they cannot be put into an RPM nor have all their dependencies bundled in a SRPM?
  - joshribakoff 9y ago
    Can you please try to communicate in some other way besides asking rhetorical questions, it comes off as rude & elitist. AFAIK most package managers are essentially a .tar with a checksum. Is there some specific toolset or methodology you follow? Eg. how does each project declare which RPMs? How are these RPMs getting created, by hand and one per package? Or one per project, via some kind of tool? I've literally never heard of anyone using rpms for nodeJS dependencies, and I am intrigued. Let's take an example project with 500 sub-folders under node_modules, are you suggesting I bundle the whole project into an rpm including its dependencies? Because I already have server backups & such, which accomplishes the same thing. Before I spent time making an RPM to backup the project, I could just take container backups or something. My actual issue is when you have only minified code, or compiled executable -- and your boss wants you to add new features, but you do not have the sources, only a package.json that references projects that have been deleted. Or worse, no package.json -- not much can be done at this point, the onus was on the original author. Or are you suggesting each dependency gets wrapped into an rpm? Because that sounds time consuming & doesn't seem like it provides any benefit [it'd be just swapping npm for rpm, it sounds like?]. Plus in my parent comment, the example I used, I inherited a project where 10% of the node_modules were simply missing, the project's author built the code, then deleted his node_modules & left the company. I had to derive what the dependencies were by weeding through minified code. Not sure how RPMs help. I'm also not sure how using RPMs helps when you have developers who may be on different distros or mac, now they have to run docker or vagrant to get the frontend deps? ew. But perhaps I'm not understanding what you're suggesting to do with RPMs, you're not communicating very clearly or nicely IMO. Perhaps linking to an example repo employing your technique or an article that explains it in more detail would be a better way to get your point across. AFAIK, RPM & NPM were made to solve totally different problems. Trying to use NPM to manage linux packages or using RPM to manage javascript packages honestly sounds like a nightmare, going against the grain. RPM was made for linux packages, not frontend web development. I'm trying to keep an open mind but you're not explaining yourself.