3 ms·

The value proposition of only checking in your code is the more dependencies you add to your source, the more you have to keep it all up to date. As your boss,

The value proposition of only checking in your code is the more dependencies you add to your source, the more you have to keep it all up to date. As your boss, I pay you for features/value, not to maintain dependencies. I get that Mozilla stuff is cool and that you want to use React or Next or whatever, but I won't pay you to work on that stuff. I'll pay you to work on <SaaS-Of-Tomorrow>. Also, I want to have smaller deployments because our cloud bill is enormous. So make sure any dependency we are using is using the same versions and that we only include 1 of them at build time. Vendoring or including your version of structlog (then npm installing someone elses version of structlog, and so on, and so on) will end up with 1024 instances of structlog package. Totaling 400mb. For a ~3mb library. We use package managers to explicitly say "These things are required to build and run but don't belong to me, my project, or even my organization". You don't want to write Yet Another Markup Language ;) or write another JSX. So you use a package. If you include that package into your source code, and that package is updated, and other dependencies or even - users of your package, will end up with version mismatch and will come with pitchforks and torches to your door.

cxr 3y ago
Thanks for engaging. Unfortunately this is still a little bit too handwavy and vague. I think it would help to flesh out your 3MB/400MB example. Something but not quite necessarily conforming to a set of steps to reproduce would be really reassuring in the, "Look, really, there's nothing up my sleeve" department. Again, what I'm after is a hypothesis that can be clearly stated and subsequently proven or disproven. We're trying to do science here[1]. > the more dependencies you add to your source, the more you have to keep it all up to date There's something unspoken here that you're leaving out, because that doesn't follow. (Or, to put it another way, "Compared to what?") > We use package managers to explicitly say "These things are required to build and run but don't belong to me, my project, or even my organization" I know what you're getting at, but that is neither explicit nor is being able to "say" that the reason why. (If that were the true reason why, a file that simply says exactly that would suffice. And indeed, continuing to use your existing package.json as-is is not something that's precluded by the commit-all-your-dependencies approach.) 1. <https://news.ycombinator.com/item?id=38427548 https://news.ycombinator.com/item?id=38427548>
- reactordev 3y ago
  If you want to store your whole dependency tree in git, be my guest, all 3rd party packages vendor into your repo. After a few months you’ll be spending more time keeping them all up to date than building features. This is something measurable. Something falsifiable. Something that has bit us in the past and is the reason why package managers exist.
  - cxr 3y ago
    > This is something measurable. Something falsifiable. Well, not quite. We're sort of getting there—much closer than before, at least. We were also kind of getting there, though, with your 3MB/400MB example, too. But you abandoned it. Why? A terse off-the-cuff, still-too-unclear comment like "you'll be spending more time keeping them all up to date" isn't exactly a rigorously specified experiment; again, this is way too vague. I don't understand why this is so excruciatingly difficult, esp. in the defense of a practice that is purportedly well-founded and presumably well-understood. It should be as simple as this: "Hello, I see that you have a project and up 'til now have been storing everything in Git. Try doing this instead: [...] What you will observe is that compared to having everything managed as first-class objects in your version control system, this package management discipline affords you $X and $Y and quantitatively it results in $Z [&c...]" It is your job to fill in the blanks. What _exactly_ are you comparing? What _exactly_ are the benefits, in quantitative terms? What are we going to see when we subject these claims to scrutiny? (And what _exactly_ should we do in order to subject these claims to scrutiny? What is the experiment?) In other words, please clearly state the hypothesis. It needs to be testable. Being vague instead provides way too much cover for bad, not-well-founded ideas. (If there's an argument that going to such lengths to state these things is needlessly tedious, there are a few things to take into account. 1. in science, you don't get to just wave your hands and skip over crucial stuff, so if you're attacking this, you're attacking the entire scientific process; 2. programmers are (or should be, at least) particularly adept at this level of tedium—programming a computer is like explaining things to a really, really dumb person by saying exactly what you mean, and providing testcases and/or written steps to reproduce an issue just an ordinary day (or should be, at least); 3. programmers are responsible for tons of bugs over their careers, most of them probably not released to the public because they've been spotted and debugged on the workbench (before any in-progress work is released into the wild) and in doing so are confronted with a bunch of misapprehensions during debugging sessions about what the code should be doing versus what it is actually doing; 4. for all the words written and energy otherwise spent in these nearly 100 comments, we could have been there by now, and ditto even if we narrowly focus only on this subthread.) And to the extent that I can make sense of what you are saying ("you’ll be spending more time keeping them all up to date"), it looks like you might actually (maybe unintentionally) be doing a bait-and-switch here. If today your workflow consists of: 1. git clone; 2. running your $PKGMGR's install step; 3. running `$PKGMGR update` from time to time + bumping the version strings in the JSON and/or lockfile ... then the suggestion to keep your dependencies in version control doesn't really change wrt updates—keeping the packages up to date remains as easy as ever (you just... run `$PKGMGR update` like before), the only difference being that it does change your workflow slightly: step 2 is no longer required, and step 3 necessarily involves an eventual push where you're not only pushing a change to a file of metadata that contains the version strings of your dependencies (you're pushing the contents of the dependencies themselves). It sounds like you're saying that if you keep your dependencies in version control, you necessarily have to stop using automation for your updates, which is where the bait-and-switch comes in: you're now talking about a different argument entirely. We're talking about keeping a copy of the packages right there in the version control system—the one that you're already using—instead of having it half in the bag and delegating the rest to package infrastructure associated with (maybe even operated by) the package manager you're using. Am I misreading you? Are you actually saying something else—that the simple process of syncing the updates to your cloud-hosted repo (pushing those changes) is slow to complete? (I also can't help but notice that in your other comments, you're actually strenuously arguing for the use of Git submodules—you don't actually disagree with the proposition that dependencies belong in version control... so what are we even doing here...?)

cxr 3y ago

Thanks for engaging. Unfortunately this is still a little bit too handwavy and vague. I think it would help to flesh out your 3MB/400MB example. Something but not quite necessarily conforming to a set of steps to reproduce would be really reassuring in the, "Look, really, there's nothing up my sleeve" department. Again, what I'm after is a hypothesis that can be clearly stated and subsequently proven or disproven. We're trying to do science here[1]. > the more dependencies you add to your source, the more you have to keep it all up to date There's something unspoken here that you're leaving out, because that doesn't follow. (Or, to put it another way, "Compared to what?") > We use package managers to explicitly say "These things are required to build and run but don't belong to me, my project, or even my organization" I know what you're getting at, but that is neither explicit nor is being able to "say" that the reason why. (If that were the true reason why, a file that simply says exactly that would suffice. And indeed, continuing to use your existing package.json as-is is not something that's precluded by the commit-all-your-dependencies approach.) 1. <https://news.ycombinator.com/item?id=38427548 https://news.ycombinator.com/item?id=38427548>

reactordev 3y ago
If you want to store your whole dependency tree in git, be my guest, all 3rd party packages vendor into your repo. After a few months you’ll be spending more time keeping them all up to date than building features. This is something measurable. Something falsifiable. Something that has bit us in the past and is the reason why package managers exist.
- cxr 3y ago
  > This is something measurable. Something falsifiable. Well, not quite. We're sort of getting there—much closer than before, at least. We were also kind of getting there, though, with your 3MB/400MB example, too. But you abandoned it. Why? A terse off-the-cuff, still-too-unclear comment like "you'll be spending more time keeping them all up to date" isn't exactly a rigorously specified experiment; again, this is way too vague. I don't understand why this is so excruciatingly difficult, esp. in the defense of a practice that is purportedly well-founded and presumably well-understood. It should be as simple as this: "Hello, I see that you have a project and up 'til now have been storing everything in Git. Try doing this instead: [...] What you will observe is that compared to having everything managed as first-class objects in your version control system, this package management discipline affords you $X and $Y and quantitatively it results in $Z [&c...]" It is your job to fill in the blanks. What _exactly_ are you comparing? What _exactly_ are the benefits, in quantitative terms? What are we going to see when we subject these claims to scrutiny? (And what _exactly_ should we do in order to subject these claims to scrutiny? What is the experiment?) In other words, please clearly state the hypothesis. It needs to be testable. Being vague instead provides way too much cover for bad, not-well-founded ideas. (If there's an argument that going to such lengths to state these things is needlessly tedious, there are a few things to take into account. 1. in science, you don't get to just wave your hands and skip over crucial stuff, so if you're attacking this, you're attacking the entire scientific process; 2. programmers are (or should be, at least) particularly adept at this level of tedium—programming a computer is like explaining things to a really, really dumb person by saying exactly what you mean, and providing testcases and/or written steps to reproduce an issue just an ordinary day (or should be, at least); 3. programmers are responsible for tons of bugs over their careers, most of them probably not released to the public because they've been spotted and debugged on the workbench (before any in-progress work is released into the wild) and in doing so are confronted with a bunch of misapprehensions during debugging sessions about what the code should be doing versus what it is actually doing; 4. for all the words written and energy otherwise spent in these nearly 100 comments, we could have been there by now, and ditto even if we narrowly focus only on this subthread.) And to the extent that I can make sense of what you are saying ("you’ll be spending more time keeping them all up to date"), it looks like you might actually (maybe unintentionally) be doing a bait-and-switch here. If today your workflow consists of: 1. git clone; 2. running your $PKGMGR's install step; 3. running `$PKGMGR update` from time to time + bumping the version strings in the JSON and/or lockfile ... then the suggestion to keep your dependencies in version control doesn't really change wrt updates—keeping the packages up to date remains as easy as ever (you just... run `$PKGMGR update` like before), the only difference being that it does change your workflow slightly: step 2 is no longer required, and step 3 necessarily involves an eventual push where you're not only pushing a change to a file of metadata that contains the version strings of your dependencies (you're pushing the contents of the dependencies themselves). It sounds like you're saying that if you keep your dependencies in version control, you necessarily have to stop using automation for your updates, which is where the bait-and-switch comes in: you're now talking about a different argument entirely. We're talking about keeping a copy of the packages right there in the version control system—the one that you're already using—instead of having it half in the bag and delegating the rest to package infrastructure associated with (maybe even operated by) the package manager you're using. Am I misreading you? Are you actually saying something else—that the simple process of syncing the updates to your cloud-hosted repo (pushing those changes) is slow to complete? (I also can't help but notice that in your other comments, you're actually strenuously arguing for the use of Git submodules—you don't actually disagree with the proposition that dependencies belong in version control... so what are we even doing here...?)