4 ms·

> Go learn vcpkg and come back to us when you learn why everyone does it this way. What reason do you have in mind? Can you articulate it in a way that can be

by cxr 3y ago

> Go learn vcpkg and come back to us when you learn why everyone does it this way. What reason do you have in mind? Can you articulate it in a way that can be quantified/falsified? The last time I pressed someone on HN about this[1], it quickly devolved into vague handwaving[2] of the same sort—that we all already know why so we don't have to talk about it. (Asking because I thought I knew, and then one day I realized I couldn't actually explain it, at least not in a way that I could be confident that everyone else would actually agree with. It feels a lot like the hand of Ra[3] is at play.) I'm curious to hear from anyone who thinks they know why or can perhaps point to where they've already explained it. I'm particularly interested in responses from people who deal with NPM. (What is the "NPM hypothesis", i.e. the value proposition for why having a skeleton source tree in Git that delegates the bulk of the work to `npm install` is objectively better—or subjectively better, even? Can you state it in the form of a falsifiable claim? If you're able to state a testable hypothesis and you have tested it, can you show your work?) 1. <https://news.ycombinator.com/item?id=37605966 https://news.ycombinator.com/item?id=37605966> 2. <https://news.ycombinator.com/item?id=37607202 https://news.ycombinator.com/item?id=37607202> 3. <https://srconstantin.wordpress.com/2016/10/20/ra/ https://srconstantin.wordpress.com/2016/10/20/ra/>

pinkgolem 3y ago
In case you are serious, and I am not super sure about that. - depencys include blobs, or other blob like things(big code bases, think react) - there is little value in seeing changes in a blob or blob like things as changes in those have a bad signal to noise ratio. Version information (minor/major) + changelog is much more interesting for the average dev - there is a performance penalty for having to many things/big things in git - there is no nice way to keep dependencies up to date in git - the discoverability of git/GitHub has not been great You can use git as a package provider, see pip for example, but using git alone is painfull.
- forrestthewoods 3y ago
  This is an argument that Git has serious limitation (it does). It's not an argument against the theory of dependencies in VCS.
  - pinkgolem 3y ago
    1/2 are pretty universal, and would make any VCS which includes dependencys basically look like any other package manager today. Every other existing VCS btw to my knowledge has the same limits
  - reactordev 3y ago
    Here's an argument for not having your dependencies in your repo as hard source. Any CVE's will need to be patched by you or you'll need to check-in a huge diff tree for the latest version patched and then figure out of you've made modifications to your local blob of libidiot that you now have to rectify. If you want to use git for version control of dependencies, for the love of god use git submodules. It's what they were designed for. If you want to check-in single file stb_lib fine but checking in MESA into your source tree is just plain stupid.
    - forrestthewoods 3y ago
      Git Submodules are a well known source of pain and suffering. For CVEs, if you use docker images you need to rebuild and redeploy all images which isn’t particularly different from updating committed code. If you’ve modified a library you have to merge changes any which way. So not sure what difference it makes. Checking in GPU drivers probably doesn’t make sense. That’s closer to Docker territory at least. Haven’t run into a case where I’ve wanted/needed to do that so no strong opinion.
      - reactordev 3y ago
        Git submodules work flawlessly for me because I use them correctly. So YMMV. There’s plenty of documentation out there on how to do a git clone —recursive or git submodule update —init —recursive Never edit the code in the submodules, edit them in the parent repos where they properly live. Your repo is for your code only. Don’t go wandering:editing in those submodules.
    - deleted 3y ago
      [deleted]
- cxr 3y ago
  > In case you are serious, and I am not super sure about that. What's unclear? I linked directly to a comment thread where we had this exact exchange. Yes, I'm being serious. Do you have solid backing for your position or not? Can you state it clearly or not? "Seriously?" is not an argument. And in this context it is—ironically—a very unserious way to respond. If I had to guess, I'd say it's probably geared more towards being a vehicle for communicating some sort of subtext in the vein of, "You should abstain from asking these sorts of questions because the reasons are so obvious and so well-known that it is a foolish and embarrassing thing to ask, which makes anyone who is asking those questions look foolish and embarrassing." > dependencys include [...] other blob like things(big code bases, think react) [...] there is a performance penalty for having to many things/big things in git This is not a self-contained argument. A performance penalty compared to what? > there is no nice way to keep dependencies up to date in git Huh? (Alternatively: "How is this relevant?") When you check your dependencies in to the Git repo, you have all the same tools to update your dependencies that you have when you don't check them in.
  - pinkgolem 3y ago
    >What's unclear? You linking to Ra and going on other tangends make you seem very trollish/edgelord like. >This is not a self-contained argument. A performance penalty compared to what? Compared to not checking them in? I don't understand what you are trying to say here. >Git repo, you have all the same tools to update So I still need a package manager of my choice & a dependency file to install & update dependencys? That also makes it easy to modify single source files which then only get overwriten on an dependency update as you have no clear separation between source code and dependencys.
    - cxr 3y ago
      > You linking to Ra and going on other tangends make you seem very trollish/edgelord like. Huh? Did you read the essay? There's nothing trollish or edgy about it. It describes exactly the type of response I got in the other thread (and now from you here). > Compared to not checking them in? I don't think you're thinking clearly. You can opt not to check your dependencies in—there's certainly less overhead involved when you don't have your dependencies around compared to having them checked into your repo. But the question then becomes, "How are you going to get your app to build/run if you don't have your dependencies?" We are not talking about building projects with dependencies versus projects that don't have dependencies. Same project, same dependencies. You need these dependencies. That's why they're dependencies. When you say there are performance problems when you have a big Git repo, what are you actually comparing that _to_? > I don't understand what you are trying to say here. I haven't strayed from my goal. I am looking for someone to provide a clear answer to the question, "What is the NPM hypothesis?" In other words, state clearly in quantitative terms how it is that keeping e.g. 12% of your source code in Git and leaving the other 88% (or whatever) to be fetched after you clone the repo and run `npm install` is better than keeping 100% of the source code, dependencies included, in Git? I'll give you an example by what I mean: if I see that you're trying to email someone some artwork and it's a bunch of Windows BMP files or raw TIFFs, I could say something like, "If you save these as PNGs and JPEGs, they will take up less space on disk and will be quicker to download." We can test this. We can figure out if it's true or not. We have a hypothesis that is falsifiable. This is the basis of science. We're doing computer science. We are not asking people their favorite cheeses. We are not even asking them whether static types are better than dynamic types or if functional programming is good and object-oriented is bad. We are talking about version control systems. There are beliefs involved here, but they are not mere beliefs—I'd guess that you have some sort of belief about what makes NPM + Git the "right" way to do things and just checking the dependencies into Git the "wrong" way that can be stated in the form of one or more fact claims that we can scrutinize, do some measurements, work out the math, etc. and figure out whether those claims are true or untrue. I'm asking you to state clearly what your claims are. What are they? > So I still need a package manager of my choice & a dependency file to install & update dependencys? I don't understand this question. Is it a question? It looks like a declarative statement followed by a question mark instead of a period.
deleted 3y ago
[deleted]
reactordev 3y ago
The value proposition of only checking in your code is the more dependencies you add to your source, the more you have to keep it all up to date. As your boss, I pay you for features/value, not to maintain dependencies. I get that Mozilla stuff is cool and that you want to use React or Next or whatever, but I won't pay you to work on that stuff. I'll pay you to work on <SaaS-Of-Tomorrow>. Also, I want to have smaller deployments because our cloud bill is enormous. So make sure any dependency we are using is using the same versions and that we only include 1 of them at build time. Vendoring or including your version of structlog (then npm installing someone elses version of structlog, and so on, and so on) will end up with 1024 instances of structlog package. Totaling 400mb. For a ~3mb library. We use package managers to explicitly say "These things are required to build and run but don't belong to me, my project, or even my organization". You don't want to write Yet Another Markup Language ;) or write another JSX. So you use a package. If you include that package into your source code, and that package is updated, and other dependencies or even - users of your package, will end up with version mismatch and will come with pitchforks and torches to your door.
- cxr 3y ago
  Thanks for engaging. Unfortunately this is still a little bit too handwavy and vague. I think it would help to flesh out your 3MB/400MB example. Something but not quite necessarily conforming to a set of steps to reproduce would be really reassuring in the, "Look, really, there's nothing up my sleeve" department. Again, what I'm after is a hypothesis that can be clearly stated and subsequently proven or disproven. We're trying to do science here[1]. > the more dependencies you add to your source, the more you have to keep it all up to date There's something unspoken here that you're leaving out, because that doesn't follow. (Or, to put it another way, "Compared to what?") > We use package managers to explicitly say "These things are required to build and run but don't belong to me, my project, or even my organization" I know what you're getting at, but that is neither explicit nor is being able to "say" that the reason why. (If that were the true reason why, a file that simply says exactly that would suffice. And indeed, continuing to use your existing package.json as-is is not something that's precluded by the commit-all-your-dependencies approach.) 1. <https://news.ycombinator.com/item?id=38427548 https://news.ycombinator.com/item?id=38427548>

by cxr 3y ago

pinkgolem 3y ago

In case you are serious, and I am not super sure about that. - depencys include blobs, or other blob like things(big code bases, think react) - there is little value in seeing changes in a blob or blob like things as changes in those have a bad signal to noise ratio. Version information (minor/major) + changelog is much more interesting for the average dev - there is a performance penalty for having to many things/big things in git - there is no nice way to keep dependencies up to date in git - the discoverability of git/GitHub has not been great You can use git as a package provider, see pip for example, but using git alone is painfull.

forrestthewoods 3y ago
This is an argument that Git has serious limitation (it does). It's not an argument against the theory of dependencies in VCS.
- pinkgolem 3y ago
  1/2 are pretty universal, and would make any VCS which includes dependencys basically look like any other package manager today. Every other existing VCS btw to my knowledge has the same limits
- reactordev 3y ago
  Here's an argument for not having your dependencies in your repo as hard source. Any CVE's will need to be patched by you or you'll need to check-in a huge diff tree for the latest version patched and then figure out of you've made modifications to your local blob of libidiot that you now have to rectify. If you want to use git for version control of dependencies, for the love of god use git submodules. It's what they were designed for. If you want to check-in single file stb_lib fine but checking in MESA into your source tree is just plain stupid.
  - forrestthewoods 3y ago
    Git Submodules are a well known source of pain and suffering. For CVEs, if you use docker images you need to rebuild and redeploy all images which isn’t particularly different from updating committed code. If you’ve modified a library you have to merge changes any which way. So not sure what difference it makes. Checking in GPU drivers probably doesn’t make sense. That’s closer to Docker territory at least. Haven’t run into a case where I’ve wanted/needed to do that so no strong opinion.
    - reactordev 3y ago
      Git submodules work flawlessly for me because I use them correctly. So YMMV. There’s plenty of documentation out there on how to do a git clone —recursive or git submodule update —init —recursive Never edit the code in the submodules, edit them in the parent repos where they properly live. Your repo is for your code only. Don’t go wandering:editing in those submodules.
  - deleted 3y ago
    [deleted]
cxr 3y ago
> In case you are serious, and I am not super sure about that. What's unclear? I linked directly to a comment thread where we had this exact exchange. Yes, I'm being serious. Do you have solid backing for your position or not? Can you state it clearly or not? "Seriously?" is not an argument. And in this context it is—ironically—a very unserious way to respond. If I had to guess, I'd say it's probably geared more towards being a vehicle for communicating some sort of subtext in the vein of, "You should abstain from asking these sorts of questions because the reasons are so obvious and so well-known that it is a foolish and embarrassing thing to ask, which makes anyone who is asking those questions look foolish and embarrassing." > dependencys include [...] other blob like things(big code bases, think react) [...] there is a performance penalty for having to many things/big things in git This is not a self-contained argument. A performance penalty compared to what? > there is no nice way to keep dependencies up to date in git Huh? (Alternatively: "How is this relevant?") When you check your dependencies in to the Git repo, you have all the same tools to update your dependencies that you have when you don't check them in.
- pinkgolem 3y ago
  >What's unclear? You linking to Ra and going on other tangends make you seem very trollish/edgelord like. >This is not a self-contained argument. A performance penalty compared to what? Compared to not checking them in? I don't understand what you are trying to say here. >Git repo, you have all the same tools to update So I still need a package manager of my choice & a dependency file to install & update dependencys? That also makes it easy to modify single source files which then only get overwriten on an dependency update as you have no clear separation between source code and dependencys.
  - cxr 3y ago
    > You linking to Ra and going on other tangends make you seem very trollish/edgelord like. Huh? Did you read the essay? There's nothing trollish or edgy about it. It describes exactly the type of response I got in the other thread (and now from you here). > Compared to not checking them in? I don't think you're thinking clearly. You can opt not to check your dependencies in—there's certainly less overhead involved when you don't have your dependencies around compared to having them checked into your repo. But the question then becomes, "How are you going to get your app to build/run if you don't have your dependencies?" We are not talking about building projects with dependencies versus projects that don't have dependencies. Same project, same dependencies. You need these dependencies. That's why they're dependencies. When you say there are performance problems when you have a big Git repo, what are you actually comparing that _to_? > I don't understand what you are trying to say here. I haven't strayed from my goal. I am looking for someone to provide a clear answer to the question, "What is the NPM hypothesis?" In other words, state clearly in quantitative terms how it is that keeping e.g. 12% of your source code in Git and leaving the other 88% (or whatever) to be fetched after you clone the repo and run `npm install` is better than keeping 100% of the source code, dependencies included, in Git? I'll give you an example by what I mean: if I see that you're trying to email someone some artwork and it's a bunch of Windows BMP files or raw TIFFs, I could say something like, "If you save these as PNGs and JPEGs, they will take up less space on disk and will be quicker to download." We can test this. We can figure out if it's true or not. We have a hypothesis that is falsifiable. This is the basis of science. We're doing computer science. We are not asking people their favorite cheeses. We are not even asking them whether static types are better than dynamic types or if functional programming is good and object-oriented is bad. We are talking about version control systems. There are beliefs involved here, but they are not mere beliefs—I'd guess that you have some sort of belief about what makes NPM + Git the "right" way to do things and just checking the dependencies into Git the "wrong" way that can be stated in the form of one or more fact claims that we can scrutinize, do some measurements, work out the math, etc. and figure out whether those claims are true or untrue. I'm asking you to state clearly what your claims are. What are they? > So I still need a package manager of my choice & a dependency file to install & update dependencys? I don't understand this question. Is it a question? It looks like a declarative statement followed by a question mark instead of a period.

deleted 3y ago

[deleted]

reactordev 3y ago

The value proposition of only checking in your code is the more dependencies you add to your source, the more you have to keep it all up to date. As your boss, I pay you for features/value, not to maintain dependencies. I get that Mozilla stuff is cool and that you want to use React or Next or whatever, but I won't pay you to work on that stuff. I'll pay you to work on <SaaS-Of-Tomorrow>. Also, I want to have smaller deployments because our cloud bill is enormous. So make sure any dependency we are using is using the same versions and that we only include 1 of them at build time. Vendoring or including your version of structlog (then npm installing someone elses version of structlog, and so on, and so on) will end up with 1024 instances of structlog package. Totaling 400mb. For a ~3mb library. We use package managers to explicitly say "These things are required to build and run but don't belong to me, my project, or even my organization". You don't want to write Yet Another Markup Language ;) or write another JSX. So you use a package. If you include that package into your source code, and that package is updated, and other dependencies or even - users of your package, will end up with version mismatch and will come with pitchforks and torches to your door.

cxr 3y ago
Thanks for engaging. Unfortunately this is still a little bit too handwavy and vague. I think it would help to flesh out your 3MB/400MB example. Something but not quite necessarily conforming to a set of steps to reproduce would be really reassuring in the, "Look, really, there's nothing up my sleeve" department. Again, what I'm after is a hypothesis that can be clearly stated and subsequently proven or disproven. We're trying to do science here[1]. > the more dependencies you add to your source, the more you have to keep it all up to date There's something unspoken here that you're leaving out, because that doesn't follow. (Or, to put it another way, "Compared to what?") > We use package managers to explicitly say "These things are required to build and run but don't belong to me, my project, or even my organization" I know what you're getting at, but that is neither explicit nor is being able to "say" that the reason why. (If that were the true reason why, a file that simply says exactly that would suffice. And indeed, continuing to use your existing package.json as-is is not something that's precluded by the commit-all-your-dependencies approach.) 1. <https://news.ycombinator.com/item?id=38427548 https://news.ycombinator.com/item?id=38427548>