5 ms·

> The naive exponential algorithm is fine (…) Define “fine”. In my experience, Go with modules[1] solves a project's dependencies exponentially more quickly t

> The naive exponential algorithm is fine (…) Define “fine”. In my experience, Go with modules[1] solves a project's dependencies exponentially more quickly than dep. And that is especially noticeable on big projects with three or more dozens of dependencies. I don't mean any disrespect to the people behind dep, and the whole debacle was a massive miscommunication disaster, but go mod is just quicker. [1] While we're at it, I'm still low-key mad that the Go authors called modules packages and packages modules. Gah!

pcwalton 7y ago
Again, that's because of I/O. The CPU bound part of the algorithm is very fast, as Cargo demonstrates.
- ainar-g 7y ago
  I don't get it. The number of dependencies is constant. Which means that both of them need to make approximately the same amount of network requests. Where does the I/O difference come from then? Also, since you've mentioned Cargo. Its slowness actually used to be one of the main pain points about which the Rust people at one of the companies I've worked for in the past complained a lot during the water cooler discussions. Granted, that was a couple of years ago, and Cargo has probably made progress since then.
  - pcwalton 7y ago
    As I recall, dep had to parse Go code over and over during the dependency resolution process instead of caching it. And any cargo slowness is likely the fault of rustc or I/O. It's received a lot of profiling work and the core dependency solver has never been a performance issue to my knowledge.
    - steveklabnik 7y ago
      It would seem this is correct. From the docs: https://github.com/golang/dep/blob/master/docs/FAQ.md#why-is-dep-slow https://github.com/golang/dep/blob/master/docs/FAQ.md#why-is... There are two things that really slow dep down. One is unavoidable; for the other, we have a plan. The unavoidable part is the initial clone. ... Fortunately, this is just an initial clone - pay it once, and you're done. The other part is the work of retrieving information about dependencies. There are three parts to this: * Getting an up-to-date list of versions from the upstream source * Reading the Gopkg.toml for a particular version out of the local cache * Parsing the tree of packages for import statements at a particular version The first requires one or more network calls; the second two usually mean something like a git checkout, and the third is a filesystem walk, plus loading and parsing .go files. All of these are expensive operations. ---------------- For context, in comparison, Cargo has the same #1 problem. But for #2, the information on all dependencies is stored in the index itself; this means that Cargo can figure out what dependencies you need without any network calls at all, let alone downloading, git checkout, and filesystem walk. I do not know how go mod compares, off the top of my head.
      - deleted 7y ago
        [deleted]
sansnomme 7y ago
The trend these days is tiny packages (similar to PaaS turning into Function-as-a-service), very fine grained libraries like Leftpad. The ultimate realisation of HN's favorite mantra "worship the Unix Philosophy". Expect the number of dependencies to only increase.
- kortex 7y ago
  Except you have things like `sed` and `cut` and `cat`, not `lpd` which solely exists to add leftwise filler characters. Ergonomics of those tools nonwithstanding, they are simple but they aren't TOO simple. Left-pad is IMHO "too simple". Packages on the median should be in a 1-2 pizza team effort with some ongoing maintenance, even if only a few pizzas a year. Left-pad is like, 2 pizzas and a weekend. (were it not for string.prototype methods) a viable package would be something on the order of Python's str.format(). Can left pad and right pad!
  - samatman 7y ago
    Unix notoriously has the command `yes`, which provides an infinite loop of `y\n`. Don't have a strong opinion on whether left-pad should exist or not (or indeed, if server-side Javascript should exist or not), but simpler than left-pad, is yes? yes.
    - iudqnolq 7y ago
      Having a basic building block that enables a lot is different from making every tiny problem it's own package. Yes is a exception, certainly, but it doesn't disprove the rule.
earthboundkid 7y ago
I complained about the term “modules” for the bigger code unit in Reddit, but RSC said they were both vague so it didn’t matter. I do still think it should have been “bundles” or something implying “bigger than a package” instead of “module”.

pcwalton 7y ago

Again, that's because of I/O. The CPU bound part of the algorithm is very fast, as Cargo demonstrates.

ainar-g 7y ago
I don't get it. The number of dependencies is constant. Which means that both of them need to make approximately the same amount of network requests. Where does the I/O difference come from then? Also, since you've mentioned Cargo. Its slowness actually used to be one of the main pain points about which the Rust people at one of the companies I've worked for in the past complained a lot during the water cooler discussions. Granted, that was a couple of years ago, and Cargo has probably made progress since then.
- pcwalton 7y ago
  As I recall, dep had to parse Go code over and over during the dependency resolution process instead of caching it. And any cargo slowness is likely the fault of rustc or I/O. It's received a lot of profiling work and the core dependency solver has never been a performance issue to my knowledge.
  - steveklabnik 7y ago
    It would seem this is correct. From the docs: https://github.com/golang/dep/blob/master/docs/FAQ.md#why-is-dep-slow https://github.com/golang/dep/blob/master/docs/FAQ.md#why-is... There are two things that really slow dep down. One is unavoidable; for the other, we have a plan. The unavoidable part is the initial clone. ... Fortunately, this is just an initial clone - pay it once, and you're done. The other part is the work of retrieving information about dependencies. There are three parts to this: * Getting an up-to-date list of versions from the upstream source * Reading the Gopkg.toml for a particular version out of the local cache * Parsing the tree of packages for import statements at a particular version The first requires one or more network calls; the second two usually mean something like a git checkout, and the third is a filesystem walk, plus loading and parsing .go files. All of these are expensive operations. ---------------- For context, in comparison, Cargo has the same #1 problem. But for #2, the information on all dependencies is stored in the index itself; this means that Cargo can figure out what dependencies you need without any network calls at all, let alone downloading, git checkout, and filesystem walk. I do not know how go mod compares, off the top of my head.
    - deleted 7y ago
      [deleted]

sansnomme 7y ago

The trend these days is tiny packages (similar to PaaS turning into Function-as-a-service), very fine grained libraries like Leftpad. The ultimate realisation of HN's favorite mantra "worship the Unix Philosophy". Expect the number of dependencies to only increase.

kortex 7y ago
Except you have things like `sed` and `cut` and `cat`, not `lpd` which solely exists to add leftwise filler characters. Ergonomics of those tools nonwithstanding, they are simple but they aren't TOO simple. Left-pad is IMHO "too simple". Packages on the median should be in a 1-2 pizza team effort with some ongoing maintenance, even if only a few pizzas a year. Left-pad is like, 2 pizzas and a weekend. (were it not for string.prototype methods) a viable package would be something on the order of Python's str.format(). Can left pad and right pad!
- samatman 7y ago
  Unix notoriously has the command `yes`, which provides an infinite loop of `y\n`. Don't have a strong opinion on whether left-pad should exist or not (or indeed, if server-side Javascript should exist or not), but simpler than left-pad, is yes? yes.
  - iudqnolq 7y ago
    Having a basic building block that enables a lot is different from making every tiny problem it's own package. Yes is a exception, certainly, but it doesn't disprove the rule.

earthboundkid 7y ago

I complained about the term “modules” for the bigger code unit in Reddit, but RSC said they were both vague so it didn’t matter. I do still think it should have been “bundles” or something implying “bigger than a package” instead of “module”.