3 ms·

People here complaining about the size of the node modules folder - this is one of the reasons people use other solutions like perforce. Checking a 2GB folder i

People here complaining about the size of the node modules folder - this is one of the reasons people use other solutions like perforce. Checking a 2GB folder into p4 is an absolute no brainer, and for all the flexibility people talk about with git, it's inability to handle this is pretty damning after so many years.

sktrdie 5y ago
Can you expand on this? Why is the size smaller on p4 compared to git?
- masklinn 5y ago
  P4 does not store the entire history locally, so your only limit is the amount of space on the server. Git will pack objects after a while (initially they’re stored as-is, just compressed), but large files or enormous amounts of changes can make the repository grow to unwieldy amounts, and while git is able to perform “shallow” clones (clones which only store part of the history), not everything handles them well
  - oftenwrong 5y ago
    Git also has "partial" clones, which can avoid some of the downsides of shallow clones. You can, for example, filter out all historical objects (--filter=blob:none). Historical objects are fetched lazily as needed, such as checking out an old revision. The same can be done with unreachable trees. This article has excellent diagrams that depict the different types of clones: https://github.blog/2020-12-21-get-up-to-speed-with-partial-clone-and-shallow-clone/ https://github.blog/2020-12-21-get-up-to-speed-with-partial-...
- maccard 5y ago
  As the other commenter replied p4 doesn't download the entire history locally, so storage is only a concern on the server. Depending on what you're storing in p4, you can use p4 archive and/or p4 obliterate to manage the actual storage used on the server. E.g. you might use p4 obliterate to only keep the latest version of your node modules folder in p4 if you're trying to optimize for CI performance, or if you're using p4 for build artifacts, you might use p4 archive and store the older revsions on slower bulk storage.

sktrdie 5y ago

Can you expand on this? Why is the size smaller on p4 compared to git?

masklinn 5y ago
P4 does not store the entire history locally, so your only limit is the amount of space on the server. Git will pack objects after a while (initially they’re stored as-is, just compressed), but large files or enormous amounts of changes can make the repository grow to unwieldy amounts, and while git is able to perform “shallow” clones (clones which only store part of the history), not everything handles them well
- oftenwrong 5y ago
  Git also has "partial" clones, which can avoid some of the downsides of shallow clones. You can, for example, filter out all historical objects (--filter=blob:none). Historical objects are fetched lazily as needed, such as checking out an old revision. The same can be done with unreachable trees. This article has excellent diagrams that depict the different types of clones: https://github.blog/2020-12-21-get-up-to-speed-with-partial-clone-and-shallow-clone/ https://github.blog/2020-12-21-get-up-to-speed-with-partial-...
maccard 5y ago
As the other commenter replied p4 doesn't download the entire history locally, so storage is only a concern on the server. Depending on what you're storing in p4, you can use p4 archive and/or p4 obliterate to manage the actual storage used on the server. E.g. you might use p4 obliterate to only keep the latest version of your node modules folder in p4 if you're trying to optimize for CI performance, or if you're using p4 for build artifacts, you might use p4 archive and store the older revsions on slower bulk storage.