7 ms·
I do wish more attention was paid to the ballooning disk space requirements for both. I think Cargo is doing better, but I've yet to get a shared directory wor
by smeagull 4y ago
I do wish more attention was paid to the ballooning disk space requirements for both.
I think Cargo is doing better, but I've yet to get a shared directory working for every build on a system. I shouldn't have to spend 2G per Rust project.
- beebmam 4y agoIt's a lot more than that if you're doing cross-platform (many) builds. It's a real pain.
- jcelerier 4y agoSame with vcpkg for c++ development... Many of my students still have laptops with 128G of storage total, good luck making a whole $world fit in this when c:\Windows already uses half of it and they have their own documents & stuff
- kelnos 4y agoYeah it seems super weird to me that cargo doesn't cache built artifacts based on crate name + version + enabled features + rustc version (and anything else that might make it unique). It seems silly that I have many built copies of the same dependency scattered across different projects on my machine. Then again, maybe it's actually fairly rare that the things that make a crate build unique are all the same all that often, and the gains vs. complexities of caching just aren't worth it.
- jackosdev 4y agosccache works really well and there’s only two steps to install it and enable it globally, speeds up compilation time a lot as well: https://github.com/mozilla/sccache https://github.com/mozilla/sccache
- LoganDark 4y ago> I've yet to get a shared directory working for every build on a system Same here. CARGO_TARGET_DIR exists, but that means every time I change a feature in some crate and rebuild, suddenly it doesn't match the version that all my other projects want, and I need to rebuild again when I return to those. But it is the closest thing we have.
- dirkg 4y agoNodejs solved all this years ago, now we have super fast shared package managers like pnpm, yarn etc, not to mention deno etc, why can't other languages adopt the same system?
- Yoric 4y agoI assume that it is harder for Rust, because it needs to deal with `features`, with cross-compilation, with the developer using `nightly` for some projects, `stable` for others, pinned versions for yet others, etc. That being said, harder doesn't mean unsolvable.
- pjmlp 4y agoMaven and CPAN did it first, and in many ways better than node.
- pxc 4y agoNode supports circular dependencies and arbitrary pre/post-install scripts, which are both disasters for downstream packaging. Node is not a model here. The CAS that gets symlinked into vendorized directories à la pnpm is the Nix model, which is indeed an improvement that other package managers which support vendorization should incorporate.
- rendaw 4y agoIf it's shared, you don't know what project pulled in which dependencies so you can't clean it up. I guess this is a trade off - 2G that's easy to clean up vs ?G that isn't. The only way around would be some sort of pinning and GC like in Nix. That said, if it's not an option, I think it'd be reasonable for people to be able to make that choice themselves...
- Yoric 4y agoCould be symlinked (or hardlinked?) from a central cache, though.
- anewhnaccount2 4y agoSee https://github.com/pauldreik/rdfind https://github.com/pauldreik/rdfind
- Yoric 4y agoWhich part? The caveat?
- anewhnaccount2 4y agoI use rdfind to deal with this: https://github.com/pauldreik/rdfind https://github.com/pauldreik/rdfind
- vlovich123 4y agoThat tool looks fairly old considering it’s talking about sorting inodes and benchmarking on spinning disk. Is there any tool that replaces with a COW copy instead of creating a hard link? COW is strictly a safer way to do this compression and honestly it seems like something filesystems should do automatically, perhaps with FastCDC deduplication.
- anewhnaccount2 4y agoYes, that would be safer when available (although generally files within library dependencies are not modified I think?). It looks like fclones implements this, is faster and is written in Rust https://github.com/pkolaczk/fclones https://github.com/pkolaczk/fclones (the last is the most important point of course /s).
- vlovich123 4y agoFantastic. Now that’s a tool I’d run as it seems actually safe. rdfind doesn’t limit itself to library dependencies so I’d be worried about it fucking up on files I’d rather not have be hardlinked just because a copy exists (eg 2 checked out git clones where modifying the code in one suddenly starts modifying files in the other)
- oblio 4y agoIt's absolutely ridiculous how little these package manager devs looked at Maven. It predates almost all of them by at least half a decade, has a lot of hard thought knowledge but "Java is icky, Java is enterprise, bleh" and I can't even count how many mistakes Maven fixed back in 2005 I still see in pip, npm, cargo, etc in 2023.
- smeagull 4y agoMy only experience of Maven was fighting it to get a single jar that included everything, and having it redownload every dependence with every single build. I don't think downloading everything is a better solution.
- gladiatr72 4y agoPerhaps the privilege of getting sued by Oracle has something to do with it. I seem to remember Sun (then Oracle) blowing a fair bit of money on lawsuits surrounding Java.
- oblio 4y agoI really doubt that a project by the Apache Software Foundation scared anyone of anything. It would have taken these package manager devs a few days of gratis and lawsuit free research on their laptops to save millions of end users years of pain.