11 ms·

Can Bundler be as fast as uv?

• dboreham 10mo ago
  I've been doing software of all kinds for a long long time. I've never, ever, been in a position where I was concerned about the speed of my package manager. Compiler, yes. Linker, sure. Package downloader. No.
  • travisd 10mo ago
    Many of these package managers get invoked countless times per day (e.g., in CI to prepare an environment and run tests, while spinning up new dev/AI agent environments, etc).
    • byroot 10mo ago
      Ye,s but if your CI isn't terrible, you have the dependencies cached, so that subsequent runs are almost instant, and more importantly, you don't have a hard dependency on a third party service. The reason for speeding up bundler isn't CI, it's newcomer experience. `bundle install` is the overwhelming majority of the duration of `rails new`.
      • maccard 10mo ago
        > Ye,s but if your CI isn't terrible, you have the dependencies cached, so that subsequent runs are almost instant, and more importantly, you don't have a hard dependency on a third party service. I’d wager the majority of CI usage fits your bill of “terrible”. No provider provides OOTB caching in my experience, and I’ve worked with multiple in house providers, Jenkins, teamcity, GHA, buildkite.
        • byroot 10mo ago
          GHA with the `setup-ruby` action will cache gems. Buildkite can be used in tons of different ways, but it's common to use it with docker and build a docker image with a layer dedicated to the gems (e.g. COPY Gemfile Gemfile.lock; RUN bundle install), effectively caching dependencies.
          • maccard 10mo ago
            > GHA with the `setup-ruby` action will cache gems. Caching is a great word - it only means what we want it to mean. My experience with GHA default caches is that it’s absolutely dog slow. > Buildkite can be used in tons of different ways, but it's common to use it with docker and build a docker image with a layer dedicated to the gems (e.g. COPY Gemfile Gemfile.lock; RUN bundle install), effectively caching dependencies. The only way docker caching works is if you have a persistent host. That’s certainly not most setups. It can be done, but if you have that running in docker doesn’t gain you much at all you’d see the same caching speed up if you just ran it on the host machine directly.
            • byroot 10mo ago
              > My experience with GHA default caches is that it’s absolutely dog slow. GHA is definitely far from the best, but it works:, e.g 1.4 seconds to restore 27 dependencies https://github.com/redis-rb/redis-client/actions/runs/20519158916/job/58951585454#step:3:123 https://github.com/redis-rb/redis-client/actions/runs/205191... > The only way docker caching works is if you have a persistent host. You can pull the cache when the build host spawns, but yes, if you want to build efficiently, you can't use ephemeral builders. But overall that discussion isn't very interesting because Buildkite is more a kit to build a CI than a CI, so it's on you to figure out caching. So I'll just reiterate my main point: a CI system must provide a workable caching mechanism if it want to be both snappy and reliable. I've worked for over a decade on one of the biggest Rails application in existence, and restoring the 800ish gems from cache was a matter of a handful of seconds. And when rubygems.org had to yank a critical gem for copyright reasons [0], we continued building and shipping without disruption while other companies with bad CIs were all sitting ducks for multiple days. [0] https://github.com/rails/marcel/issues/23 https://github.com/rails/marcel/issues/23
              • maccard 10mo ago
                > So I'll just reiterate my main point: a CI system must provide a workable caching mechanism if it want to be both snappy and reliable. The problem is that none of the providers really do this out of the box. GHA kind of does it, but unless you run the runners yourself you’re still pulling it from somewhere remotely. > I've worked for over a decade on one of the biggest Rails application in existence, and restoring the 800ish gems from cache was a matter of a handful of seconds. I kind of suspected - the vast majority of orgs don’t have a team of people who can run this kind of a system. Most places with 10-20 devs (which was roughly the size of the team that ran the builds at our last org) have some sort of script, running on cheap as hell runners and they’re not running mirrors and baking base images on dependency changes.
    • zingar 10mo ago
      Is the package manager a significant amount of time compared to setting up containers, running tests etc? (Genuine question, I’m on holiday and can’t look up real stats for myself right now)
      • maxbond 10mo ago
        Anecdotally unless I'm doing something really dumb in my Dockerfile (recently I found a recursive `chown` that was taking 20m+ to finish, grr) installing dependencies is longest step of the build. It's also the most failure prone (due to transient network issues).
  • m00x 10mo ago
    When you're in a big company, the problem really starts showing. A service can have 100+ dependencies, many in private repos, and once you start adding and modifying dependencies, it has to figure out how to version it to create the lock file across all of these and it can be really slow. Cloud dev environments can also take several minutes to set up.
  • deleted 10mo ago
    [deleted]
  • maccard 10mo ago
    There is no situation where toolchain improvements or workflow improvements should be snuffed at.
    • mikepurvis 10mo ago
      It can be harder to justify in private tooling where you might only have a few dozen or hundred devs saving those seconds per each invocation. But in public tooling, where the benefit is across tens of thousands or more? It's basically always worth it.
      • maccard 10mo ago
        Obviously effort vs reward comes in here, but if you have 20 devs and you save 5 seconds per run, you save a context switch on every tool invocation possibly.
        • mikepurvis 10mo ago
          This is true, but I think the other side of it is that in most shops there is lower hanging fruit than 5 seconds per tool run, especially if it's not the tool that's in the build / debug / test loop but rather the workspace setup / packaging / lockfile management tool. Like, I switched my team's docker builds to Depot and we immediately halved our CI costs and shed like 60% of the build time because it's a persistent worker node that doesn't have to re-download everything every time. I have no association with them, just a happy customer; I'm only giving it to illustrate how many more gains are typically on the table before a few seconds here and there are the next thing to seriously put effort into.
  • IshKebab 10mo ago
    Must be nice not to use Python!
  • skinnymuch 10mo ago
    I agree for my own projects where the code and libraries are not enormous. The speed and size gains aren’t going to be something that matters enough.
  • blibble 10mo ago
    try conda took an hour to install 30 dependencies
• nightpool 10mo ago
  Really interesting post, but this part from the beginning stuck out to me: Ruby Gems are tar files, and one of the files in the tar file is a YAML representation of the GemSpec. This YAML file declares all dependencies for the Gem, so RubyGems can know, without evaling anything, what dependencies it needs to install before it can install any particular Gem. Additionally, RubyGems.org provides an API for asking about dependency information, which is actually the normal way of getting dependency info (again, no eval required). It would be interesting to compare and contrast the parsing speed for a large representative set of Python dependencies compared to a large representative set of Ruby dependencies. YAML is famously not the most efficient format to parse. We might have been better than `pip`, but I would be surprised if there isn't any room left on the table to parse dependency information in a more efficient format (JSON, protobufs, whatever). That said, the points at the end about not needing to parse gemspecs to install "most" dependencies would make this pretty moot (if the information is already returned from the gemserver)
  • masklinn 10mo ago
    Although Yaml is a dreadful thing, given the context and the size of a normal gemspec I would be very surprised if it showed up in any significant capacity when psych should be in the low single digit MB/s throughput.
  • tedivm 10mo ago
    It mostly doesn't matter, because these metadata files are pulled into their respective package managers. When you publish to RubyGems the file is read into their database and made available to their API, just like when you publish a Python file the pyproject.toml is parse into the PyPI database and made available. This is a major reason why UV is faster than older python package managers, as they were able to take advantage of the change in the PyPI registry that enabled this. Now these package managers can run their dependency calculations without needing to download the entire package, decompress the package files, and then parse them.
    • nightpool 10mo ago
      Ah, I had misremembered the article as saying that the API was sufficient for "most" tasks, all it actually says is that it's the "normal" way of getting the dependency info. I was wondering if there was some less common operations (e.g. upgrading?) that needed more info beyond what was in the API.
• dang 10mo ago
  Recent and related: How uv got so fast - https://news.ycombinator.com/item?id=46393992 https://news.ycombinator.com/item?id=46393992 - Dec 2025 (457 comments)
• dbalatero 10mo ago
  I appreciate that Aaron is focusing on the practical algorithm/design improvements that could be made to Bundler, vs. prematurely going all in on "rewrite in Rust".
  • AlphaSite 10mo ago
    Speed would be nice, but more than that I want it to also manage Ruby installs. I’m infuriated at the mess of Rubys and version managers.
    • stouset 10mo ago
      Mise is the answer to this. I no longer use chruby/rbenv/rvm. And it manages multiple languages, project-local environment, etc.
      • NSPG911 10mo ago
        mise is pretty nice, though I don't use it for python, Python is handled by uv with poethepoet as the task runner.
        • dirtbag__dad 10mo ago
          Can you help me understand what the value or use case of poethepoet is?
          • jacobtomlinson 10mo ago
            It allows you to define common tasks such as linting, running tests, building docs, etc under an alias. So you can run uv run poe docs Instead of uv run sphinx-build -W -b dirhtml docs/source docs/build Many languages have a task runners baked into their package manager, but many others don’t. In Ruby it’s roughly the equivalent of Rake.
            • stouset 10mo ago
• quotemstr 10mo ago
  Well, now my opinion of uv has been damaged. It... > Ignoring requires-python upper bounds. When a package says it requires python<4.0, uv ignores the upper bound and only checks the lower. This reduces resolver backtracking dramatically since upper bounds are almost always wrong. Packages declare python<4.0 because they haven’t tested on Python 4, not because they’ll actually break. The constraint is defensive, not predictive Man, it's easy to be fast when you're wrong. But of course it is fast because Rust not because it just skips the hard parts of dependency constraint solving and hopes people don't notice. > When multiple package indexes are configured, pip checks all of them. uv picks from the first index that has the package, stopping there. This prevents dependency confusion attacks and avoids extra network requests. Ambiguity detection is important. > uv ignores pip’s configuration files entirely. No parsing, no environment variable lookups, no inheritance from system-wide and per-user locations. Stuff like this sense unlikely to contribute to overall runtime, but it does decrease flexibility. > No bytecode compilation by default. pip compiles .py files to .pyc during installation. uv skips this step, shaving time off every install. ... thus shifting the bytecode compilation burden to first startup after install. You're still paying for the bytecode compilation (and it's serialized, so you're actually spending more time), but you don't associate the time with your package manager. I mean, sure, avoiding tons of Python subprocesses helps, but in our bold new free threaded world, we don't have to spawn so many subprocesses.
  • IshKebab 10mo ago
    > Man, it's easy to be fast when you're wrong. There's never going to be a Python 4 so I don't think they are wrong. Even if lighting strikes thrice there's no way they could migrate people to Python 4 before uv could be updated to "fix" that. > Ambiguity detection is important. I'm not sure what you mean here. Pip doesn't detect any ambiguities. In fact Pip's behaviour is a gaping security hole that they've refused to fix, and as far as I know the only way to avoid it is to use `uv` (or register all of your internal company package names on PyPI which nobody wants to do). > thus shifting the bytecode compilation burden to first startup after install Which is a much better option.
    • jjgreen 10mo ago
• kimos 10mo ago
  I’ve been squinting at the “global cache for all bundler instances” issue[1] and I’m trying to figure out if it’s a minefield of hidden complication or if it’s actually relatively straight forward. It’s interesting as a target because it pays off more the longer it has been implemented as it only would be shared from versions going forward. [1] https://github.com/ruby/rubygems/issues/7249 https://github.com/ruby/rubygems/issues/7249
  • adithyareddy 10mo ago
    It's definitely not super straightforward, but there's plenty of recent prior art to steal from. Ruby was probably not the best place to solve this for the first time given the constraints (similar to pip), but there's no reason the Ruby ecosystem shouldn't now benefit from the work other ecosystems have done to solve it.
• amazingman 10mo ago
  If any Bundler replacement isn't using Minimal Version Selection, I'm not interested.
• raggi 10mo ago
  It’s definitely possible, I wrote a prototype many many years ago https://ra66i.org/tmp/how_gems_should_be.mov https://ra66i.org/tmp/how_gems_should_be.mov
• prescriptivist 10mo ago
  I think fibers (or, rather, the Async library) in Ruby tends to be fetishized by junior Rails engineers who don't realize higher level thread coordination issues (connection pools, etc) equally apply to fibers. That said this could be a pretty good use case for fibers -- the code base I use every day has ~230 gems and if you can peel off the actual IO bound installation of all those into non-blocking calls, you would see a meaningful performance difference vs spinning up threads and context switching between them.
  • raggi 10mo ago
    What I would do to really squeeze the rest out in pure ruby (bear in mind I’ve been away about a decade so there _might be_ new bits but nothing meaningful as far as I know): Use a cheaper to parse index format (the gists I wrote years ago cover this: https://gist.github.com/raggi/4957402 https://gist.github.com/raggi/4957402) Use threads for the initial archive downloads (this is just io, and you want to reuse some caches like the index) Use a few forks for the unpacking and post install steps (because these have unpredictable concurrency behaviors)
    • prescriptivist 10mo ago
      > there _might be_ new bits but nothing meaningful as far as I know If you didn't need backwards compatibility with older rubies you could use Ractors in lieu of forks and not have to IPC between the two and have cleaner communication channels. I can peg all the cores on my machine with a simple Ractor pool doing simple computation, which feels like a miracle as a Ruby old head. Bundler could get away with creating their own Ractor safe installer pool which would be cool as it'd be the first large scale use of Ractors that I know of.
• mberning 10mo ago
  I never found Bundler to be all that slow compared to other package managers.
  • faizshah 10mo ago
    Rails 8.1 and ruby 3 are also very surprisingly fast, and coming back to an “omakase” framework is honestly a breath of fresh air especially now that with AI tools you can implement a lot of stuff from scratch instead of using deps.
  • plorkyeran 10mo ago
    I wouldn't say it stands out as being unusually slow, but it certainly isn't snappy.
• lamontcg 10mo ago
  The biggest thing that gems could do to make rubygems faster is to have a registry/database of files for each gem, so that rubygems didn't have to scan the filesystem on every `require` looking for which gem had which file in it. That would mean that if you edited your gems directly, things would break. Add a file, and it wouldn't get found until the metadata got rehashed. The gem install, uninstall, etc commands would need to be modified to maintain that metadata. But really, you shouldn't be hacking up your gem library like that ith shellcommands anyway (and if you are doing manual surgery, having to regen the metadata isn't really that burdensome).
  • pmahoney 10mo ago
    I wrote some code to do almost this many years ago (if I recall correctly, it doesn’t cache anything to disk, but builds the hash fresh each time, which can still result in massive speed up). Probably obsolete and broken by now, but one of my favorite mini projects. (And I just realized the graph is all but impossible to read in dark mode) https://github.com/pmahoney/fastup https://github.com/pmahoney/fastup
  • vidarh 10mo ago
    100%. Optimising "bundle install" etc. is optimizing the wrong thing. You don't even need this to work from gems in general. It'd have solved a lot of problems just to have it work for "bundle install" in standalone mode, where all the files are installed to a directory anyway. But in general, one of the biggest problems with Ruby for me is how $LOAD_PATH causes combinatoric explosion when you add gems because every gem is added, due to the lack of any scoping of require's to packages. The existence of multiple projects to cache this is an illustration that this is a real issue. I've had project in the past where starting the app took minutes purely due to require's, and where we shaved minutes off by crude manipulation of the load path, as most of that time was pointless stat calls.
    • lamontcg 10mo ago
      Yeah, chef-client got a lot faster, particularly on Windows, just by liberally using `require_relative` whenever possible and sprinkling `require "Foo" unless defined?(FOO)` all over the codebase.
• dmix 10mo ago
  tenderlove never misses
• rossjudson 10mo ago
  Optimization: Doing nothing is faster than doing something.
  • mkoubaa 10mo ago
    Do smart thing make code go brr? No. Stop do dumb thing make code go brr.
• franciscop 10mo ago
  > PEP 658 (2022) put package metadata directly in the Simple Repository API, so resolvers could fetch dependency information without downloading wheels at all. > Fortunately RubyGems.org already provides the same information about gems. > [...] > After we unpack the gem, we can discover whether the gem is a native extension or not. Why not adding the meta information of whether the gem is a native extension or not directly to rubygems.org? You could fully parallelize whole installation trees of dependencies then.
  • olivierlacan 10mo ago
    Had the same thought reading this but I suspect what's in the gemspec could accidentally differ from what's in the RubyGems.org metadata, although that should probably not be possible. From working on RubyGems.org a long time ago I vaguely remember that the metadata extracted from the gemspec is version-specific. So if you add a new native_extension boolean you'd have to artificially reprocess those previously published gemspecs to change the metadata for all past versions. Being able to mutate metadata for past versions is dangerous enough that I'd be surprised it's allowed or even possible. So that might not even be something Aaron considered here for that reason. That said, it seems reasonable to me to suggest this improvement going forward to make unpacking the gem unnecessary to know whether it'll affect installation order.
    • derefr 10mo ago
      Just make the rule apply only to packages published after a given date, and then manually backfill that metadata into the service-backend DB with a one-time scrape through all packages from before that date.
• charliermarsh 10mo ago
  I'm not super familiar with Bundler's architecture but I think the most impactful thing would be adopting uv's cache design, which is a big part of what makes uv so fast and should be replicable in other languages and ecosystems. > Ignoring requires-python upper bounds. When a package says it requires python<4.0, uv ignores the upper bound and only checks the lower. This reduces resolver backtracking dramatically since upper bounds are almost always wrong. I don't think that ignoring upper bounds has a significant impact on uv's performance. We do this for a totally different reason, which is that it leads to better solves. For example, if you say your project requires Python 3.8 or later, but some dependency said it works for ">=3.8,<4", then suddenly your project isn't installable on Python 4, and you'd be implicitly required to put a "<4" bound on your own project. uv solves for all of your supported Python versions, not a single version, so discounting the upper bounds doesn't actually save us any time in the solve. (See, e.g. https://discuss.python.org/t/requires-python-upper-limits/12663 https://discuss.python.org/t/requires-python-upper-limits/12....)