Bundler: Bundler v2.4: new resolver, gems with Rust extensions, and more

Bundler v2.4: new resolver, gems with Rust extensions, and more

by David Rodríguez on Jan 31 2023

2022 has been a busy year for the Bundler team, and we’re glad to present several improvements that we hope will make our users happy :)

A better, PubGrub based, resolver

Bundler now uses the most advanced algorithm to resolve versions, PubGrub. Kudos to Natalie Weizenbaum for inventing it and to John Hawthorn for porting it to Ruby!

Our previous resolver, Molinillo, worked pretty well, but it really got in the middle when it didn’t.

This may sound familiar for some:

$ bundle
Fetching gem metadata from https://rubygems.org/............
Resolving dependencies....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................^C
$ # Ok, that was enough waiting

Our new resolver, PubGrub, is usually much faster, because it learns from the resolution conflicts it finds during the resolution process to avoid redoing the same work over and over again. You can find more about this “conflict-driven clause learning” techniques in its presentation blog post back from 2018.

Molinillo sometimes took too long to resolve because it would try the same things, backtrack, and run into the same conflicts over and over again, having to traverse very inefficiently a huge search space. But that was a relatively rare case in the real world.

What’s probably more common is specifying version requirements in your Gemfile, that can’t be all satisfied at the same time. This is when the version solving problem does not have a solution, and when it becomes crucial to explain to users why, so that they can fix their set of version requirements to become solvable.

Molinillo run into trouble here, and in cases with many moving parts, like upgrading Rails for example, it could end up printing a lot of conflicts, not easy to understand and solve. This is an old example from a public ticket:

Bundler could not find compatible versions for gem "actionpack":
  In Gemfile:
    inherited_resources (= 1.6.0) was resolved to 1.6.0, which depends on
      actionpack (>= 3.2, < 5)

    rails (= 4.2.0) was resolved to 4.2.0, which depends on
      actionpack (= 4.2.0)

Bundler could not find compatible versions for gem "activesupport":
  In Gemfile:
    inherited_resources (= 1.6.0) was resolved to 1.6.0, which depends on
      has_scope (~> 0.6.0.rc) was resolved to 0.6.0, which depends on
        activesupport (>= 3.2, < 5)

    rails (= 4.2.0) was resolved to 4.2.0, which depends on
      activesupport (= 4.2.0)

Bundler could not find compatible versions for gem "railties":
  In Gemfile:
    inherited_resources (= 1.6.0) was resolved to 1.6.0, which depends on
      railties (>= 3.2, < 5)

    rails (= 4.2.0) was resolved to 4.2.0, which depends on
      railties (= 4.2.0)

    inherited_resources (= 1.6.0) was resolved to 1.6.0, which depends on
      responders was resolved to 1.1.2, which depends on
        railties (>= 3.2, < 4.2)

Not easy to know what to do about it.

With PubGrub, you should now get human-readable explanations of failures. The most complex cases may are still, well… complex. But explanations should always make sense and point to the root cause of resolution failures.

Here’s an example from our test suite:

Because every version of c depends on a < 1
  and every version of b depends on a >= 2,
  every version of c is incompatible with b >= 0.
So, because Gemfile depends on b >= 0
  and Gemfile depends on c >= 0,
  version solving has failed.

We tried to make this migration as backwards-compatible as possible, but there’s a chance of experiencing some different solutions to the ones found by Molinillo, since the version solving problem does not have unique solutions. Please report any issues you find with the new resolver.

Easily generate gems with Rust extensions using `bundle gem`

It’s now easier than ever to get started using Rust inside your gems. Check out this blog post to learn how to generate a gem with all the boilerplate necessary with just a few commands.

Faster git sources

In the Bundler world, it’s common to point to git repositories when there’s no version released to rubygems.org that includes the changes that you need. This works fine, but it can get slow and use a lot of disk space when dealing with very big repositories.

We have improved the way we clone these repositories to be faster and use less disk space. For example, something like

gem "rails", github: "rails/rails"

in your Gemfile could previously take ~30s and use up to 1Gb of disk space, because we would clone the full Rails repository, which has a large history.

Now we just clone what’s strictly necessary for Bundler to work, resulting in big disk space savings, and much faster bundling.

New CLI features

We added a few small CLI features, such as a new --pre flag to bundle update and bundle lock to explicitly opt-in to prereleases of selected (of all) gems without having to explictly change your Gemfile with pre-release requirements such as >= 7.1.0.beta.

Some minor breaking changes

We took new year’s release to move on and get rid of some stuff that was causing maintenance burden for us:

Ruby 2.3, 2.4, and 2.5 are no longer supported.
RubyGems 2.5, 2.6, and 2.7 are no longer supported.

In general, this support drop should not break anything because RubyGems should be able to choose the latest supported Bundler on the Ruby version that you’re using. But there are still some old RubyGems out there that don’t have this feature, and the gem install bundler command could break there. We have warned using Bundler on those old rubies for a year now, so we believe it’s time to move on.

We also completely removed a controversial (mis-)feature from the Bundler code base, where Bundler would automatically acquire sudo permissions when not having the proper access rights. A great majority of users considered this feature harmful and hardly useful, so we decided to get rid of it.

And bug fixes

As always, we continue to smooth the experience of using Bundler, so that it gets the job done and does not get in the middle other than that. And we’re also shipping a bunch of bug fixes to keep moving towards that goal.

That’s all from the Bundler team. Have a happy new year, and enjoy using Bundler 2.4! 🎉