A tale of three Debian build tools

Many people have asked me about my Debian workflow. Which is funny, because it's hard to believe that when you use three different build tools that you're doing it right, but I have figured out a process that works for me. I use git-buildpackage (gbp), sbuild, and pbuilder, each for different purposes. Let me describe why and how I use each, and the possible downsides of each tool.

Note: This blog post is aimed at people already familiar with Debian packaging, particularly using Vcs-Git. If you'd like to learn more about the basics of Debian packaging, I recommend you check out my Clojure Packaging Tutorial and my talk about packaging Leiningen.

git-buildpackage: the workhorse

I use git-buildpackage (aka gbp) to do the majority of my package builds. Because it runs by default without any special sandboxing from your base system, gbp builds things fast—much faster than other build tools—and as such, it's a great tool if you are iterating quickly to work bugs out of a build.

I usually invoke gbp like this:

gbp buildpackage -uc -us

The flags ensure that 1) we don't sign the .changes file generated (-uc) 2) we don't sign the .dsc file (-us), since typically I only want to sign build artifacts right before upload.

Other handy flags include

Typically, if I use gbp to build binary packages, I will only do so in the confines of an updated, minimal sid LXC container, in order to reduce the risk of contaminating my build with stuff that's not available in a clean build environment.

gbp tastes great with sadt, a script in devscripts that runs your package's autopkgtests directly on your base system without root. You will need to install your test package and any dependencies before you can run sadt, but it requires much less setup and infrastructure than autopkgtest does.

***

So while gbp is awesome for the majority of my development workflow, I don't rely on the output of gbp when I want to upload a package, however clean I think my build LXC is. For uploads, I still use gbp, but exclusively to perform source-only (-S) builds:

gbp buildpackage -S -uc -us

Sometimes I may also need to pass the -d flag when building source packages to ignore installed build dependencies. By default, gbp won't proceed with a build when build dependencies are not installed, but when a said dependency is only needed for building the binary (and not the source) package, we can safely override this check.

Typically, when I'm uploading an update to a package, I'll just build the source package with gbp as above, sign the .changes and .dsc files, and complete a source-only upload so the buildds can handle all the build details.

sbuild: the gating build

"But wait!" you undoubtedly object. "How do you know your source package isn't full of garbage?!" That's a great question, because... I don't. So it behooves me to test it. And since the buildds use sbuild, why not use that for my own QA?

Setting up sbuild is a giant pain. You need a machine you have root access on, which wasn't the case for git-buildpackage.

To use sbuild without sudo, you'll need to add your user to the sbuild group:

sudo sbuild-adduser myusername

And create a schroot for builds:

sudo sbuild-createchroot unstable /srv/chroot/unstable-amd64-sbuild http://deb.debian.org/debian

If you ever need to update your sbuild schroot (you will), you can run:

sudo sbuild-update -udcar unstable-amd64-sbuild

I remember these flags by pronouncing them as one word, "ud-car", which sounds absurd and is hence memorable. I can never remember the name of my schroot, but I can look that up by running ls /srv/chroot.

Okay, time to build. Once we have our source package from gbp, we should have a .dsc file to pass to sbuild. We can build like so:

sbuild mypackage_1.0-1.dsc -d unstable -A --run-autopkgtest

We specify the target distribution with -d unstable, and -A ensures that our "arch: all" package will actually build (not necessary if your package targets specific architectures). To run the autopkgtests after a successful build, we pass --run-autopkgtest.

I don't like typing very much so I stick all these parameters in an .sbuildrc in my home directory. You can reference or copy the one in /usr/share/doc/sbuild/examples/example.sbuildrc because it's a Perl script that ends in 1;.

I add

$distribution = 'unstable';
$build_arch_all = 1;

# Optionally enable running autopkgtests
# $run_autopkgtest = 1;

so I don't have to type these in all the time. I don't enable autopkgtests by default because they prompt for a sudo password midway through the build (but perhaps that won't bother you, so feel free to uncomment those lines). Once we have our ~/.sbuildrc created, then we can just run

sbuild mypackage_1.0-1.dsc

Much better!

After my package successfully builds and tests, I take a quick look at the changes, build environment, and package contents. sbuild automatically prints these out, which is very convenient. If everything looks okay, I will sign the source package with debsign and upload it with dput (from dput-ng).

pbuilder: I need to upload binaries!

One irritation I have with sbuild is I can never figure out the right flags to get the right build artifacts to do a binary upload. Its defaults are too minimal for sending to NEW without some additional fancy incantations (it doesn't include the package tarball, only the buildinfo and produced .deb), and I have a hard enough time remembering the flags that I listed above. Remember, this is what the manpage for sbuild looks like:

MANPAGE OF THE DAY: sbuild https://t.co/e7nwB5PUUZ

...this was a little traumatizing tbh pic.twitter.com/zSSEbe4ROH

— e. hashman (@ehashdn) December 24, 2017

So when I have to upload a NEW package, I usually use pbuilder.

There are two ways to invoke pbuilder. The first is the easiest, but "not recommended" for uploads by the manpage: simply navigate to the root of the repository you want to build and run pdebuild.

pdebuild

Wow, that's the simplest thing we've run yet! Why don't we run it all the time?! Well, because of the way pbuilder sets up its filesystem, it can be slower than sbuild, so I've moved it out of my development workflow. It also requires my sudo password, and as I mentioned earlier, I don't particularly like having to enter that mid-build.

The second way I usually apply when I need to upload a build: invoking pbuilder directly. Like with sbuild, we need to provide it a .dsc, so we should build a source package first. However, pbuilder is smarter than sbuild and doesn't need me to give it architectures and target distros and whatnot, so there is significantly less headache if I haven't tweaked a personal configuration. With pbuilder, I can run

sudo pbuilder build mypackage_1.0-1.dsc

and I get a package!

One of the annoying things about pbuilder is that it doesn't output files in my current build directory. Instead, by default, it places build artifacts inside /var/cache/pbuilder/result. So I always have to remember to copy things out of there before I upload.

Also, pbuilder doesn't print out some build information that I should check over before uploading, so I have to do that manually with dpkg:

# Print out package information, including dependencies
dpkg -I libmypackage_1.0-1_all.deb

# List all contents of the package
dpkg -c libmypackage_1.0-1_all.deb

Now I can go ahead and sign my built .changes file and perform an upload!

In summary

Here are what I'd say the pros and cons of each of these three build tools I run are.

git-buildpackage

Pros:

Cons:

sbuild

Pros:

Cons:

pbuilder

Pros:

Cons:

cowbuilder, qemubuilder, whalebuilder, mylittlepersonalbuilder, etc.

Pros:

Cons:

Hope you enjoyed the tour. Happy building!

References