Building up our filesystem with debootstrap and QEMU

Debian provides a couple of tools to create new root filesystems. The one we’ll be using here is debootstrap.

The first step is to ensure that our environment has the tools we need, specifically debootstrap and qemu-user-static (more on that later!).

apt-get install -y debootstrap qemu-user-static

This will install both packages without further input from the user, essential if we want the build server to run these commands in an automated fashion.

debootstrap --foreign --verbose --arch=armhf --variant=minbase stretch rootfs

Now that debootstrap is installed, we can call it to create a root filesystem for the given architecture.

The foreign argument tells debootstrap that we need to produce a filesystem for a different architecture than the one we’re running on. This is important as debootstrap will postpone installing packages as we’re not in a position to run armhf code (yet!).

The verbose argument simply tells debootstrap not to be conservative with logging information to the console — this is how we’ll follow along with the execution of the commands on the build server to ensure everything is behaving as we expected.

The arch argument tells debootstrap we want to prepare a filesystem for armhf, but we’ll use variables to switch this out for the other architectures we need later.

The variant argument in effect tells debootstrap how expansive of an operating system we want to produce in the new filesystem. The minbase option installs the least amount of packages necessary to produce a widely useful operating system — which is what we want in our case, as if this is for use in arbitrary builds we won’t know the dependencies ahead of time.

Finally we indicate which Debian release we want to build a filesystem for. Stretch is (at time of writing) the current stable release, so we’ll go with that, and we specify which subdirectory the new filesystem should be created in (rootfs).

We’ve now hit the first speedbump. We need to be able to emulate our new filesystem, but the execution of the packages inside the filesystem depend on virtual devices that the filesystem won’t have access to. We need to take the virtual devices from our host and bind them to the new filesystem so it can execute in the context of an operating system.

cd rootfsmount --bind /dev dev/mount --bind /sys sys/mount --bind /proc proc/mount --bind /dev/pts dev/pts/cd ..

We enter the filesystem, and mount the virtual devices in our host (/dev, /sys, /proc and /dev/pts) into the same locations within the filesystem — making them available for later.

cp /usr/bin/qemu-arm-static rootfs/usr/bin/chmod +x rootfs/usr/bin/qemu-arm-static

In order for us to execute the compiled armhf code in our file system later in the process, we need to use the QEMU emulator we installed into /usr/bin. Unfortunately, once we change our root directory into the new filesystem, we’ll be pointing at the /usr/bin of the new filesystem, which does not contain the emulator. So, we need to copy it into place first — and use chmod +x to make it executable.

chroot rootfs /debootstrap/debootstrap --second-stage --verbose

It’s at this point we’ll enter the filesystem to complete the debootstrap process. The chroot command will change our root directory to the rootfs sub-directory we created with debootstrap earlier.

The verbose argument is given again to ensure we get as much information as possible from our logs.

The second-stage argument tells debootstrap that we’re now running on the target architecture. We’re not, of course, but thanks to QEMU debootstrap will happily execute as if we were to complete the setup of the filesystem.

rm rootfs/usr/bin/qemu-arm-static

Just for housekeeping sake, we’ll remove QEMU from the root filesystem when we’re done, so that it’s effectively in the same state it would be if we were working on physical armhf hardware. This is quite important from an emulation standpoint, as leaving the emulator in place could confuse matters later with regards to testing for this architecture.

cd rootfs;tar -zcvf $OUTPUT_DIR/prebootstrap_stretch_minbase_armhf_rootfs.tar.gz *;

Now that the root filesystem is built, all we need to do is compress it into a tarball and send it to our output directory, as denoted by the $OUTPUT_DIR environment variable.

In the next article in this series I’ll discuss taking these scripts and setting up continuous delivery in the cloud, so that the filesystems stay up-to-date automatically.