Thursday 8 August 2013

speeding up package builds

I had to look at many "core" packages, which are needed for a normal and a
buildd chroot, and and which are needed to build these packages for the chroots.
And I was not that pleased to see how the packaging wasted CPU time. I think
even for our migration from proposed to release it would make sense to speed up
some builds, so that the slower architectures can catch up earlier. So here is
what I found ...

Parallel builds

Unfortunately not enabled very often. The buildds set
DEB_BUILD_OPTIONS="parallel=<N>", and if the build supports parallel builds, it
can speed up the build substantially. Common mistakes:

- debhelper based packages don't call with --parallel
- debhelper based packages overwrite dh_auto_build and don't call
--parallel
- cdbs based packages don't set DEB_BUILD_PARALLEL = yes
- Packages deep down in the stack live still in the stone age (bind9,
db, ghostscript, ...)


Documentation builds

Many packages always build the documentation, even when only building the
architecture dependent packages (that is, on our slowest buildds). The easy
solution where the upstream docs are not built by default is to build these in
an extra target.

However I didn't find any good recipes how to write a modern debhelper or
semi-modern cdbs rules file how to correctly configure a package to disable the
docs build for an arch-only build, and enable the docs for an arch-indep build.
Pointers welcome.


Flavour builds

There is no benefit for our proposed to release migration, however when testing
a package, or building a package for the first time, I'm maybe only interested
in one flavour. Disabling one flavour reduces the build time. Mostly seen for
packages using different builds for building udeb's. It would be nice to
disable these if DEB_STAGE=<something> is set.


Shell-script like rules files

Using a short "modern" debhelper rules file doesn't mean that you cannot define
your own targets to break a configure or build target in many sub targets. It
doesn't speed up the build itself, however if you debug an issue, and the build
starts again ... something could be done better. Please try to at least
separate the configure and build steps. Look at boost1.xy to see what I don't like.


As an example, gtk+3.0 did become eight times faster by enabling parallel builds
(parallel=8), disabling the doc build, and the udeb flavour. Just picking this
up, because I had to build it several times.

Hope this helps a bit in making package builds and migrations a bit faster.

Matthias

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel