Friday 21 June 2013

Speeding up sbuild / PPA / distro builds (optiPNG)

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with undefined - http://www.enigmail.net/

iQIcBAEBAgAGBQJRxMQnAAoJEGnv7NPGHSZF/J8QALbuihHnt4hDKvH85cUdhF44
SFgI+UIwBklll+/SVcA6tccT4aud9/N8GBOLsui0A65YzZ3XueJsF2XT2AjG5ywy
7OXlrYWfPoCxo8ebflQ0sBAZapmBWJRZV3RChszsXlNJoUPoujBorO2ysj19Ik5Q
LF5+UMDRQi5fgx2+Ypxhl+o8LNmU3E9mzuYbjULnLr+IlEKmvxDf14U5gcI9uqJi
BfdG1jwbSf2Bbw9Z3btvtUTK1MP4TYOKPF8TimLJwzkbFRYupNtfGxp0YsSskIzJ
Myf6Ynz1Ze/7UgpbWXhvXTiWZk4OhqQGPBoLfvA3ZnJHemEOFmSTzhNmH+EuPUvd
Ww+7cjKTj9szYpQZM5uRDt/L7yHuhhqbfUgGz8e5hhdfJvSLXaqCcZyNjDHw6Ij0
aWTwdKbvHAkJ/LG1T/7JaXkwSxal7VJdx+/+s+1ku9SwU9QYLjAL+kymai8VhWmo
EqBjlQDCJR+KirKs/2hVvYUHAsmLxz7GBdRppbJmvnCQZnPEzHfKxGoKKmdEvou8
addSDMFJNLHQvzN2kMDXR+FqZ9bKwGIqp1OIfRi471sg58xkJqN1Vr9Cc4jhZzpf
oI8hy2Qa7/9cXhh/tTaaATvGNq00HqzYBELsNy67uGrBKtL/T0Mytbk5l4ZT2x0A
7GlGeZ+OlNWt+nq1lniB
=w6DG
-----END PGP SIGNATURE-----
Hey all,

There are times, when you *really* want for a package to build
already... It's almost there... the tests passed... the package is
built! And then optiPNG kicks in and takes 33 minutes ([1] vs. [2]) to
finish...

For unity8 that time was Tuesday night - it took some 25 minutes to
build the package on armhf and then pkgbinarymangler went ahead and
optimized the PNGs, which on a pandaboard took another half an hour.

On the plus side, this saved us 571kB in image size (or 9.4%).

But it's done for every package build, for each architecture separately
- the time sacrifice was just too big. It doesn't have to be!
See [3], which is:

$ optipng -o7 `find -name *png`

Because of -o7 (slowest) we saved another 112kB, or 2%. Then, added
export NO_PNG_PKG_MANGLE=1 to debian/rules and voilá! that's a _lot_ of
our pandas' time saved.

The drawback is that you need to remember to optimize the PNGs when
adding new ones or changing them. It could be possible to mark the
already-optimized PNGs somehow (tagging comes to mind, but that would
usually survive modifications), so that we don't re-optimize them again
during pkgbinarymangle? We could probably use some manifest, but that
would mean shipping it in the binary package (is it possible to ship in
DEBIAN/ so that it isn't installed anywhere?), and we'd need to make
sure modified files are not omitted, so some kind of checksum would need
to be used - still much faster than optipng, probably...

Anyway! I recommend to all and everyone to consider pre-optimizing their
PNGs (that might be the case for other image formats / files as well,
not sure what else pkgbinarymangler [4] does). It might save you time,
the builders time, save power, trees, babies, baby pandas and bunnies.
For real!

[1] https://launchpad.net/~phablet-team/+archive/ppa/+build/4729025
[2] https://launchpad.net/~phablet-team/+archive/ppa/+build/4735955
[3] https://code.launchpad.net/~saviq/unity/8.optimize-pngs/+merge/170619
[4] https://launchpad.net/pkgbinarymangler/

Have a good weekend, people!
--
MichaƂ (Saviq) Sawicz <michal.sawicz@canonical.com>
Canonical Services Ltd.