Wednesday 26 February 2014

Analysis of current ppc64el test failures

Hello fellow developers,

I went through the current failures of our autopkgtests on ppc64el
[1]. A lot of them are due to running in a container with much fewer
packages preinstalled than in our cloud-image VMs, i. e. they are
missing test depends (and sometimes point out missing binary depends);
I got two handfuls of those fixed in trusty already, but not yet all
of them.

I categorized the current failures for packages which don't fail on x86.

Tests that fail due to ppc64el specific issues
----------------------------------------------
apt-clone: tries to get indexes for ppc64el/saucy which don't exist
pbuilder: tries to debootstrap from archive.u.c., needs ports.u.c.
sbuild: tries to debootstrap from archive.u.c., needs ports.u.c.
software-properties: additional test fails that succeeds on x86
ubuntu-release-upgrader: no fglrx package, need to make that test x86 only
glib2.0: often hangs in monitor tests, but we had successes already
upstart: test failures during package build

Tests run, but some failures
----------------------------
The test failures which might be specific to ppc64el or due to running
them in a container:

adequate
bowtie2: no error message at all
bzr
devscripts
eglibc
exim4
firestring
floodlight
gearmand
ip4r
mysql-5.5
nagios3: no error message at all in test "nagios3"
neutron
notify-osd: no error message at all
nut: same test also often fails on x86, race condition
openvswitch: no useful error message
php5
postfix
puppet
ubuntu-drivers-common

Tests that fail due to packages not (yet) existing for ppc64el
--------------------------------------------------------------
asis
aspcud
bzr-git
chromium-browser
click-update-manager: missing Qt
colord: missing valgrind
dbus-test-runner
fatrace: needs powertop which probably won't work on ppc64el, so should skip the test on non-x86
firefox
friends
fsharp
git-annex
ipython
juju-core
juju-deployer
libacpi (should be an x86 only test)
libaunit
libfann
libgmpada
libgtkada
libncursesada
libtemplates-parser
libtexttools
libxmlada
matplotlib
mercurial
mod-german
mongodb
node-marked
nova
ofono-phonesim
opentoken
pandas
psqlodbc
pyqt5
python-dbusmock
python-memprof
python-numpy
roboptim-core
rubygems-integration
skimage
system-image
trac-bzr
u1db
ubuntu-purchase-service
ubuntuone-credentials
unity-firefox-extension
unity-scope-click
visp
winff


Tests that fail due to missing test/binary dependencies
-------------------------------------------------------
It could of course be that after adding them they fail for different reasons.

deja-dup
lapack
open-iscsi
postgresql-pgmp
pycparser
pygments
python-boto: looks like missing binary depends to p-requests?
python-leveldb
spamassassin
varnish: package doesn't even install, looks like it's missing deps


Tests that don't work in a container
------------------------------------
These can be dealt with by marking the tests as "don't work in a container".
The plan for that is [2], the fix is [3], and I asked for this new autopkgtest
feature in the FFE request [4].

apport
bluez
click-apparmor: EPERM to change /sys (disallowed in container), getpwnam() error
crash: trying to access the network [5], and probably won't work in a container
coreutils: two failures which are container specific (fails on x86 too); should fix/disable these two
gvfs
libnih: hangs eternally in test_file 78; some errors opening /proc/*/ns/, could be that it does not work in a container
network-manager
python2.7/3.3/3.4: I sent the fix to Matthias, he applied it in Debian; should work on next upload
squid3: tries to access the network [5], apparmor test fails
systemd
systemd-shim
udisks2

Fixing/testing
--------------

I'll look into the "missing depends" and "don't work in a container"
failures, but I can't single-handedly handle all the others. So if you
care about ppc64el, please feel invited to fix tests yourself. I'm
happy to assist with doing test runs of a .dsc from you on the ppc64el
boxes after they run in LXC on x86 for you:

First, create a suitable container:

$ debcheckout autopkgtest
$ sudo autopkgtest/tools/adt-build-lxc ubuntu trusty

Sorry, the adt-build-lxc script is not yet packaged properly, but next version
will do that as we use it in production now and it works well enough.

Run the test on the version of the source package "mypkg" in trusty:

$ autopkgtest/run-from-checkout mypkg --- lxc -es adt-trusty

Run them with -proposed enabled:

$ autopkgtest/run-from-checkout --apt-pocket=proposed -U mypkg --- lxc -es adt-trusty

Run the test of a local .dsc:

$ autopkgtest/run-from-checkout /path/to/mypkg.dsc --- lxc -es adt-trusty

If you only changed the tests, not the package itself, you can run

$ autopkgtest/run-from-checkout -B /path/to/mypkg.dsc --- lxc -es adt-trusty

to avoid building the .dsc first. See man adt-run for details of the
options.

Thanks,

Martin

[1] https://jenkins.qa.ubuntu.com/view/Trusty/view/AutoPkgTest%20ppc64el/

[2] http://lists.alioth.debian.org/pipermail/autopkgtest-devel/2014-January/000565.html
http://lists.alioth.debian.org/pipermail/autopkgtest-devel/2014-February/000586.html

[3] http://anonscm.debian.org/gitweb/?p=autopkgtest/autopkgtest.git;a=commitdiff;h=ff5809720

[4] https://launchpad.net/bugs/1285026

[5] I sent a ticket for IS to allow the test boxes to access our standard proxy,
so that tests can at least reach *.ubuntu.com.
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)