Thursday 25 July 2013

Review of current autopkgtest failures, bucket into teams

Hello all,

in the past few weeks we got quite a number of autopkgtest
regressions, as well as a bunch of new tests added which either never
worked, or only worked once, etc. I usually review every failure
whether it's caused by infrastructure and also fix quite a few which
e. g. come in through Debian, but I can only keep up so much. So I
think we need to find a better process for this.

For some failures it is quite easy to spot a particular person who
should own this, but for others that's not the case. For example,
ubuntu-release-upgrader hasn't changed since raring (and succeeded
there), now it's failing because update-manager changed. The
autopkgtests are usually meant to hold back these cases, but we have
piled up quite a lot of release team overrides by now, which subvert
that process (that's not meant as a criticism per se, I know that we
sometimes need to push stuff through).

I reviewed the current list of failures and bucketed them into
team/person, and also noted whether that test never succeeded or
regressed. The latter are particularly worrysome and ought to be fixed
ASAP, for the former we could discuss whether nobody cares about this
package (any more) and we just drop the test (examples:
unity-firefox-extension or oneconf).

For the team-owned ones I propose that the relevant team's tech lead
is a default fallback if there is no clear owner, and fans out these
to developers?

For the broken ones from Debian, we could maybe do some community
rallying / hack day. If anyone is interested in fixing them, please
stop by #ubuntu-quality and I'm happy to provide
guidance/testing/sponsoring.

Thanks,

Martin

== broken import from Debian ==
saucy-adt-abi-compliance-checker (never worked)
saucy-adt-beets (regression)
saucy-adt-libgtkada (never worked)
saucy-adt-plv8 (never worked)
saucy-adt-postgresql-pgmp (never worked)
saucy-adt-pyparsing (broken test metadata, never worked)
saucy-adt-python-csb (never worked)
saucy-adt-s3ql (never worked)

== problem with infrastructure (QA owned) ==
saucy-adt-adequate (we don't support "breaks-testbed")
saucy-adt-linux (VM is too small)
saucy-adt-ubuntuone-client (supposed to work now, problem with "allow-stderr")

== foundations ==
saucy-adt-apt-clone (regression)
saucy-adt-eglibc (rebuild test FTBFS)
saucy-adt-lintian (regression)
saucy-adt-pandas (regression)
saucy-adt-python2.7 (never worked on amd64, ok on i386)
saucy-adt-python3.3 (never worked on amd64, ok on i386)
saucy-adt-ubuntu-release-upgrader (regression)

== desktop ==
saucy-adt-colord (never worked)
saucy-adt-firefox (regression)
saucy-adt-friends (regression)
saucy-adt-libreoffice (regression)
saucy-adt-notify-osd (regression)
saucy-adt-oneconf (never worked)
saucy-adt-software-center (never worked)
saucy-adt-unity-firefox-extension (never worked)

== server ==
saucy-adt-lxc (regression)
saucy-adt-maas-cluster-controller (never worked)
saucy-adt-maas-manual (never worked)
saucy-adt-mysql-5.5 (just one test failing, never worked)
saucy-adt-nagios3 (could be a python-simplejson regression on amd64, regression)

== uploaded by particular developer ==
saucy-adt-dirspec (never worked, dobey)
saucy-adt-freeradius (regression, yolanda)
saucy-adt-gvfs (regression, pitti)
saucy-adt-heat (never worked, yolanda)
saucy-adt-mailman (regression, yolanda)
saucy-adt-quantum (regression, yolanda)
saucy-adt-squid3 (never worked on amd64, i386 ok, yolanda)


Thanks,

Martin
[1] https://jenkins.qa.ubuntu.com/view/Saucy/view/AutoPkgTest/
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel