Wednesday 11 November 2015

Re: ANN: autopkgtests are now specific for "triggering" package for better isolation

Hello Adam, Steve,

Steve Langasek [2015-11-10 13:52 -0800]:
> On Tue, Nov 10, 2015 at 02:16:39PM -0700, Adam Conrad wrote:
> > Robert's still on the right track here, though. This is just poor use of
> > pinning. See the following:
>
> > (wily-amd64)root@nosferatu:~# cat /etc/apt/preferences.d/10adt-pinning
> > Package: *
> > Pin: release a=xenial
> > Pin-Priority: 900
>
> > Package: *
> > Pin: release a=xenial-proposed
> > Pin-Priority: 800
> > (wily-amd64)root@nosferatu:~# apt-get dist-upgrade
> > Reading package lists... Done
> > Building dependency tree
> > Reading state information... Done
> > Calculating upgrade... Done
> > 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
>
> So the notable difference between this, and what autopkgtest is currently
> doing, is the use of a Pin-Priority of 800 vs. 100. Martin, do you want to
> check that raising the pin priority for xenial-proposed solves the problems
> we were seeing earlier?

I actually already did that earlier, and using 100 vs. 800 for
"release a=xenial-proposed" makes no noticeable difference.
autopkgtest's test case for this where the -proposed version of a
package has a versioned depends on a lib that can only be satisfied in
-proposed fails with either, and conversely the real-life tests that
are running seem to work well enough with 100.

I'm fine to bump it to 800 as that seems to work equally well, though;
done in [1].

Note that several tests failed with "test dependencies are
unsatisfiable" over night. The reason for that was that autopkgtest
had an insufficient fallback to no apt pinning (now fixed in [2]), and
that we currently dist-upgrade the testbed to the entirety of
-proposed before we start installing test dependencies and running
tests. That dist-upgrade can make the testbed's installed packages
incompatible with the apt pinning. E. g. in [3] libsqlite3-0 (which is
part of the cloud image) already got upgraded, but then the test tried
to install "sqlite3" from xenial-release, which explodes. With the fix
from [2] this is now handled properly [4].

The question remains whether in the new "do isolated tests" regime we
still actually want to dist-upgrade the entire base testbed to
-proposed, or just leave it as -release. Many of our basic packages
(kernel, systemd, coreutils, etc.) have proper autopkgtests, some have
tests with bad coverage (util-linux) and many don't have tests at all
(initramfs-tools, openssl, curl, grub). As long as that's the case I'd
still prefer breaking other tests due to broken versions of those over
silently promoting them because they lack tests.

Thanks,

Martin

[1] http://anonscm.debian.org/cgit/autopkgtest/autopkgtest.git/commit/?id=e41b89a51f7eb
[2] http://anonscm.debian.org/cgit/autopkgtest/autopkgtest.git/commit/?id=cc4334633
[3] https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-xenial/xenial/amd64/l/lxd/20151111_030750@/log.gz
[4] https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-xenial/xenial/amd64/l/lxd/20151111_080012@/log.gz

--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)