On +1 maintenance this week, I noticed an unusual failure of icinga2 on
kinetic with a trigger of boost1.74/1.74.0-14ubuntu4:
https://autopkgtest.ubuntu.com/results/autopkgtest-kinetic/kinetic/arm64/i/icinga2/20220509_190945_354db@/log.gz
icinga2 does have an unusual test here. It runs its own binary and
verifies that the output from "--version" matches the upstream release
version it sees in debian/changelog from the source tree. "Source tree"
means the one it's given during the test run, which includes
debian/tests/* and debian/changelog.
This is resulting in the following unexpected behaviour:
Ubuntu's infrastructure calls the test with [unrelated options removed]:
autopkgtest --apt-pocket=proposed=src:boost1.74 --apt-upgrade icinga2
autopkgtest then runs "apt-get source icinga2" (or the equivalent),
pulling icinga2 2.13.3-1 from kinetic-proposed. But when it installs
icinga2 for the test, the pinning specified by --apt-pocket means that
it installs icinga2 2.13.2-1build2 from the release pocket.
Since the test is given the source tree from 2.13.3-1, it expects
"icinga2 --version" to return 2.13.3, but it returns 2.13.2 and so the
test fails.
Is the test wrong, or are we running the test wrong?
It seems to me that with the test being embedded into the source, it's
reasonable for the test to assume that it's testing the actual build it
itself ships. Otherwise a new test could for example test a new binary
that ships with a new upstream release, and it would fail when running
against older builds from the same package.
I suppose a hint would be appropriate here, or do you think that
something else needs doing instead?
This looks like an issue that also needs fixing generally, since it
suggests that we're often running the wrong version of the test suite
when using "triggers". Although admittedly running a "too-new" test
suite is certainly not as bad as running a "too-old" one.
Thoughts?
Robie
