On Wed, Apr 10, 2019 at 07:20:19AM +0200, Matthias Klose wrote:
> Please can we delegate the hinting for autopkg test failures to core
> developers and MOTU?
Short answer: not without some engineering work around britney; and I don't
think that work would be the best use of someone's time who was trying to
improve development velocity in -proposed.
The autopkgtest hints are done through hints files in a bzr branch which is
owned by the ubuntu-release team. The hints files don't just control
whether to ignore autopkgtest failures; they control how to override all
manner of checks in britney, they control the behavior during the freeze,
etc. It would be work to decouple auotpkgtest hint privileges from the
rest.
I would definitely be -1 on giving all Ubuntu Developers (or core-devs, or
MOTU) free reign on being able to commit the other sorts of hints.
And I think that the more important work is to fix the known bugs, that the
release team has agreed should be addressed, that cause us to have to
manually add hints to override autopkgtest failures when they are not
regressions vs the release pocket. The vast majority of time spent on
autopkgtest hints is on either proving that a failure is not a regression vs
release, or manually adding the hints for these non-regressions; and I think
it's much more valuable to reduce the amount of human effort that goes into
managing autopkgtest regressions than to increase the pool of humans that
have access to override britney.
> When ignoring an autopkg test failure, you usually have a reason to do so.
> As a core developer you already can work around autopkg test failures with
> uploading a work-around in a new package version, both for main and
> universe packages. The disadvantage is a longer turn-around, re-triggered
> autopkg tests, introducing a delta which has to be maintained. MOTU could
> be limited to only hint failures for universe packages.
Ubuntu developers *can* work around autopkgtest failures in package uploads
by disabling tests. Most developers *don't* do this, because they recognize
that disabling tests that are pointing out actual bugs in the software, in
order to let that software migrate to the release pocket, does not generally
serve the goal of releasing a high-quality OS.
Nevertheless, I see a lot more willingness on the part of developers to
ignore regressing autopkgtests in britney - even though this has a much
GREATER negative impact on the quality of a release. Unlike disabling tests
in the test suite, a hint to override autopkgtest failures cannot be
selective, so any future further regressions in the test suite would not be
caught, completely negating the value of those tests for CI.
I think it's correct, in terms of encouraging the behavior we want, that it
is harder to get an autopkgtest override in britney than it is to fix the
testsuite in the package.
For this reason, I'm also -1 on the idea of giving non-release-team members
direct access to override autopkgtest regressions as well.
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer https://www.debian.org/
slangasek@ubuntu.com vorlon@debian.org
