On Sun, May 15, 2022 at 08:06:44AM -0400, Jeremy Bicha wrote:
> On Sat, May 14, 2022 at 10:18 AM Brian Murray <
brian@canonical.com> wrote:
> > On Fri, May 13, 2022 at 08:05:20PM +0200, Julian Andres Klode wrote:
> > > On Fri, May 13, 2022 at 12:06:46PM -0400, Jeremy Bicha wrote:
> > > > I am concerned that the excuses page is ignoring that webkit2gtk
> > > > 2.36.1-1 caused an autopkgtest regression for devhelp.
> > > >
https://autopkgtest.ubuntu.com/packages/d/devhelp/kinetic/amd64
> > > >
https://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html
> > > > The regression was correctly caught in Debian:
> > > >
https://tracker.debian.org/pkg/webkit2gtk
> > > > I am concerned because I would expect other packages could have
> > > > introduced regressions but been allowed in to Kinetic.
> > > It seems the state of the 2 autopkgtest-web workers is inconsistent,
> > > not all results were copied up on all instances.
> > > I noticed that it retried a migration-reference/0 test after the failure
> > > which indicated it did not allow the failure, but a later run might have
> > > hit the different backend and hence might have a different view.
> > > So it's unclear if that's the reason, but bdmurray is cleaning that up
> > > right now.
> > The cleanup finished overnight and now both instances of the
> > autopkgtest-web servers have the same information.
> My test case is still broken. devhelp is not blocking webkit2gtk on
> the excuses page.
This is because the test failure is not a regression.
* autopkgtest for devhelp/41.2-2: amd64: Not a regression, arm64: Not a regression, armhf: Test in progress, ppc64el: Not a regression, s390x: Not a regression
The test has never passed; it has had neutral results and failing results,
but by policy, proposed-migration does not treat neutral->fail as a
migration-blocking regression, only pass->fail. Neutral test results are
informational only.
If you want failures of this package's tests to be treated as regressions,
then the devhelp source package should be fixed so that its autopkgtests
return the correct error code.
This version of webkit2gtk has yet to migrate, so you could do this now, get
new devhelp to migrate, and then have webkit2gtk be blocked; or you could
just open a block-proposed bug if this is something that should be
considered a one-off.
In any case, hopefully this clarification about the policy is useful. It is
a policy that should be kept on the proposed-migration end because it
provides greater flexibility than a strictly pass/fail system, and also we
shouldn't deviate from Debian's behavior here and categorically have a
stricter gate than Debian which we don't have the capacity to maintain.
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer
https://www.debian.org/
slangasek@ubuntu.com vorlon@debian.org
