On Thu, 4 Apr 2019 at 15:59, Dimitri John Ledkov <xnox@ubuntu.com> wrote:
>
> Hi all,
>
> On Thu, 4 Apr 2019 at 14:26, Lukasz Zemczak
> <lukasz.zemczak@canonical.com> wrote:
> >
> > I would also prefer not to see those in the description, especially
> > that we'll have automated bug comments on autopkgtest regressions
> > soon.
> >
> > That being said, I don't think someone not part of the SRU team should
> > be editing the SRU tempate without discussion with the SRU team. Can
> > we get this change reverted and at least have a discussion started
> > about it? If we allow anyone updating the SRU process as they see fit,
> > soon we'll have no official documentation and more people will be
> > confused.
> >
>
> The SRU template was started by me, out of experience of having many
> of my SRU uploads either linger in unapproved or getting rejected over
> repeating reasons. It has improved velocity and quality of the SRU
> process as experienced by the submitters. I have no idea if the SRU
> team likes the template or not, as it's neither mandatory nor required
> to be used, although many SRUs do use it.
I have not been around in the early SRU team years, so I don't know
who was the original author of the template. One thing I know for sure
is that we, the current SRU team, are pointing people to the
StableReleaseUpdates wiki page's SRU template whenever the paperwork
is insufficient. This means that one way or the other, this became the
de-facto standard for SRU paperwork, so at least currently changes to
it shouldn't be made without consultation. Especially that for most
people, this is the only thing they read, copy-pasting the template
and editing it in place, not necessarily reading the other parts of
the wikipage. Besides, since it's on the page where the official
documentation is present, how can someone know that this particular
part of the document is less official because it's somehow isn't under
the guidance of the owning team? That wouldn't make sense to me.
>
> The autopkgtest rergressions section is added by me, based on the
> observation that many SRUs in verification-done state are neither
> getting released nor getting any comments from neither submitters nor
> the SRU team members. I also observed that this often happens when
> there are autopkgtest regressions of the reverse triggers, and at the
> moment there is no documentation on doing hints merges, nor automatic
> notifications to the bug reports about them. I also observed that many
> SRU submitters are proactively documenting autopkgtest regressions and
> that their SRUs are getting released by the SRU team members faster,
> than the other uncommented aging verification-done SRUs.
So we will be having automated bug comments on autopkgtest
regressions, this is something that already has an MP but just needs a
final tweak before getting merged.
As for autopkgtest regressions per-se. So I don't know what
experiences you had, but generally what the process here is that, as
with any upload, the uploader is responsible for making sure the
autopkgtest situation is clear. What I mean by that is, the uploader
should look at the regressions for their landing and act on those
whenever possible. I guess this might need some formal-defining on the
wiki page, but generally this means that the uploader checks for
regressions, checks if they're real regressions or not, retries if he
thinks it's a transient issue. In case where the test regressed in
release or should be hinted one way or another, I personally never
ever requested people to submit hints by themselves (as MPs or diffs
or whatever). I know for the devel series there were some discussions
to make that the 'official' way of getting hints into the release, I
don't know of anything like that for SRUs. What I personally want is
for the uploader to mention the ADT regression situation in the bug as
a comment, possibly part of the verification or following the
verification. It is then generally my responsibility as the SRU team
member to submit the hints when they are necessary.
For me at least, so far this worked. Whenever I see a verified upload
with lots of ADT regressions, I always open up the bugs and check the
latest comments for any leads regarding autopkgtest regression
verification. If I see some and agree with the verification, I commit
hints for the issues and release (there are really rare cases where I
don't commit hints, but those are like 5% of all the cases). Sure, we
weren't really strict about applying hints, but on the Malta SRU
meeting we have put a strict rule on hinting anything we want to 'let
through'.
>
> Thus the addition of this section, is merely a reflection of the
> currently best-known way (which is proven to achieve the intended
> result) to get an SRU released as seen performed by multiple people
> across foundations, server, STS teams.
I am not specifically saying we should not go with this approach. I am
saying we should discuss this and decide on a solution that's
acceptable by the SRU team. I personally don't like having the
autopkgtest regressions in the description because of my current
process mentioned above (only looking at comments). Besides, using
comments for this seems more consistent with the overall validation.
We request comments with update verification and switching tags, and I
consider this part of verification (since there are reported
regressions, so I'd expect update validation to also check the
automated regressions). Having to look for similar things in two
different places seems, well, odd. But that's my preference, I guess?
>
> I have attempted to submit hints branch merge proposals, and attempted
> to document them, but failed at this task due to multiple reasons.
> Different sru team members preffer different style of hints, my hints
> often were rewritten in a different format and/or consolidated, or out
> right rejected "go and fix unrelated regressed in -updates
> autopkgtests due to security uploads" and etc. My previous request to
> the SRU team to document which hints, in what syntax, and where, and
> in which situations you'd accept has been so far not actioned. See
> previous mailing list thread in which Robie and I participated in.
> Dear SRU team, have you written the documentations to stir people to
> create hints merge proposals with guidance on syntax with examples?
>
> Or given the complexity of the hints, and how infrequently most SRU
> submitters have to work with them (unlike e.g. release team & sru
> team), it would be nice if SRU team members upon releasing SRUs commit
> appropriate hints to the hints branches as you see fit. It is beyond
> doubt that SRU team members have the best working knowledge of hints.
> And the burden to iterate hints with merge proposals & rejections /
> rewrites based on SRU team member comments shouldn't be with the SRU
> submitters. Especially since SRU team has never yet stated which hints
> are desired/expected.
That is how the process should look like currently. If someone from
the SRU team is not following this process (i.e. does not commit hints
for regressions that have been checked by the uploader and verified to
needing hinting), then it is something we should solve within the
team. We are currently not requesting uploaders to submit hint MPs for
regressions of their uploads. All we request is verification of those
regressions in the SRU bugs.
>
> I am happy to revert this new section in the template, once there is
> something better for SRU submitters to proactively help sheperd the
> SRU themselves. At the moment, there is none, despite previous
> requests.
>
> --
> Regards,
>
> Dimitri.
Let's get this sorted and find a solution we all agree on.
Cheers,
--
Łukasz 'sil2100' Zemczak
Foundations Team
lukasz.zemczak@canonical.com
www.canonical.com
--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
