Thursday 4 April 2019

Re: Proposal: drop rails and its reverse-dependencies from Ubuntu 19.04 [Re: Maintaining language-specific module package stacks]

Hi Matthias,

On Thu, Apr 04, 2019 at 02:07:50PM +0200, Matthias Klose wrote:
> On 04.04.19 01:52, Steve Langasek wrote:
> > Thanks, Robie, for kicking off this discussion.

> > In regards to rails in particular, I would like to note that the same
> > version of the rails package is present in the bionic, cosmic, and disco
> > releases. Despite several uploads and syncs/merges from Debian, no new
> > version of the rails package that has landed in devel-proposed has been
> > releasable to devel since 7 April 2018
> > (https://launchpad.net/ubuntu/+source/rails/2%3A4.2.10-1/+publishinghistory).

> > Autopkgtest results show that we do not have a coherent set of rails-related
> > packages in devel-proposed that are releasable. And despite the best
> > efforts of Ubuntu developers working on proposed-migration over the past
> > year, we have not succeeded in untangling this, in part because there is a
> > lack of expertise in this stack among those working on it.

> > I think it's clear that this is a case where Ubuntu is not providing value
> > to our users by syncing the subset of modules that Debian have packaged,
> > particularly since - as you point out - we know that upstream is going to
> > recommend installing from the upstream language-specific repository (gems).
> > There's also no correlation between what version of these components was
> > current at the time of an Ubuntu release and what version is interesting for
> > deployment of real services over time.

> This is short-sighted, and greatly influenced by the voices of
> language-specific upstream communities. As seen at several occasions at
> PyCon: Ask an upstream community, which Linux distribution they use
> (majority of hands go up for Ubuntu), and then how many of those use the
> system Python (majority of hands go down). Please ask this Python
> question to an upstream Java, upstream Ruby community, and I assume that
> nobody cares and uses the Python as distributed by Ubuntu. Ask the Java
> and Python communities the same question about Ruby, and these are
> probably happy with the Ruby found on Ubuntu. Now remove the Ruby, Python
> and Java stacks, and probably nobody will be happy anymore.

I don't think it's anyone's intent to remove the language runtimes and
bindings as a whole. We're looking at rails in particular as a subset of
the packages in the archive which depend on ruby because:

- rails is a web framework, and we know historically that debs aren't a
good fit for delivering web apps to systems
- updates to the stack have been unreleasable for a year without either
removing packages or shipping known broken packages, which requires a
disproportionate amount of effort to resolve
- ruby as a whole is a pain point for Ubuntu because no one from the
upstream community is engaged with its maintenance as an Ubuntu developer

My goal is not to remove rails (or any other specific package) per se. My
goal is to have a better process by which we can come to a decision that a
related group of packages is not well-maintained and should be removed.
Currently, the Archive Team and the Release Team have the authority to
decide that a package should be removed from the release; but the actual
process followed involves filing of bug reports against individual packages
and per-package soul-searching on the part of an archive admin.

Do you disagree that we should remove rails from the archive, given its
current condition? Or are you concerned that the rationale is wrong and
sets a bad precedent?


> Pointing to the upstream language-specific repositories is problematic.
> These are usually only well maintained for x86_64, maybe some have support
> for ARM32, but many don't have support for newer architectures like arm64,
> ppc64el and s390x, or have dropped support (like for ix86). Didn't Ubuntu
> promise to support new architectures the same way we support x86_64?

> Even x86_64 specific upstream repositories can be problematic. Some
> architecture specific Python manylinux wheels have subtle issues because they
> are built on an old CentOS release, and you see issues in the Launchpad tracker
> which wrongly attribute issues in these third party wheels to packages found in
> Ubuntu. Well, manylinux should have been called somelinux or even centos...

Another hacky script, here's the complete list of architecture-dependent
reverse-dependencies (non-transitive) of rails in disco:

$ for src in $(
(for bin in $(reverse-depends src:rails \
| sed -n -e'/^\* / { s/^\* //; s/[[:space:]].*//; p }')
do
grep-dctrl -sSource:Package -n -X -FPackage $bin \
/chroots/disco/var/lib/apt/lists/*_ubuntu_dists_disco*amd64_Packages
done
reverse-depends src:rails -b \
| sed -n -e'/^\* / { s/^\* //; s/[[:space:]].*//; p }'
) | sort -u)
do
rmadison -s disco $src \|grep -E 'amd64|arm64|armhf|i386|s390x'
done
ruby-escape-utils | 1.2.1-1build1 | disco/universe | source, amd64, arm64, armhf, i386, ppc64el, s390x
ruby-hamlit | 2.9.2-2 | disco/universe | source, amd64, arm64, armhf, i386, ppc64el, s390x
sonic-pi | 2.10.0~repack-2.1 | disco/universe | source, amd64, arm64, armhf, i386
$

Do you think we should keep rails in the archive on account of these three
packages specifically, and remove the arch: all packages?

Do you think that we should categorically remove all reverse-dependencies of
rails which have autopkgtests showing that they are broken with the new
version of rails, allowing the new version in?


> > So I agree with the general principle that we should be willing to remove
> > such language-specific stacks from the archive if they are not being
> > maintained (which, with my AA hat on, means: remove from devel and
> > devel-proposed and add to the sync blacklist).

> > I also agree for the specific case of rails that it should be removed from
> > Ubuntu releases going forward, unless something changes; and I believe that
> > we should put this into effect immediately for the Ubuntu 19.04 release,
> > despite the nearness of the release date.

> > Provided that there are no objections here, I will plan to start removing
> > this stack from disco and disco-proposed on Friday, April 5.

> I disagree with this approach. There are at least two packages with
> failing autopkg tests which were removed in Debian. So why are those not
> removed in Ubuntu as well? I think we should fix our ubuntu-archive tasks
> first, and see what kind of actions we are missing, and only then decide
> if those packages should be removed or not.

> With your AA hat on, why are those removals not handled before proposing
> the removal of every dependency?

I think it would've been material to this discussion for you to mention
which specific packages here you see that should be removed.

I as an AA do regularly run process-removals and try to chase down the
various loose ends that prevent us from removing from Ubuntu various
packages that have been removed from Debian.

It's a lot of work.

I think that's also ignoring the fact that rails has been unreleasable for a
year, while - if I've correctly identified the packages you're referring to
- the Debian removals happened a month ago. So this is a single
point-in-time view that obscures the fact that the amount of effort the
Ubuntu community is currently able to put into the archive is not sufficient
to make this package releasable over the course of a year up to that point;
AND it wrongly implies that if only the archive admins processed removals
more regularly, that there would not be a maintenance cost to having this
stack in the archive.

That's simply not true. It ignores the fact that there is another
reverse-dependency of rails that has not been removed from Debian unstable
and also has a regressing autopkgtest. It ignores the fact that rails
itself in devel-proposed is also blocked by a dependency on another package
stuck in -proposed which also has regressing autopkgtests. It ignores the
aggregate effort over the last year that Ubuntu developers have put into
repeatedly fix the release blockers, only to fall short.


The premise here is: it takes more work than anyone is willing to put into
it to keep the rails stack in the Ubuntu archive in a healthy state. So
counterproposals that identify additional work you think we should be
willing to do really don't address the problem. Juggling your own
priorities, in response to this thread, in order to make time to "prove"
that the additional amount of effort required to maintain rails is trivial,
doesn't really address the problem.

We are where we are having this conversation because the facts on the ground
show that rails and related packages are not being effectively maintained,
given available resources. Assuming that the available resources are fixed,
what do you think we should do here to make Ubuntu releases as high quality
and as useful to our users as possible?

> > Here's a list of rails packages that look like immediate candidates for
> > removal based on this hackish script:

> [...]
> > debci

> Is this a commitment of the ubuntu-release team to maintain the stack
> needed for the autopkg tests outside the archive?

The Ubuntu autopkgtest infrastructure does not rely on the debci package.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer https://www.debian.org/
slangasek@ubuntu.com vorlon@debian.org