Thanks, Robie, for kicking off this discussion.
In regards to rails in particular, I would like to note that the same
version of the rails package is present in the bionic, cosmic, and disco
releases. Despite several uploads and syncs/merges from Debian, no new
version of the rails package that has landed in devel-proposed has been
releasable to devel since 7 April 2018
(https://launchpad.net/ubuntu/+source/rails/2%3A4.2.10-1/+publishinghistory).
Autopkgtest results show that we do not have a coherent set of rails-related
packages in devel-proposed that are releasable. And despite the best
efforts of Ubuntu developers working on proposed-migration over the past
year, we have not succeeded in untangling this, in part because there is a
lack of expertise in this stack among those working on it.
I think it's clear that this is a case where Ubuntu is not providing value
to our users by syncing the subset of modules that Debian have packaged,
particularly since - as you point out - we know that upstream is going to
recommend installing from the upstream language-specific repository (gems).
There's also no correlation between what version of these components was
current at the time of an Ubuntu release and what version is interesting for
deployment of real services over time.
So I agree with the general principle that we should be willing to remove
such language-specific stacks from the archive if they are not being
maintained (which, with my AA hat on, means: remove from devel and
devel-proposed and add to the sync blacklist).
I also agree for the specific case of rails that it should be removed from
Ubuntu releases going forward, unless something changes; and I believe that
we should put this into effect immediately for the Ubuntu 19.04 release,
despite the nearness of the release date.
Provided that there are no objections here, I will plan to start removing
this stack from disco and disco-proposed on Friday, April 5.
(These removals are reversible right up until the day before release, so I
don't see much point in a long comment period. If objections are raised
after Friday, we can still revisit.)
Here's a list of rails packages that look like immediate candidates for
removal based on this hackish script:
$ for src in $(
(for bin in $(reverse-depends src:rails
| sed -n -e'/^\* / { s/^\* //; s/[[:space:]].*//; p }')
do
grep-dctrl -sSource:Package -n -X -FPackage $bin \
/chroots/disco/var/lib/apt/lists/*_ubuntu_dists_disco*amd64_Packages
done
reverse-depends src:rails -b
| sed -n -e'/^\* / { s/^\* //; s/[[:space:]].*//; p }'
) | sort -u)
do
if reverse-depends src:$src | grep -q 'No reverse dependencies found' \
&& reverse-depends src:$src -b | grep -q 'No reverse dependencies found'
then
echo $src
fi
done
camping
debci
open-build-service
ruby-actionpack-action-caching
ruby-actionpack-page-caching
ruby-actionpack-xml-parser
ruby-activeldap
ruby-active-model-serializers
ruby-activerecord-import
ruby-activerecord-nulldb-adapter
ruby-acts-as-api
ruby-acts-as-taggable-on
ruby-after-commit-queue
ruby-api-pagination
ruby-appraiser
ruby-asset-sync
ruby-babosa
ruby-backbone-on-rails
ruby-blade
ruby-browser
ruby-carrierwave
ruby-cocaine
ruby-data-migrate
ruby-default-value-for
ruby-devise-lastseenable
ruby-devise-two-factor
ruby-diaspora-federation-rails
ruby-dropzonejs-rails
ruby-email-validator
ruby-entypo-rails
ruby-enumerize
ruby-exception-notification
ruby-factory-bot-rails
ruby-factory-girl-rails
ruby-flipper
ruby-gettext-i18n-rails-js
ruby-gon
ruby-grape-path-helpers
ruby-graphiql-rails
ruby-hamlit
ruby-haml-magic-translations
ruby-haml-rails
ruby-hashie-forbidden-attributes
ruby-health-check
ruby-http-accept-language
ruby-i18n-inflector-rails
ruby-ice-cube
ruby-jira
ruby-jquery-atwho-rails
ruby-jquery-datatables-rails
ruby-jquery-scrollto-rails
ruby-jquery-turbolinks
ruby-js-image-paths
ruby-js-routes
ruby-leaflet-rails
ruby-lograge
ruby-mail-gpg
ruby-markerb
ruby-messagebus-api
ruby-mobile-fu
ruby-model-tokenizer
ruby-momentjs-rails
ruby-omniauth-crowd
ruby-origin
ruby-paranoia
ruby-premailer-rails
ruby-rack-google-analytics
ruby-rails-i18n
ruby-rails-timeago
ruby-rbpdf
ruby-roadie-rails
ruby-roxml
ruby-rqrcode-rails3
ruby-seed-fu
ruby-sentry-raven
ruby-sequenced
ruby-simple-captcha2
ruby-simple-form
ruby-snorlax
ruby-spring-watcher-listen
ruby-state-machines-activerecord
ruby-task-list
ruby-test-after-commit
ruby-validates-hostname
ruby-versionist
ruby-voight-kampff
ruby-webpack-rails
ruby-yaml-db
$
On Wed, Apr 03, 2019 at 12:41:04PM +0100, Robie Basak wrote:
> I'd like to talk about addressing the difficulty in maintenance of long
> tail language-specific stacks in Ubuntu. For example, right now
> `src:rails` is stuck in disco-proposed[1]. It seems to me that we spend
> a disproportionate amount of effort trying to get this class of package
> migrated to the release pocket compared to the number of Ubuntu users
> who actually care and use them.
>
> I suggest that:
>
> 1. If a language-specific package, or stack of packages, is stuck in
> proposed, and nobody is volunteering to get them migrated, then we are
> more willing to delete them from the release pocket and release without
> that stack.
>
> 2. We recommend, as a project, that users who wish to use these
> language stacks directly do so via the language-specific packaging
> tooling.
>
> Background
> ----------
>
> It'd be great to get input from language-specific communities who may
> not be intimately familiar with distribution development process, so
> here's a quick summary of what I'm talking about. Those already familiar
> with distribution development workflow can skip to the next section.
>
> You're presumably aware that language-specific communities generally
> have their own package repositories and package managers, such as
> PyPI/pip, RubyGems.org/gem, npm Registry/npm and so on. My understanding
> is that these communities generally advise that users consume from these
> repositories directly.
>
> Debian often packages a subset of these repositories - usually for the
> purpose of fulfilling the dependency requirement of some higher level
> component, such as Rails in my example. Ubuntu then autosyncs these.
> Some users prefer these higher level components to come through their
> distribution, rather than via a language-specific package manager -
> presumably for the release management consolidation that this provides.
> However, it's my understanding that in practice the majority of users do
> not consume the distribution packaging for these components, instead
> using the language-specific package managers directly as generally
> recommended by those upstreams. It also seems quite common for language
> communities to specifically recommend against consuming language module
> packages through the distribution.
>
> In general distributions include only one version of each component in a
> given distribution release. Bringing the entire dependency web in line
> to make this possible can take considerable effort on the part of
> distribution developers, particularly when upstreams tend to use a large
> number of small dependencies and also require specific dependency
> versions due to frequent API breaks.
>
> In practice, the dependency web is more complicated than this, and
> entire swathes of packages get held up at once until all the
> dependencies can be resolved together; this takes work in figuring out
> what the holdups are, making decisions about which versions to ship, and
> possibly patching code to change what dependency versions are actually
> required, in order to make everything work together. Think of it like
> trying to come up with a single `requirements.txt` (as generated by `pip
> freeze`), `Gemfile.lock` or `package-lock.json` file that works for all
> packages across the entire distribution release.
>
> Until this is resolved, distribution package updates are held in a
> staging area, and won't be part of the next distribution release. In
> Ubuntu we say that that our packages are "stuck in proposed". Once
> resolved, the packages "migrate" to the "release pocket".
>
> The details of this process in Ubuntu are documented here:
> https://wiki.ubuntu.com/ProposedMigration
>
> For example, we are blocked from shipping a newer version of the PHP
> interpreter until the `wordpress` package (which is written in PHP)
> works with the proposed newer PHP version. Now consider that there are
> hundreds of these reverse dependency packages like `wordpress`,
> including many PHP language modules. The addition of each one makes it a
> little harder for Ubuntu to move on updating PHP itself. My suggestion
> is that we more readily remove these reverse dependencies from the
> distribution release to free up the update of PHP itself in this
> example, rather that spend what seems like a disproportionate amount of
> effort fixing things that we suspect very few users actually care about.
>
> Discussion
> ----------
>
> Part of the purpose of my suggestion is to make it far easier to
> transition to new language interpreters without worrying too much about
> the very long tail of barely used reverse dependency language modules
> that usually hold up these transitions. My understanding is that far
> more users rely on the distribution supplying the language interpreter
> and language-specific package manager than the modules themselves. In
> case of a transition being held up like this, I'm proposing to simply
> permit the deletion of the long tail from the release pocket, get the
> newer interpreter stack migrated into the release pocket, and consider
> it done. The long tail will then migrate if maintained actively by
> others, and if it isn't, we'll ship without it.
>
> Right now for example, I'm suggesting that we simply delete `src:rails`
> from the release pocket, including its reverse dependencies, unless the
> reverse dependency list contains something outside the Rails stack that
> is unacceptable to us to delete. Rails users who use `gem install
> rails`, as recommended by Rails upstream[2], will not be affected.
>
> My suggestion deliberately leaves the door open for able volunteers to
> be able to maintain these packages in Ubuntu if they wish.
>
> One downside to my suggestion is that availability of particular
> language module packages may become unreliable between Ubuntu releases
> from a user's perspective (if a particular package skips an Ubuntu
> release before being restored, for example). I suggest that this can be
> tackled later if it becomes a problem, for example by blacklisting such
> packages for longer unless a team is prepared to commit to preventing
> this from happening.
>
> Another potential downside is in packages that are generally useful to
> users outside their own language ecosystems, yet depend on these
> language-specific dependency stacks. For example, take Vagrant. Vagrant
> is written in Ruby, so needs packages originating from RubyGems.org. I'm
> sure there are plenty of users who aren't deploying a Ruby-based stack
> but who do use Vagrant. I think that a significant number of these users
> probably prefer to consume Vagrant from the distribution package (`apt
> install vagrant`) rather than from RubyGems.org (`gem install
> vagrant`)[3]. Because distributions include all of their dependencies in
> their own repositories, this means that to ship Vagrant as a package in
> Ubuntu, we also need to ship a package for everything in RubyGems.org
> that Vagrant requires. I suggest that we don't apply my deletion policy
> to such packages and their dependencies - that we continue trying to
> maintain them on a best effort basis as we do today.
>
> If we do change anything in this regard, I think an important part of
> this is for us to decide as a project what affected users can expect,
> what we recommend that they do, and that we communicate this clearly. I
> suggest that, in the absence of Ubuntu development teams volunteering to
> maintain this class of packages in Ubuntu, we generally follow
> upstream's recommendations in using their language-specific package
> management stack rather than apt/dpkg. This isn't all that different
> from the situation today, where users can't rely on us having packaged
> the specific modules that they need at particular versions anyway.
>
> We might want to be more specific in our recommendations to users. For
> example, it generally is better for system stability, particularly when
> installing software from third party sources, for third party software
> to be confined well. Where such a system is available, we could
> specifically recommend its use, and recommend against installing to the
> "system". For example, with the use of virtualenv for Python stacks. I
> note that Ruby is available from upstream as a snap, so if suitable for
> general deployments that might be the gold standard for recommended
> confinement; if not, then at least rvm.
>
> Please discuss. I intend to draw this thread to the attention of
> language-specific communities too, to try and get their input. In
> particular it'd be great to agree on the same recommendations for Ubuntu
> users. Note that this list is moderated; I propose to permit emails from
> language-specific communities in response to this thread for a few weeks
> to avoid fragmenting discussion between here and the unmoderated list.
> I'll make sure replies get through moderation promptly. Don't worry
> about not being subscribed: just replying to the list will be fine.
>
> Thanks,
>
> Robie
>
>
> [1] http://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html#rails
>
> [2] https://guides.rubyonrails.org/getting_started.html#installing-rails
>
> [3] I note that Vagrant upstream specifically advises against `apt
> install vagrant` from distributions[4]. Contrary to my understanding of
> typical _direct_ use of language-specific stacks with which I'm
> justifying my suggestion, however, my understanding is that many of
> those who use Vagrant but aren't Ruby-based developers prefer the
> distribution package regardless. It makes sense to me that a
> non-Ruby-developer wouldn't want to learn, use and maintain an entirely
> different package manager or add an additional third party software
> source just for one top level package that the distribution ships
> anyway, unless they specifically really need a version newer than one
> provided by the distribution release.
>
> [4] https://www.vagrantup.com/docs/installation/
> --
> ubuntu-devel mailing list
> ubuntu-devel@lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer https://www.debian.org/
slangasek@ubuntu.com vorlon@debian.org
