Monday 12 March 2018

Re: zstd compression for packages

On Mon, Mar 12, 2018 at 03:05:13PM +0100, Julian Andres Klode wrote:
> On Mon, Mar 12, 2018 at 01:49:42PM +0000, Robie Basak wrote:
> > On Mon, Mar 12, 2018 at 11:06:11AM +0100, Julian Andres Klode wrote:
> > > We are considering requesting a FFe for that - the features are not
> > > invasive, and it allows us to turn it on by default in 18.10.
> >
> > libzstd has only been stable in the archive since Artful. We had to SRU
> > fixes to Xenial because it was added to Debian (and outside
> > experimental) before the format was stable upstream.
> >
> > Of all the general uses of a new compression algorithm, I'd expect our
> > distribution archival case to be near the end of a develop/test/rollout
> > cycle. Are you sure we want to rely on it so completely by switching to
> > it by default in 18.10?
>
> So the goal is to have it in 20.04, which means we should ship it now, so
> we can do upgrades from 18.04 to it. Whether we change the default in
> 18.10 or not, I don't know, but:

Sure. I don't have any objection to making it available now for future
use (apart from the usual post-FF required care etc. which the release
team will decide upon).

I can understand why it may be a goal for 20.04, but I assume that's
subject to it having proven itself by then. So while it makes sense to
start this by default in 18.10 to flush out any issues, that also
pre-supposes that it will have proven itself in the future. A tough call
I think, and not one I have enough information to have an opinion upon.
I mention it to point out that the other side of the trade-off exists.

> IMO, better 18.10 than later. We should gain experience with it,
> and if it turns out to be problematic, we can switch the default back
> and do no-change rebuilds for 20.04 :)
>
> That said, if we have problems, I expect people using zstd in filesystems
> (btrfs) or backup tools (borg) to be off worse.

I think there are certain classes of possible problems for which we will
be worse off than the users in the use cases you point out. The
publication of our archives is somewhat more permanent and we can't, for
example, restore from backup using a different compression to repair our
filesystem. It's providing an *automatic* and seamless upgrade path for
affected Ubuntu users that could prove difficult. In some other cases
where users have individually opted in, a seam isn't necessarily a
problem; but it can be for us.

Robie