Wednesday 14 March 2018

ext4 metadata_csum and backwards compatibility

I'd like to draw attention to:
https://bugs.launchpad.net/ubuntu/+source/e2fsprogs/+bug/1601997

IRC discussion:
https://irclogs.ubuntu.com/2018/03/13/%23ubuntu-devel.html#t12:05

tl;dr: by doing nothing we're implicitly breaking some filesystem
forward compatibility. I'd like us to make an explicit decision about
whether we want this.

I think this probably doesn't matter in the desktop use case. Wearing my
server hat, I think it'll impact more server users, so I'm also CC-ing
the ubuntu-server list.

Theodore Ts'o enabled metadata_csum in mke2fs by default in Debian "so
that in the testing and unstable branches, metadadata_csum checking
would get some additional exposure, and hence testing" (from the bug).
At that time he said he hadn't decided if he wanted it there in time for
the following Debian release. AFAICT, it was enabled by stretch. AIUI,
Xenial's e2fsprogs currently has no support for metadata_csum at all.

A consequence of this (I'm told) is that Xenial's e2fsck doesn't work
against filesystems created by Bionic's installer. IOW, this change
breaks forwards compatibility between LTSs for Ubuntu.

I'm not aware that this kind of compatibility is a promise that we've
ever made, but IME it is a reasonably common thing for sysadmins to want
to do. IMHO, breaking compatibility like this is sometimes necessary but
is certainly inconvenient.

I would prefer us to make an explicit decision on this, rather than have
it happen to us implicitly as a consequence of how our ecosystem works.

Here are some options:

1) [the default if we do nothing] Bionic will have metadata_csum enabled
in new filesystems by default. Xenial's e2fsck will not be able to act
on Bionic-default-created ext4 filesystems.

2) We disable metadata_csum in e2fsprogs in Bionic by default. The
feature will only be available on an opt-in basis, which in practice
will mean that only a minority of users will be able to benefit from the
feature itself. There will be no compatibility problems.

2b) 2, but we also enable metadata_csum again in Bionic+1. There will be
subsequent forward compatibilty problems when skipping across multiple
LTSs, but not across any single LTS.

3) We backport metadata_csum support to Xenial in an SRU[1] without
changing the default there. Xenial users will be able to fsck
Bionic-created ext4 filesystems. There will be forward compatibilty
problems when skipping across multiple LTSs (eg. Trusty accessing a
Bionic-created ext4 filesystem), but not across any single LTS.

3b) 3, but we also SRU to Trusty. There will be no compatibility
problems across still-supported releases.

4) [I don't like this but state it for completeness] Leave e2fsprogs as
it is in Bionic, but adjust the installer to create filesystems without
metadata_csum. There will be no compatibility problems except in the
case that users run mkfs.ext4 directly. I don't like this because I
think it will make compatibility behaviour too surprising for users.


I'm not keen on the default option 1 only because I have experiences of
this kind of forward incompatibility being inconvenient for me in the
past (when migrating systems in datacentres, etc). If the cost of
maintaining at least some level of forward compatibility isn't too
great, I would prefer that we keep it. If we do go with option 1 by
doing nothing, I would at least like to release note this
incompatibility.

What do you think?

Robie


[1] I'm not sure that this fits existing SRU policy exactly. It's a
little subjective. I assume that if consensus is to do this, then the
required approvals will be forthcoming.