Monday 24 May 2021

Re: nfs-common won't restart all services after upgrade

On Thu, May 13, 2021 at 8:34 AM Andreas Hasenack <andreas@canonical.com> wrote:
>
> Hi,
>
> tl;dr
>
> I filed https://bugs.launchpad.net/ubuntu/+source/nfs-utils/+bug/1928259 found while testing https://bugs.launchpad.net/ubuntu/+source/nfs-utils/+bug/1927745 and verifying that rpc.gssd was not restarted after a package upgrade. Which means the fix wasn't available until the service was restarted manually.
>
>
> # Troubleshooting story
>
> I was testing https://bugs.launchpad.net/ubuntu/+source/nfs-utils/+bug/1927745, and while the fix is correct, it didn't always "stick" after I upgraded the packages.
>
> Further troubleshooting showed that some of the NFS services are not restarted after a package upgrade, under a specific condition which took a while to figure out.
>
> Many services make up a NFS server or client, so sometime ago debian decided to wrap them all around a systemd service called nfs-utils.service,

actually upstream added the service, not debian
http://git.linux-nfs.org/?p=steved/nfs-utils.git;a=commit;h=c0512981b7e10487378e5c6fec5d7d72dd4f7e1a

It certainly doesn't appear to be right way to handle
management/coordination of the various services, but probably the
discussion should involve upstream to figure out the proper way to
redesign things.

> which is a bit fake, used just to coordinate all the real services. Its header explains it:
> $ systemctl cat nfs-utils.service
> # /lib/systemd/system/nfs-utils.service
> [Unit]
> Description=NFS server and client services
> # This service should never be stopped, only restarted.
> # When it is re-started, all other services which declare
> # themselves to be "PartOf" this service will also be
> # restarted. Thus
> # systemctl restart nfs-utils
> # will restart all daemons which are part of nfs-utils
> # and which are running. This is useful after a software
> # update.
>
> # This is a "service" rather than "target" so that we
> # don't need to say "systemctl restart nfs-utils.target".
> [Service]
> Type=oneshot
> RemainAfterExit=yes
> ExecStart=/bin/true
>
> d/rules has these, and we can see it does not enable nfs-utils.service, but asks for it to be restarted on upgrade:
> dh_systemd_enable -p nfs-common nfs-client.target
> dh_systemd_enable -p nfs-kernel-server nfs-server.service
> dh_installinit -pnfs-common -R
> dh_systemd_start -p nfs-common --restart-after-upgrade nfs-utils.service
> dh_systemd_start -p nfs-kernel-server --restart-after-upgrade nfs-server.service
>
> And this "fake" service really can't be enabled:
> $ sudo systemctl enable nfs-utils.service
> The unit files have no installation config (WantedBy, RequiredBy, Also, Alias
> settings in the [Install] section, and DefaultInstance for template units).
> (...long explanation follows this output ...)
>
> We get this during package install:
> Setting up nfs-common (1:1.3.4-2.1ubuntu5.3) ...
> nfs-utils.service is a disabled or a static unit not running, not starting it.
>
> Even when upgrading:
> Setting up nfs-common (1:1.3.4-2.1ubuntu5.4) ...
> nfs-utils.service is a disabled or a static unit not running, not starting it.
>
> This is because the service is not enabled.
>
> Critically for the bug I'm fixing, rpc.gssd is not restarted, so the fix isn't applied :/
> Before upgrade:
> 442 ? Ss 0:00 /usr/sbin/blkmapd
> 7146 ? Ss 0:00 /usr/sbin/rpc.gssd
> 7399 ? Ss 0:00 /usr/sbin/rpc.idmapd
> 7406 ? Ss 0:00 /usr/sbin/rpc.mountd --manage-gids
> 7400 ? Ss 0:00 /usr/sbin/rpc.svcgssd
> After pkg upgrade:
> 442 ? Ss 0:00 /usr/sbin/blkmapd
> 7146 ? Ss 0:00 /usr/sbin/rpc.gssd
> 8421 ? Ss 0:00 /usr/sbin/rpc.idmapd
> 8422 ? Ss 0:00 /usr/sbin/rpc.mountd --manage-gids
> 8420 ? Ss 0:00 /usr/sbin/rpc.svcgssd
>
> If I do a manual "sudo systemctl start nfs-utils.service" (or restart) before upgrading the package, then all these processes are restarted after the package upgrade, because deb-systemd-invoke sees nfs-utils.service as "started". From its code:
> # If the job is disabled and is not currently running, the job is not started or restarted.
> # However, if the job is disabled but has been forced into the running state, we *do* stop
> # and restart it since this is expected behaviour for the admin who forced the start.
> # We don't autostart static units either.
>
> I filed https://bugs.launchpad.net/ubuntu/+source/nfs-utils/+bug/1928259 and https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=988430.
>
> Some ideas I'm considering:
> a) just do a systemctl start before the #DEBHELPER# marker, like this:
> --- a/debian/nfs-common.postinst
> +++ b/debian/nfs-common.postinst
> @@ -43,6 +43,10 @@ case "$1" in
> if [ -f /lib/init/rw/sendsigs.omit.d/statd ]; then
> mv /lib/init/rw/sendsigs.omit.d/statd /run/sendsigs.omit.d/statd
> fi
> +
> + # always "start" nfs-utils.service, so package upgrades will restart it,
> + # see LP: #1928259
> + systemctl start nfs-utils.service > /dev/null || true
> ;;
> esac
>
>
> b) Don't use dh_systemd_* in d/rules for nfs-utils.service, and do my own handling in d/nfs-common.postinst using systemctl directly, instead of deb-systemd-invoke, i.e., something like (also untested):
> diff --git a/debian/nfs-common.postinst b/debian/nfs-common.postinst
> index f709d53..feb375a 100644
> --- a/debian/nfs-common.postinst
> +++ b/debian/nfs-common.postinst
> @@ -43,11 +43,19 @@ case "$1" in
> if [ -f /lib/init/rw/sendsigs.omit.d/statd ]; then
> mv /lib/init/rw/sendsigs.omit.d/statd /run/sendsigs.omit.d/statd
> fi
> -
> - # always "start" nfs-utils.service, so package upgrades will restart it,
> - # see LP: #1928259
> - systemctl start nfs-utils.service > /dev/null || true
> ;;
> esac
>
> +if [ "$1" = "configure" ] || [ "$1" = "abort-upgrade" ] || [ "$1" = "abort-deconfigure" ] || [ "$1" = "abort-remove" ] ; then
> + if [ -d /run/systemd/system ]; then
> + systemctl --system daemon-reload >/dev/null || true
> + if [ -n "$2" ]; then
> + _dh_action=restart
> + else
> + _dh_action=start
> + fi
> + systemctl $_dh_action 'nfs-utils.service' >/dev/null || true
> + fi
> +fi
> +
> #DEBHELPER#
> diff --git a/debian/rules b/debian/rules
> index 8bb2f25..3259d3b 100755
> --- a/debian/rules
> +++ b/debian/rules
> @@ -57,7 +57,6 @@ binary-arch: build
> dh_systemd_enable -p nfs-common nfs-client.target
> dh_systemd_enable -p nfs-kernel-server nfs-server.service
> dh_installinit -pnfs-common -R
> - dh_systemd_start -p nfs-common --restart-after-upgrade nfs-utils.service
> dh_systemd_start -p nfs-kernel-server --restart-after-upgrade nfs-server.service
> install -m 0755 debian/nfs-kernel-server.init debian/nfs-kernel-server/etc/init.d/nfs-kernel-server
> install -m 0644 debian/nfs-common.bugcontrol debian/nfs-common/usr/share/bug/nfs-common/control
>
> I just grabbed the bits that debhelper added for the dh_systemd_start line I removed, and replaced deb-systemd-invoke with systemctl
>
>
>
> Any other ideas?
>
> --
> ubuntu-devel mailing list
> ubuntu-devel@lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel