> On Sun, Jul 09, 2023 at 04:28:42AM +0200, Heinrich Schuchardt wrote:
>>>> Benjamin Drung <bdrung@ubuntu.com> schrieb am Sa., 8. Juli 2023, 02:19:
>
>>>>> Hi all,
>
>>>>> a year ago we changed the default compression and level for the
>>>>> initramfs to zstd -1. This fixed the very slow creation times on
>>>>> development boards (see bug #1958148), but that leads to bigger
>>>>> initramfs sizes that triggered other bugs (like bug #1842320).
>>>>> Big initramfs sizes can also fill up small sized /boot partitions easily
>>>>> (grooming the 850 initramfs-tools bugs revealed several such reports).
>
>>>>> Using xz -9 would give very good compression, but it takes very long
>>>>> (especially on slow development boards) and a lot of memory (good luck
>>>>> on Raspberry Pis with small memory like Pi Zeros).
>
>>>>> I propose following approach to address the drawback: Create cpio
>>>>> archives (compressed with xz -9) for the kernel modules and firmware
>>>>> files when building the kernel/firmware Debian package. Then ship those
>>>>> cpio archives in the package (or in a separate binary package). Then the
>>>>> CPU load it put on the builders. The cpio archives would contain the
>>>>> modules for MODULES=most.
>
>>>>> mkinitramfs will then look for those cpio archives and uses those in
>>>>> case they are present. Such a initramfs would look like this:
>
>>>>> * AMD/Intel microcode cpio archive (on amd64)
>>>>> * main cpio archive compressed with zstd -1
>>>>> * kernel modules from the Debian package compressed with xz -9
>>>>> * firmware files from the Debian package compressed with xz -9
>
>>>>> After working on initramfs-tools as part my day job, my fingers were
>>>>> itching and I had to create a quick and dirty draft in my free night
>>>>> time. You can find the result of the last two hours in [1]. This draft
>>>>> has a mkinitramfs-kernel script that creates a cpio archive containing
>>>>> the kernel modules and firmware (that needs to be split later on).
>
>>>>> The lunar test result on my AMD Ryzen 7 5700G look promising: Building
>>>>> 6.2.0-24-generic-modules-most.cpio.xz takes around 90 seconds and is
>>>>> 54.9 MiB in size. Creating the initramfs speeds up from around 8.7
>>>>> seconds to 3.5 seconds (saves 60 %). The size reduces from 133.1 MiB to
>>>>> 80.7 MiB (saves 39.4 %). So the boot needs 52.4 MiB less, but
>>>>> /lib/modules need 54.9 MiB for the cpio archive.
>
>>>>> The drawback is that building the kernel would take longer, the package
>>>>> takes more space on the archive and mirrors, and downloading them could
>>>>> take longer on slow connections.
>
>>>>> Implementing my proposal would be relative easy for initramfs-tools, but
>>>>> would mean some work for the kernel team.
>
>>>>> What do you think?
>
>> Will the user still be able to add further modules and will machine specific
>> configuration files (e.g. for booting from iSCSI) still be included into the
>> initrd?
>
> I think a robust implementation of this on the initramfs-tools side looks
> like:
>
> - identify all the contents that belong in the initramfs
> - among those contents, find all zstd-compressed files, if any, and store
> them in an uncompressed initramfs
> - put the rest of the contents in a compressed initramfs
>
> This would be compatible with kernel packages whether they are changed to
> ship zstd-compressed modules or not and allow for a smooth transition.
>
>
I built a kernel with CONFIG_MODULE_COMPRESS_ZSTD=y.
update-initramfs cannot find a module specified in
/etc/initramfs-tools/conf.d/modules_list.conf due to the .zstd extension.
Modules are uncompressed before being added to the initrd.
We would need an updated initramfs-tools package for evaluating that path.
Best regards
Heinrich
--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
