> On Sat, Jul 08, 2023 at 02:18:52AM +0200, Benjamin Drung wrote:
> > Using xz -9 would give very good compression, but it takes very long
> > (especially on slow development boards) and a lot of memory (good luck
> > on Raspberry Pis with small memory like Pi Zeros).
>
> Please also test decompression times for the various algorithms. I have
> a special dislike for xz after seeing how long it takes even decent
> machines to unpack Debian packages that were compressed with xz.
>
> It's hard to know how exactly to balance compress and decompress times,
> different users will have different ratios of booting vs installing
> updates.
There is a little-know but very interesting property of LZMA: its
decompression speed does not depend on the uncompressed size but only on
the compressed size. What this means is that if you compress a 100MB
down to 20MB, it will take roughly twice as long to decompress than if
you compress it down to 10MB. In other words, higher compression means
faster decompression.
There are two main ways to improve compression ratio with xz: use a
large dictionary or use a higher compression level (which increases the
dictionary level too). The drawback to larger dictionary is larger
memory usage: during compression roughly 10*${dict_size} is needed and
during decompression, it is roughtly ${dict_size}. With xz -9's 64MB
dictionary, you would get ~65MB for decompression and 674MB for
compression (you can run "xz -9vv" to get that information).
As such it is worth deciding what is our max memory usage during
compression and decompression. I think we can afford to use much more
than 64MB of memory when decompressing initrds and that the builders
have enough memory too.
This might also apply to some of zstd's algorithm but I can't guarantee
it because while I'm at least somewhat familiar with xz, zstd is much
more complex and AFAIU uses different algorithms depending on the
compression level that is used.
--
Adrien
--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
