Monday, 12 March 2018

Re: zstd compression for packages

On Mon, Mar 12, 2018 at 11:06:11AM +0100, Julian Andres Klode wrote:
> Hey folks,
> We had a coding day in Foundations last week and Balint and Julian added support for zstd compression to dpkg [1] and apt [2].
> [1]
> [2]
> Zstd is a compression algorithm developed by Facebook that offers far
> higher decompression speeds than xz or even gzip (at roughly constant
> speed and memory usage across all levels), while offering 19 compression
> levels ranging from roughly comparable to gzip in size (but much faster)
> to 19, which is roughly comparable to xz -6:
> In our configuration, we run zstd at level 19. For bionic main amd64,
> this causes a size increase of about 6%, from roughly 5.6 to 5.9 GB.
> Installs speed up by about 10%, or, if eatmydata is involved, by up to
> 40% - user time generally by about 50%.
> Our implementations for apt and dpkg support multiple frames as used by
> pzstd, so packages can be compressed and decompressed in parallel
> eventually.

More links:

APT merge request:
dpkg patches:

I'd also like to talk a bit more about libzstd itself: The package is
currently in universe, but btrfs recently gained support for zstd,
so we already have a copy in the kernel and we need to MIR it anyway
for btrfs-progs.

debian developer - | - free software dev
ubuntu core developer i speak de, en

ubuntu-devel mailing list
Modify settings or unsubscribe at: