@nephros, do I understand correctly that you are suggesting to put
%define _binary_payload w2.xzdio
in spec files to set the compression to xz -2
(as it might have been before)?
Notes:
-
xz -2
is slow on decompression, but better thanbzip2 -9
(w9.bzdio
). - The only real alternative is to use
gzip
by%define _binary_payload w9.gzdio
(=gzip -9
): Slightly worse compression ratio, similar compression speed, but much faster decompression. OTOH, SailfishOS 2.2.1 providesxz / liblzma 5.0.4
(released 22 June 2012), so I assume it has been in SailfishOS since its inception (first releases 0.99.x.y in November 2013). -
xz
's default compression level 6 is very slow, memory- and CPU-intensive for both compression and decompression, and its levels {7,8,9} extremely so (even more thanzstd
's “ultra”-levels {20,21,22}). Seexz
's man-page, sections “Memory usage” and “-0 … -9” for details (!). - Do not set
%define _source_payload w2.xzdio
or something similar (as often suggested), because “SRPM payload compression should stay at gzip (there’s almost no benefit in changing the compression, because SRPM’s contents is compressed already)”. This also implies that any%define _<*>_payload w<?>.gzdio
continues to work fine (although only9
makes sense for substituting<?>
).
References:
- Excellent documentation with background information for Fedora 30→31 switching from
xz -2
tozstd -19
: Changes/Switch RPMs to zstd compression - Fedora Project Wiki- How to test an installed package (requires RPM ≥ 4.14.0, which is in SailfishOS since at least v2.2.1):
rpm -q --qf "%{PAYLOADCOMPRESSOR} %{PAYLOADFLAGS}\n" <package>
- IMO
zstd -15
would have been slightly more desirable: Much faster compression speed, faster decompression, only slightly worse compression ratio; this can be achieved by setting%define _binary_payload w15.zstdio
in a spec file for SailfishOS ≥ 4.4.0 (requires the corresponding SDK release for building). - BTW, RHEL 8→9 followed this switch, as SailfishOS 4.4.0→4.5.0 does, Ubuntu 22.04 did and Arch Linux did in January 2020.
- But
zstd
/libzstd
is not available for Fedora < 28, RHEL < 8.2, *buntu < 18.10 etc.; I assume it has been introduced into SailfishOS only recently (v4.4.0?). BTW, Linux integrateszstd
for kernel space users (no, this does not address persons ) since 4.14 (November 2017). - Using
xz -2
orbzip2 -9
compresses only slightly better thangz -9
at roughly the same compression speed but much slower decompression; butgzip
's default6
is compressing much quicker and usually yields the same compression ratio: Now I remember why I stubbornly kept usingtar -cvzf <tarball>
and never switched to anything else, also for compatibility’s sake. This is also the reason why Debian 9 / Ubuntu 18.04 started usingxz / lzma
compression for packages and usedgzip
prior to that. OpenWRT and other embedded Linux distributions continue to usegzip
compression for their packages due to its lightness (WRT CPU, memory and decompression speed).
- How to test an installed package (requires RPM ≥ 4.14.0, which is in SailfishOS since at least v2.2.1):
- Need new option to control RPM compression level · Issue #1675 · jordansissel/fpm · GitHub
- 1715799 – Re-enable zstd compression support in RHEL-8 rpm
- https://github.com/eclipse-archived/packagedrone/pull/133#discussion_r228995829
- Historic (on RHEL 4), but informative: rpm - rpmbuild change compression format - Stack Overflow
Can someone please try to install an RPM package built with the Sailfish-SDK for 4.5.0 (or the sailfishos-4.5.0 DoD repository at the SailfishOS-OBS) on SailfishOS ≤ 4.3.0, to ensure that nothing else changed. It must emit at least rpmlib(PayloadIsZst) is needed by <package name>-<version>-<release>.<arch>
, but any other rpmlib(…) is needed by <…>
would be something to also take into account.
Background: With the switch from w9.gzdio
to w2.xzdio
in RHEL 5→6, also the filedigest
was altered from md5
(to sha1
, IIRC), see here (it is historic as for RHEL 5→6, the write-up is erratic, the suggestions flawed, it is just to provide a reference and evidence what happened).