Index | Thread | Search

From:
Kirill A. Korinsky <kirill@korins.ky>
Subject:
archivers/xz: use RLIMIT_DATA as the default maximum memory limit
To:
Christian Weisgerber <naddy@openbsd.org>
Cc:
OpenBSD ports <ports@openbsd.org>
Date:
Mon, 08 Sep 2025 09:42:16 +0200

Download raw body.

Thread
  • Kirill A. Korinsky:

    archivers/xz: use RLIMIT_DATA as the default maximum memory limit

Christian,

I'd like to include a small patch for our archivers/xz.

It limits the defaul limit of the maximum availavle memory from 1/4 of RAM
to an actual RLIMIT_DATA, if it smaller.

Without this patch I can't unpack any of chromium on a machine with 256Gb
RAM when I run a make build MAKE_JOBS=64. It fails as:

xz: (stdin): Cannot allocate memory
tar: End of archive volume 1 reached


Ok?


Index: Makefile
===================================================================
RCS file: /home/cvs/ports/archivers/xz/Makefile,v
diff -u -p -r1.55 Makefile
--- Makefile	15 Apr 2025 19:34:02 -0000	1.55
+++ Makefile	7 Sep 2025 08:44:07 -0000
@@ -1,6 +1,7 @@
 COMMENT=	library and tools for XZ and LZMA compressed files
 
 VERSION=	5.8.1
+REVISION=	0
 DISTNAME=	xz-${VERSION}
 SHARED_LIBS=	lzma                 2.4      # 13.1
 CATEGORIES=	archivers
Index: patches/patch-src_xz_hardware_c
===================================================================
RCS file: patches/patch-src_xz_hardware_c
diff -N patches/patch-src_xz_hardware_c
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ patches/patch-src_xz_hardware_c	7 Sep 2025 15:01:09 -0000
@@ -0,0 +1,27 @@
+https://github.com/tukaani-project/xz/pull/196
+
+Index: src/xz/hardware.c
+--- src/xz/hardware.c.orig
++++ src/xz/hardware.c
+@@ -11,6 +11,7 @@
+ 
+ #include "private.h"
+ 
++#include <sys/resource.h>
+ 
+ /// Maximum number of worker threads. This can be set with
+ /// the --threads=NUM command line option.
+@@ -320,6 +321,13 @@ hardware_init(void)
+ 	// One Linux-specific suggestion is to use MemAvailable from
+ 	// /proc/meminfo as the starting point.
+ 	memlimit_mt_default = total_ram / 4;
++
++	struct rlimit rlp;
++	if (getrlimit(RLIMIT_DATA, &rlp) == 0
++	    && rlp.rlim_cur != RLIM_INFINITY
++	    && memlimit_mt_default > rlp.rlim_cur
++	)
++		memlimit_mt_default = rlp.rlim_cur;
+ 
+ #if SIZE_MAX == UINT32_MAX
+ 	// A too high value may cause 32-bit xz to run out of address space.



-- 
wbr, Kirill