From: Dima Pasechnik Subject: Re: numpy 1.26.4 (python 3.12 and cython 3 support) To: ports@openbsd.org Date: Tue, 27 Aug 2024 17:28:41 +0100 On 27 August 2024 17:03:10 BST, Stuart Henderson wrote: >On 2024/08/21 21:32, Daniel Dickman wrote: >> >> >> On Sat, 17 Aug 2024, Theo Buehler wrote: >> >> > On Tue, Aug 13, 2024 at 06:20:27PM -0400, Daniel Dickman wrote: >> > > Here's a possible update of numpy to 1.26.4. >> > > >> > > The build switches to meson in this release. And this release is the first >> > > one to support Python 3.12 and also the first to officialy support Cython >> > > 3. >> > > >> > > This should probably be put into bulks on multiple amd64 and other archs >> > > before being committed. >> > >> > After working around a debug package issue in numpy itself, this >> > completed a bulk without fallout. Further testing on other >> > architectures should probably wait until this and issues with >> > regress is sorted. >> > >> >> Thanks to feedback from tb@, new diff below with 2 changes: >> >> - add "DWZ=:" to fix the build of debug packages, this is an issue on both >> Python 3.11 and 3.12 likely due to 128 bit math. >> >> - disable tests. The regress tests work fine under Python 3.12 but fail >> under Python 3.11 due to what looks to me like incomplete refactoring >> of code after numpy.distutils was removed from numpy. >> >> Believe tb@ ran a bulk on amd64 with no issues, so !amd64 bulks would be >> very useful at this point. > >Bulk builds on the archs which are most likely to have problems is >really unlikely to happen - few people have the type of machines >suitable for running them in any reasonable timeframe. > >Results from building and "make test" on other archs (especially >sparc64) are easier and would be useful. > >The extra patches below fix things so tests can run on 3.11 (remove >NO_TEST from Makefile too). "make test" on amd64 Py 3.11 results below >the diff (summary: 31 failed, 35649 passed, 2372 skipped, 1305 >deselected, 33 xfailed, 1 xpassed, 21 warnings in 207.79s"; I'll run >it on aarch64 when my machine's free and reply if there are many >differences. Maybe i386 too when the current bulk finishes (a day or >two). > > >Index: patches/patch-numpy__pytesttester_py >=================================================================== >RCS file: patches/patch-numpy__pytesttester_py >diff -N patches/patch-numpy__pytesttester_py >--- /dev/null 1 Jan 1970 00:00:00 -0000 >+++ patches/patch-numpy__pytesttester_py 27 Aug 2024 15:50:18 -0000 >@@ -0,0 +1,14 @@ >+1.26.4 no longer has numpy.distutils but tests still reference it >+ >+Index: numpy/_pytesttester.py >+--- numpy/_pytesttester.py.orig >++++ numpy/_pytesttester.py >+@@ -135,7 +135,7 @@ class PytestTester: >+ # offset verbosity. The "-q" cancels a "-v". >+ pytest_args += ["-q"] >+ >+- if sys.version_info < (3, 12): >++ if sys.version_info < (3, 11): >+ with warnings.catch_warnings(): >+ warnings.simplefilter("always") >+ # Filter out distutils cpu warnings (could be localized to >Index: patches/patch-numpy_tests_test_public_api_py >=================================================================== >RCS file: patches/patch-numpy_tests_test_public_api_py >diff -N patches/patch-numpy_tests_test_public_api_py >--- /dev/null 1 Jan 1970 00:00:00 -0000 >+++ patches/patch-numpy_tests_test_public_api_py 27 Aug 2024 15:50:18 -0000 >@@ -0,0 +1,41 @@ >+1.26.4 no longer has numpy.distutils but tests still reference it >+ >+Index: numpy/tests/test_public_api.py >+--- numpy/tests/test_public_api.py.orig >++++ numpy/tests/test_public_api.py >+@@ -159,7 +159,7 @@ PUBLIC_MODULES = ['numpy.' + s for s in [ >+ "typing.mypy_plugin", >+ "version" # Should be removed for NumPy 2.0 >+ ]] >+-if sys.version_info < (3, 12): >++if sys.version_info < (3, 11): >+ PUBLIC_MODULES += [ >+ 'numpy.' + s for s in [ >+ "distutils", >+@@ -240,7 +240,7 @@ PRIVATE_BUT_PRESENT_MODULES = ['numpy.' + s for s in [ >+ "random.bit_generator", >+ "testing.print_coercion_tables", >+ ]] >+-if sys.version_info < (3, 12): >++if sys.version_info < (3, 11): >+ PRIVATE_BUT_PRESENT_MODULES += [ >+ 'numpy.' + s for s in [ >+ "distutils.armccompiler", >+@@ -335,7 +335,7 @@ SKIP_LIST = [ >+ "numpy.core.generate_numpy_api", >+ "numpy.core.umath_tests", >+ ] >+-if sys.version_info < (3, 12): >++if sys.version_info < (3, 11): >+ SKIP_LIST += ["numpy.distutils.msvc9compiler"] >+ >+ >+@@ -380,7 +380,7 @@ SKIP_LIST_2 = [ >+ 'numpy.matlib.ctypeslib', >+ 'numpy.matlib.ma', >+ ] >+-if sys.version_info < (3, 12): >++if sys.version_info < (3, 11): >+ SKIP_LIST_2 += [ >+ 'numpy.distutils.log.sys', >+ 'numpy.distutils.log.logging', >Index: patches/patch-numpy_typing_tests_test_isfile_py >=================================================================== >RCS file: patches/patch-numpy_typing_tests_test_isfile_py >diff -N patches/patch-numpy_typing_tests_test_isfile_py >--- /dev/null 1 Jan 1970 00:00:00 -0000 >+++ patches/patch-numpy_typing_tests_test_isfile_py 27 Aug 2024 15:50:18 -0000 >@@ -0,0 +1,14 @@ >+1.26.4 no longer has numpy.distutils but tests still reference it >+ >+Index: numpy/typing/tests/test_isfile.py >+--- numpy/typing/tests/test_isfile.py.orig >++++ numpy/typing/tests/test_isfile.py >+@@ -21,7 +21,7 @@ FILES = [ >+ ROOT / "random" / "__init__.pyi", >+ ROOT / "testing" / "__init__.pyi", >+ ] >+-if sys.version_info < (3, 12): >++if sys.version_info < (3, 11): >+ FILES += [ROOT / "distutils" / "__init__.pyi"] >+ >+ > >$ make test >===> Regression tests for py3-numpy-1.26.4 >cd /pobj/py-numpy-1.26.4-python3 && CC=cc PYTHONUSERBASE= PORTSDIR="/usr/ports" LIBTOOL="/usr/bin/libtool" CCACHE_DIR=/usr/obj/ports/.ccache CCACHE_SLOPPINESS=pch_defines,time_macros PATH='/pobj/py-numpy-1.26.4-python3/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11R6/bin' PREFIX='/usr/local' LOCALBASE='/usr/local' X11BASE='/usr/X11R6' CFLAGS='-O2 -pipe -g' TRUEPREFIX='/usr/local' DESTDIR='' HOME='/py-numpy-1.26.4_writes_to_HOME' PICFLAG="-fpic" BINGRP=bin BINOWN=root BINMODE=755 NONBINMODE=644 DIRMODE=755 INSTALL_COPY=-c INSTALL_STRIP= MANGRP=bin MANOWN=root MANMODE=644 BSD_INSTALL_PROGRAM="/pobj/py-numpy-1.26.4-python3/bin/install -c -m 755" BSD_INSTALL_SCRIPT="/pobj/py-numpy-1.26.4-python3/bin/install -c -m 755" BSD_INSTALL_DATA="/pobj/py-numpy-1.26.4-python3/bin/install -c -m 644" BSD_INSTALL_MAN="/pobj/py-numpy-1.26.4-python3/bin/install -c -m 644" BSD_INSTALL_PROGRAM_DIR="/pobj/py-numpy-1.26.4-python3/bin/install -d -m 755" BSD_INSTALL_SCRIPT_DIR="/pobj/py-numpy-1.26.4-python3/bin/install -d -m 755" BSD_INSTALL_DATA_DIR="/pobj/py-numpy-1.26.4-python3/bin/install -d -m 755" BSD_INSTALL_MAN_DIR="/pobj/py-numpy-1.26.4-python3/bin/install -d -m 755" /usr/local/bin/python3.11 -c 'import numpy ; numpy.test(verbose=2)' >NumPy version 1.26.4 >NumPy relaxed strides checking option: True >NumPy CPU features: SSE SSE2 SSE3 SSSE3* SSE41* POPCNT* SSE42* AVX* F16C* FMA3* AVX2* AVX512F? AVX512CD? AVX512_KNL? AVX512_KNM? AVX512_SKX? AVX512_CLX? AVX512_CNL? AVX512_ICL? >============================= test session starts ============================== >platform openbsd7 -- Python 3.11.9, pytest-8.2.0, pluggy-1.5.0 >benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000) >rootdir: /usr/obj/ports/py-numpy-1.26.4-python3 >plugins: subtests-0.12.1, benchmark-4.0.0, flaky-3.8.1, filter-subpackage-0.2.0, timeout-1.4.2, typeguard-4.3.0, hypothesis-6.47.1, anyio-3.7.1, cov-5.0.0, mock-3.14.0, doctestplus-1.2.1 >collected 39391 items / 1305 deselected / 38086 selected > >array_api/tests/test_array_object.py ........................ [ 0%] >array_api/tests/test_creation_functions.py .............. [ 0%] >array_api/tests/test_data_type_functions.py ..... [ 0%] >array_api/tests/test_elementwise_functions.py .. [ 0%] >array_api/tests/test_indexing_functions.py .... [ 0%] >array_api/tests/test_manipulation_functions.py ... [ 0%] >array_api/tests/test_set_functions.py .. [ 0%] >array_api/tests/test_sorting_functions.py .... [ 0%] >array_api/tests/test_validation.py ... [ 0%] >compat/tests/test_compat.py . [ 0%] >core/tests/test__exceptions.py ........... [ 0%] >core/tests/test_abc.py ..... [ 0%] >core/tests/test_api.py ................................................. [ 0%] >........... [ 0%] >core/tests/test_argparse.py ..... [ 0%] >core/tests/test_array_coercion.py ...................................... [ 0%] >.......................x................................................ [ 0%] >........................................................................ [ 0%] >.......................... [ 0%] >core/tests/test_arraymethod.py ........................................ [ 1%] >core/tests/test_arrayprint.py ..x....................................... [ 1%] >..................... [ 1%] >core/tests/test_casting_floatingpoint_errors.py ........................ [ 1%] >........................................................................ [ 1%] >........................................................................ [ 1%] >............... [ 1%] >core/tests/test_casting_unittests.py ................................... [ 1%] >........................................................................ [ 1%] >........................................................................ [ 2%] >........................................................................ [ 2%] >................................................................ [ 2%] >core/tests/test_conversion_utils.py ........................... [ 2%] >core/tests/test_cpu_dispatcher.py . [ 2%] >core/tests/test_cpu_features.py .......ssss [ 2%] >core/tests/test_custom_dtypes.py ................................... [ 2%] >core/tests/test_cython.py ....... [ 2%] >core/tests/test_datetime.py ............................................ [ 2%] >........................................................................ [ 3%] >........................................................................ [ 3%] >...........................................x..........x..x..........x... [ 3%] >. [ 3%] >core/tests/test_defchararray.py ........................................ [ 3%] >................................................. [ 3%] >core/tests/test_deprecations.py ........................................ [ 3%] >........................................................................ [ 3%] >........... [ 3%] >core/tests/test_dlpack.py .............................................. [ 4%] >............. [ 4%] >core/tests/test_dtype.py ............................................... [ 4%] >................................................ssss.................... [ 4%] >........................................................................ [ 4%] >........................................................................ [ 4%] >........................................................................ [ 4%] >........................................................................ [ 5%] >........................................................................ [ 5%] >........................................................................ [ 5%] >........................................................................ [ 5%] >........................................................................ [ 5%] >........................................................................ [ 6%] >........................................................................ [ 6%] >........................................................................ [ 6%] >........................................................................ [ 6%] >................................................. [ 6%] >core/tests/test_einsum.py .............................................. [ 6%] >..................... [ 6%] >core/tests/test_errstate.py .... [ 7%] >core/tests/test_extint128.py ............ [ 7%] >core/tests/test_function_base.py ..................................... [ 7%] >core/tests/test_getlimits.py ................ [ 7%] >core/tests/test_half.py ..................................... [ 7%] >core/tests/test_hashtable.py ......... [ 7%] >core/tests/test_indexerrors.py ........ [ 7%] >core/tests/test_indexing.py ............................................ [ 7%] >....................................... [ 7%] >core/tests/test_item_selection.py ...................................... [ 7%] >........................................................................ [ 7%] >........................................................................ [ 8%] >........................................................................ [ 8%] >............ [ 8%] >core/tests/test_limited_api.py . [ 8%] >core/tests/test_longdouble.py .Fsssss.s......ssss.sss...........s. [ 8%] >core/tests/test_machar.py . [ 8%] >core/tests/test_mem_overlap.py .................. [ 8%] >core/tests/test_mem_policy.py .....s.... [ 8%] >core/tests/test_memmap.py ................... [ 8%] >core/tests/test_multiarray.py .......................................... [ 8%] >........................................................................ [ 8%] >.......................xx............................................... [ 8%] >........................................................................ [ 9%] >........................................................................ [ 9%] >...........................................s............................ [ 9%] >........................................................................ [ 9%] >........................................................................ [ 9%] >........................................................................ [ 10%] >........................................................................ [ 10%] >........................................................................ [ 10%] >........................................................................ [ 10%] >........................................................................ [ 10%] >........................................................................ [ 11%] >........................................................................ [ 11%] >........................................................................ [ 11%] >........................................................................ [ 11%] >........................................................................ [ 11%] >........................................................................ [ 11%] >........................................................................ [ 12%] >........................................................................ [ 12%] >........................................................................ [ 12%] >........................................................................ [ 12%] >........................................................................ [ 12%] >........................................................................ [ 13%] >........................................................................ [ 13%] >........................................................................ [ 13%] >........................................................................ [ 13%] >........................................................................ [ 13%] >........................................................................ [ 14%] >........................................................................ [ 14%] >........................................................................ [ 14%] >........................................................................ [ 14%] >........................................................................ [ 14%] >........................................................................ [ 14%] >........................................................................ [ 15%] >........................................................................ [ 15%] >........................................................................ [ 15%] >........................................................................ [ 15%] >........................................................................ [ 15%] >........................................................................ [ 16%] >........................................................................ [ 16%] >........................................................................ [ 16%] >........................................................................ [ 16%] >........................................................................ [ 16%] >........................................................................ [ 17%] >........................................................................ [ 17%] >........................................................................ [ 17%] >........................................................................ [ 17%] >........................................................................ [ 17%] >........................................................................ [ 18%] >........................................................................ [ 18%] >........................................................................ [ 18%] >........................................................................ [ 18%] >........................................................................ [ 18%] >........................................................................ [ 18%] >........................................................................ [ 19%] >........................................................................ [ 19%] >........................................................................ [ 19%] >........................................................................ [ 19%] >........................................................................ [ 19%] >........................................................................ [ 20%] >........................................................................ [ 20%] >........................................................................ [ 20%] >........................................................................ [ 20%] >........................................................................ [ 20%] >........................................................................ [ 21%] >........................................................................ [ 21%] >........................................................................ [ 21%] >........................................................................ [ 21%] >........................................................................ [ 21%] >........................................................................ [ 21%] >........................................................................ [ 22%] >........................................................................ [ 22%] >........................................................................ [ 22%] >........................................................................ [ 22%] >........................................................................ [ 22%] >........................................................................ [ 23%] >........................................................................ [ 23%] >........................................................................ [ 23%] >........................................................................ [ 23%] >........................................................................ [ 23%] >........................................................................ [ 24%] >........................................................................ [ 24%] >........................................................................ [ 24%] >........................................................................ [ 24%] >........................................................................ [ 24%] >........................................................................ [ 25%] >........................................................................ [ 25%] >........................................................................ [ 25%] >........................................................................ [ 25%] >........................................................................ [ 25%] >........................................................................ [ 25%] >........................................................................ [ 26%] >........................................................................ [ 26%] >........................................................................ [ 26%] >........................................................................ [ 26%] >........................................................................ [ 26%] >........................................................................ [ 27%] >........................................................................ [ 27%] >........................................................................ [ 27%] >........................................................................ [ 27%] >........................................................................ [ 27%] >........................................................................ [ 28%] >........................................................................ [ 28%] >........................................................................ [ 28%] >........................................................................ [ 28%] >........................................................................ [ 28%] >........................................................................ [ 28%] >........................................................................ [ 29%] >........................................................................ [ 29%] >........................................................................ [ 29%] >........................................................................ [ 29%] >........................................................................ [ 29%] >........................................................................ [ 30%] >........................................................................ [ 30%] >........................................................................ [ 30%] >........................................................................ [ 30%] >........................................................................ [ 30%] >........................................................................ [ 31%] >........................................................................ [ 31%] >........................................................................ [ 31%] >........................................................................ [ 31%] >........................................................................ [ 31%] >........................................................................ [ 32%] >........................................................................ [ 32%] >........................................................................ [ 32%] >........................................................................ [ 32%] >................................ [ 32%] >core/tests/test_nditer.py .............................................. [ 32%] >........................................................................ [ 32%] >........................................................................ [ 33%] >........................................................................ [ 33%] >........................................................................ [ 33%] >........................................................................ [ 33%] >........................................................................ [ 33%] >........................................................................ [ 34%] >........................................................................ [ 34%] >........................................................................ [ 34%] >........................................................................ [ 34%] >.................................... [ 34%] >core/tests/test_nep50_promotions.py ................s............... [ 34%] >core/tests/test_numeric.py ..............................x...x.......... [ 34%] >..................s..s.................................................. [ 35%] >........................................................................ [ 35%] >...........................................................X............ [ 35%] >........................................................................ [ 35%] >.................... [ 35%] >core/tests/test_numerictypes.py ........................................ [ 35%] >........................................................................ [ 36%] >...... [ 36%] >core/tests/test_numpy_2_0_compat.py ... [ 36%] >core/tests/test_overrides.py ........................................... [ 36%] >........................................................................ [ 36%] >....................... [ 36%] >core/tests/test_print.py .....................F [ 36%] >core/tests/test_protocols.py .. [ 36%] >core/tests/test_records.py ......................................... [ 36%] >core/tests/test_regression.py .......................................... [ 36%] >........................................................................ [ 36%] >........................................................................ [ 37%] >........................................................................ [ 37%] >.........s.............................................................. [ 37%] >........................................................................ [ 37%] >............. [ 37%] >core/tests/test_scalar_ctors.py ........................................ [ 37%] >........................................................................ [ 38%] >.................................................. [ 38%] >core/tests/test_scalar_methods.py ............................s......... [ 38%] >........................................................................ [ 38%] >. [ 38%] >core/tests/test_scalarbuffer.py ........................................ [ 38%] >...................................... [ 38%] >core/tests/test_scalarinherit.py ..... [ 38%] >core/tests/test_scalarmath.py .........................s................ [ 38%] >........................................................................ [ 38%] >........................................................................ [ 39%] >......................ssssss............................................ [ 39%] >........................................................................ [ 39%] >...... [ 39%] >core/tests/test_scalarprint.py ....Fs [ 39%] >core/tests/test_shape_base.py .......................................... [ 39%] >........................................................................ [ 39%] >........................................................................ [ 40%] >........................ [ 40%] >core/tests/test_simd.py ................................................ [ 40%] >........................................................................ [ 40%] >........................................................................ [ 40%] >........................................................................ [ 40%] >........................................................................ [ 40%] >........................................................................ [ 41%] >........................................................................ [ 41%] >........................................................................ [ 41%] >........................................................................ [ 41%] >........................................................................ [ 41%] >........................................................................ [ 42%] >........................................................................ [ 42%] >........................................................................ [ 42%] >........................................................................ [ 42%] >......... Can you build and test scipy using this numpy port? Dima