From: Marco van Hulten Subject: Re: NEW: archivers/libaec To: Martin Reindl Cc: ports@openbsd.org Date: Wed, 30 Jul 2025 11:34:47 +0200 On Tue, 29 Jul 2025 20:01:53 +0200 Martin Reindl wrote: > Am 29.07.25 um 16:50 schrieb Marco van Hulten: > > On Tue, 29 Jul 2025 14:16:31 +0200 Martin Reindl wrote: > >> Am 2025-07-29 10:58, schrieb Marco van Hulten: > >>> On Tue, 29 Jul 2025 08:42:26 +0200 Martin Reindl wrote: > >>>> here is the update for hdf5 to use szip+aec as I/O filter. > >>>> Examples are moved to an own project. > >>>> > >>>> Looking for more tests + feedback. > >>> > >>> [...] > > > > Moreover, I tested Szip support through libaec of hdf5 with cdo by > > reading a grib file. This works now with the new version! > > > > Nonetheless, I get warnings: > > > > marco@foobar:~/data$ cdo -infon 2020010100_sf-regular.grb > > cdo:/usr/local/lib/libhdf5.so.7.0: /usr/local/lib/libhdf5.so.8.0 : WARNING: symbol(H5E_stack_g) size mismatch, relink your program > > cdo:/usr/local/lib/libhdf5.so.7.0: /usr/local/lib/libhdf5.so.8.0 : WARNING: symbol(H5_lib_vers_info_g) size mismatch, relink your program > > cdo:/usr/local/lib/libhdf5.so.7.0: /usr/local/lib/libhdf5.so.8.0 : WARNING: symbol(H5build_settings) size mismatch, relink your program > > -1 : Date Time Level Gridsize Miss : Minimum Mean Maximum : Parameter name > > 1 : 2020-01-01 00:00:00 0 2882400 974786 : 0.0000 0.16137 1.0000 : var31 > > [...] > > cdo infon: Processed 89354400 values from 31 variables over 1 timestep [5.99s 44MB] > > > > I thought that if I uninstall the previous hdf5, then build and install > > the new one and then clean and build cdo, it would link to the new > > libhdf5, but apparently it does not. What is wrong here? > > You linked with the old library. > > $ ./src/cdo -infon test/data/psl_DJF_anom.grb > > > -1 : Date Time Level Gridsize Miss : Minimum > Mean Maximum : Parameter name > 1 : 1980-01-31 23:52:00 0 2240 0 : -1238.4 > -70.548 1005.8 : var151 > 2 : 1981-01-31 23:52:00 0 2240 0 : -403.36 > -53.654 349.68 : var151 > 3 : 1982-01-31 23:52:00 0 2240 0 : -595.87 > 15.319 693.66 : var151 > 4 : 1983-01-31 23:52:00 0 2240 0 : -810.26 > 31.162 958.68 : var151 > 5 : 1984-01-31 23:52:00 0 2240 0 : -512.45 > 102.78 1016.9 : var151 > 6 : 1985-01-31 23:52:00 0 2240 0 : -824.82 > -11.434 575.68 : var151 > 7 : 1986-01-31 23:52:00 0 2240 0 : -584.45 > 171.28 1640.7 : var151 > 8 : 1987-01-31 23:52:00 0 2240 0 : -667.68 > -9.2706 773.39 : var151 > 9 : 1988-01-31 23:52:00 0 2240 0 : -1155.7 > -198.26 699.56 : var151 > 10 : 1989-01-31 23:52:00 0 2240 0 : -447.86 > 196.62 1080.1 : var151 > 11 : 1990-01-31 23:52:00 0 2240 0 : -929.19 > -100.58 426.71 : var151 > 12 : 1991-01-31 23:52:00 0 2240 0 : -1119.0 > 52.054 1309.9 : var151 > 13 : 1992-01-31 23:52:00 0 2240 0 : -954.14 > -90.602 619.79 : var151 > 14 : 1993-01-31 23:52:00 0 2240 0 : -607.19 > -31.790 537.97 : var151 > 15 : 1994-01-31 23:52:00 0 2240 0 : -570.08 > 51.584 732.51 : var151 > 16 : 1995-01-31 23:52:00 0 2240 0 : -375.38 > 67.094 1201.6 : var151 > 17 : 1996-01-31 23:52:00 0 2240 0 : -641.53 > 93.666 498.60 : var151 > 18 : 1997-01-31 23:52:00 0 2240 0 : -746.78 > -8.0068 859.88 : var151 > 19 : 1998-01-31 23:52:00 0 2240 0 : -621.87 > 102.84 1010.3 : var151 > 20 : 1999-01-31 23:52:00 0 2240 0 : -773.35 > 149.15 1672.2 : var151 > 21 : 2000-01-31 23:52:00 0 2240 0 : -657.09 > 41.559 810.98 : var151 > 22 : 2001-01-31 23:52:00 0 2240 0 : -1218.9 > -169.16 438.17 : var151 > 23 : 2002-01-31 23:52:00 0 2240 0 : -817.85 > 72.477 1606.7 : var151 > 24 : 2003-01-31 23:52:00 0 2240 0 : -799.78 > -110.61 357.97 : var151 > 25 : 2004-01-31 23:52:00 0 2240 0 : -1073.8 > -77.942 938.71 : var151 > 26 : 2005-01-31 23:52:00 0 2240 0 : -531.63 > -189.17 101.92 : var151 > 27 : 2006-01-31 23:52:00 0 2240 0 : -933.01 > 18.958 807.46 : var151 > 28 : 2007-01-31 23:52:00 0 2240 0 : -521.18 > 42.755 1036.7 : var151 > 29 : 2008-01-31 23:52:00 0 2240 0 : -394.27 > -88.275 254.01 : var151 > cdo infon: Processed 64960 values from 1 variable over 29 timesteps ( > 0.01s 6668KB ) > $ ldd src/cdo > src/cdo: > Start End Type Open Ref GrpRef Name > 00000deaff876000 00000deaffdb8000 exe 2 0 0 src/cdo > 00000ded6c7eb000 00000ded6c7f8000 rlib 0 1 0 > /usr/lib/libexecinfo.so.4.0 > 00000ded587a0000 00000ded587d1000 rlib 0 1 0 > /usr/lib/libm.so.10.1 > 00000ded91a80000 00000ded91b9e000 rlib 0 1 0 > /usr/lib/libc++.so.11.0 > 00000dedd3b6b000 00000dedd3bb3000 rlib 0 2 0 > /usr/lib/libc++abi.so.8.0 > 00000ded0a0a5000 00000ded0a0b2000 rlib 0 1 0 > /usr/lib/libpthread.so.28.0 > 00000deda0758000 00000deda0861000 rlib 0 1 0 > /usr/lib/libc.so.101.0 > 00000ded346ee000 00000ded346ee000 ld.so 0 1 0 /usr/libexec/ld.so > Great, I get the same stats and also on a Rocky Linux parallel system where there are many cdo users. Of course the tests, which are surely more exhaustive, succeed as well. marco@foobar:.../pobj/cdo-2.5.2$ ./build-amd64/src/cdo -infon cdo-2.5.2/test/data/psl_DJF_anom.grb cdo:/usr/local/lib/libhdf5.so.7.0: /usr/local/lib/libhdf5.so.8.0 : WARNING: symbol(H5E_stack_g) size mismatch, relink your program cdo:/usr/local/lib/libhdf5.so.7.0: /usr/local/lib/libhdf5.so.8.0 : WARNING: symbol(H5_lib_vers_info_g) size mismatch, relink your program cdo:/usr/local/lib/libhdf5.so.7.0: /usr/local/lib/libhdf5.so.8.0 : WARNING: symbol(H5build_settings) size mismatch, relink your program -1 : Date Time Level Gridsize Miss : Minimum Mean Maximum : Parameter name 1 : 1980-01-31 23:52:00 0 2240 0 : -1238.4 -70.548 1005.8 : var151 … marco@foobar:.../pobj/cdo-2.5.2$ ldd build-amd64/src/cdo build-amd64/src/cdo: Start End Type Open Ref GrpRef Name 000009987bf45000 000009987c942000 exe 2 0 0 build-amd64/src/cdo 0000099b37191000 0000099b373eb000 rlib 0 1 0 /usr/local/lib/libnetcdf.so.10.0 0000099a7d43b000 0000099a7d448000 rlib 0 2 0 /usr/lib/libexecinfo.so.4.0 0000099ad7b9d000 0000099ad7bc3000 rlib 0 1 0 /usr/local/lib/libhdf5_hl.so.7.0 0000099aa36a6000 0000099aa3a7a000 rlib 0 2 0 /usr/local/lib/libhdf5.so.8.0 0000099a82c5f000 0000099a82c64000 rlib 0 3 0 /usr/local/lib/libsz.so.0.0 0000099af1bb1000 0000099af1bbd000 rlib 0 4 0 /usr/local/lib/libaec.so.0.0 0000099aef6ff000 0000099aef71d000 rlib 0 10 0 /usr/lib/libz.so.7.1 0000099b47c90000 0000099b47cc1000 rlib 0 8 0 /usr/lib/libm.so.10.1 0000099b6d587000 0000099b6d5aa000 rlib 0 2 0 /usr/local/lib/libzip.so.4.2 0000099b40b98000 0000099b40bb0000 rlib 0 3 0 /usr/local/lib/libbz2.so.10.4 0000099b3705f000 0000099b3712f000 rlib 0 4 0 /usr/local/lib/libzstd.so.7.0 0000099b5b6e5000 0000099b5b6fa000 rlib 0 2 0 /usr/local/lib/libblosc.so.2.0 0000099a97755000 0000099a978be000 rlib 0 2 0 /usr/local/lib/libxml2.so.22.0 0000099b0db6d000 0000099b0dc4d000 rlib 0 2 0 /usr/local/lib/libcurl.so.26.32 0000099a9f74a000 0000099a9f77a000 rlib 0 3 0 /usr/local/lib/libnghttp3.so.3.2 0000099b5267f000 0000099b5268c000 rlib 0 3 0 /usr/local/lib/libngtcp2_crypto_quictls.so.0.2 0000099b14023000 0000099b14076000 rlib 0 4 0 /usr/local/lib/libngtcp2.so.4.1 0000099ae5951000 0000099ae59be000 rlib 0 4 0 /usr/lib/libssl.so.60.0 0000099a84f1e000 0000099a8516a000 rlib 0 6 0 /usr/lib/libcrypto.so.57.0 0000099b5ee6d000 0000099b5eea2000 rlib 0 3 0 /usr/local/lib/libnghttp2.so.0.23 0000099ac5460000 0000099ac546d000 rlib 0 7 0 /usr/lib/libpthread.so.28.0 0000099a7d97f000 0000099a7da9d000 rlib 0 3 0 /usr/lib/libc++.so.11.0 0000099b605bd000 0000099b60605000 rlib 0 4 0 /usr/lib/libc++abi.so.8.0 0000099a9d363000 0000099a9d46c000 rlib 0 1 0 /usr/lib/libc.so.101.0 0000099ab825b000 0000099ab8281000 rlib 0 1 0 /usr/local/lib/libhdf5_hl.so.6.0 0000099b6e1cd000 0000099b6e5f3000 rlib 0 2 0 /usr/local/lib/libhdf5.so.7.0 0000099b19f69000 0000099b19f9f000 rlib 0 3 0 /usr/local/lib/liblzma.so.2.4 0000099a95e80000 0000099a95eb3000 rlib 0 2 0 /usr/local/lib/liblz4.so.3.3 0000099ab399a000 0000099ab39a7000 rlib 0 2 0 /usr/local/lib/libsnappy.so.3.0 0000099b5eea2000 0000099b5efb4000 rlib 0 2 0 /usr/local/lib/libiconv.so.7.1 0000099afee9e000 0000099afee9e000 ld.so 0 1 0 /usr/libexec/ld.so Apparently cdo was linked both with the old and the new library. So I got rid of the old hdf5 (pkg_delete hdf5). After a clean build of netcdf, it complained that the old hdf5 was not there. I updated the version of netcdf to 4.9.3. Then the warning about library symbols was gone, as well as the libhdf5.so.7.0 ldd entry: marco@foobar:.../pobj/cdo-2.5.2$ ./build-amd64/src/cdo -infon cdo-2.5.2/test/data/psl_DJF_anom.grb -1 : Date Time Level Gridsize Miss : Minimum Mean Maximum : Parameter name 1 : 1980-01-31 23:52:00 0 2240 0 : -1238.4 -70.548 1005.8 : var151 … marco@foobar:.../pobj/cdo-2.5.2$ ldd build-amd64/src/cdo build-amd64/src/cdo: Start End Type Open Ref GrpRef Name 0000066e503e8000 0000066e50de5000 exe 2 0 0 build-amd64/src/cdo 00000671442bb000 00000671444ae000 rlib 0 1 0 /usr/local/lib/libnetcdf.so.10.0 00000670e0490000 00000670e049d000 rlib 0 2 0 /usr/lib/libexecinfo.so.4.0 000006705f67e000 000006705f6a4000 rlib 0 2 0 /usr/local/lib/libhdf5_hl.so.7.0 00000670ffaed000 00000670ffec1000 rlib 0 3 0 /usr/local/lib/libhdf5.so.8.0 000006713c62f000 000006713c634000 rlib 0 4 0 /usr/local/lib/libsz.so.0.0 00000670e3e89000 00000670e3e95000 rlib 0 5 0 /usr/local/lib/libaec.so.0.0 000006710e8b6000 000006710e8d4000 rlib 0 8 0 /usr/lib/libz.so.7.1 000006706f2de000 000006706f30f000 rlib 0 6 0 /usr/lib/libm.so.10.1 0000067055ea6000 0000067055ec9000 rlib 0 2 0 /usr/local/lib/libzip.so.4.2 00000670d3c16000 00000670d3c2e000 rlib 0 3 0 /usr/local/lib/libbz2.so.10.4 00000670f42d4000 00000670f43a4000 rlib 0 4 0 /usr/local/lib/libzstd.so.7.0 000006711c3e7000 000006711c3fc000 rlib 0 2 0 /usr/local/lib/libblosc.so.2.0 0000067074595000 00000670746fe000 rlib 0 2 0 /usr/local/lib/libxml2.so.22.0 000006708dba1000 000006708dc81000 rlib 0 2 0 /usr/local/lib/libcurl.so.26.32 000006705d2d6000 000006705d306000 rlib 0 3 0 /usr/local/lib/libnghttp3.so.3.2 00000670cca1c000 00000670cca29000 rlib 0 3 0 /usr/local/lib/libngtcp2_crypto_quictls.so.0.2 00000670887fa000 000006708884d000 rlib 0 4 0 /usr/local/lib/libngtcp2.so.4.1 00000670b7ba2000 00000670b7c0f000 rlib 0 4 0 /usr/lib/libssl.so.60.0 0000067146b79000 0000067146dc5000 rlib 0 6 0 /usr/lib/libcrypto.so.57.0 00000670637d3000 0000067063808000 rlib 0 3 0 /usr/local/lib/libnghttp2.so.0.23 00000670b01a4000 00000670b01b1000 rlib 0 7 0 /usr/lib/libpthread.so.28.0 000006705c283000 000006705c3a1000 rlib 0 3 0 /usr/lib/libc++.so.11.0 000006711f049000 000006711f091000 rlib 0 4 0 /usr/lib/libc++abi.so.8.0 00000670dee63000 00000670def6c000 rlib 0 1 0 /usr/lib/libc.so.101.0 00000670fc983000 00000670fc9b9000 rlib 0 3 0 /usr/local/lib/liblzma.so.2.4 0000067059bb9000 0000067059bec000 rlib 0 2 0 /usr/local/lib/liblz4.so.3.3 000006705d313000 000006705d320000 rlib 0 2 0 /usr/local/lib/libsnappy.so.3.0 00000670eaa64000 00000670eab76000 rlib 0 2 0 /usr/local/lib/libiconv.so.7.1 00000670aba3c000 00000670aba3c000 ld.so 0 1 0 /usr/libexec/ld.so There are many more libraries linked than in your case. How is it possible that your cdo reads the *.grb file without it being linked to hdf5? I imagine that I should have done something to make cdo work with netcdf-4.9.2 and that installing netcdf-4.9.3 was not strictly needed. Is 'make clean' not enough for the ports system to relink to existing libraries an forget about old ones? Apropos, for the updated netcdf, not all patches to the test files worked, so I skipped the patching (and have not done a make test). If you want, I could check which patch files are still needed and need adjustment, but please go ahead if you have the time. Thanks for the help and the porting! Marco