From: byteskeptical Subject: Re: [NEW WIP]: net/monero (help needed) To: David Uhden Collado Cc: lucas@sexy.is, ports@openbsd.org Date: Tue, 27 Aug 2024 14:21:14 +0000 Reply-To: byteskeptical On Tue, Aug 20, 2024 at 07:03:15AM +0200, David Uhden Collado wrote: >>Hey, >> >>I've been working on and off in a port for Monero for quite some time. >>Thanks to your's and byteskeptical interest, I got back to it and now I >>have a working port attached here. >> > This looks great added the test dependencies back in and a version bump. >Honestly, I don't understand why people create ports for OpenBSD >without trying to get them added to the official ports tree. In your >case, I imagine it might be to test it more thoroughly, but it's still >beneficial to share it to receive feedback from others. > >>>Dear all, >>> >>>I am currently working on creating a port of the Monero CLI wallet for >>>OpenBSD, the work I have done so far is attached in this email. This project >>>is of personal interest to me, as well as to others who view privacy as an >>>essential component of cybersecurity. >> >>First of all, it wasn't super clear to me if you only ported (or >>intended to port) monero-wallet-cli. The attached port is for all the >>binaries you build from upstream, and in particular monerod. >> > I'm on the side of porting all the binaries. >It's all part of the Monero CLI wallet, as the Monero developers >intended. I think that separating the binaries could be >counterproductive. On the other hand, my interest, along with that of >many others porting this software to OpenBSD, is primarily to host a >node. Personally, I currently host a public Monero node on a Raspberry >Pi 5 running Debian. However, once OpenBSD is compatible with this >hardware, I plan to install it and attempt to run this and other >services there. > >>>I am in the process of learning how to create ports for OpenBSD, which is >>>why I have chosen to start with software that seems relatively >>>straightforward to port. >> >>Hell no. ^^ > >I have realized this the hard way. > >> >>>I think this port is complete. However, it is still a work in progress and I >>>am hesitant to propose it for inclusion in the ports tree due to the >>>following issue: >>> >>>When building this software as a non-root user, the following error occurs: >>> >>>LLVM ERROR: out of memory >>>c++: error: unable to execute command: Abort trap (core dumped) >>> >>>This issue is documented in the original project, and the solution is to >>>increase the ulimit size to 2 GB: >>> >>>ulimit -d 2000000 >>> >>>I would like to know if it is possible to resolve this issue within the port >>>itself, or if it would be acceptable to add this port to the tree despite >>>the problem, perhaps by including a note in the README file. >> >>Didn't deal with this. I run with PORTS_PRIVSEP=Yes and _pbuild runs >>with 8GB base in amd64. > >I am glad this is not a problem. > Build works for me using Lucas's port without additional configuration. >> >>>Additionally, while working on this port and reading the documentation, I >>>encountered several questions that I had to deduce based on how other >>>OpenBSD ports handle them: >>> >>>a. The directories pointed to by certain macros in bsd.port.mk, such as >>>PREFIX and LOCALSTATEDIR. >> >>SUBST_VARS in bsd.port.mk(5) explains a bit, item 16 of [1] explains how >>each directory is used, "make show=VAR" shows the value of VAR. >> >>Having given that piece of information, I don't see a specific question >>in your comment. >> >>>b. How to configure a port to create a new user when needed, for instance, >>>to run a daemon, as is the case with this port. >> >>Take a look at pkg_create(1) @newuser. That needs to be added to >>pkg/PLIST, like done in the attached port. >> >>>These have been the most challenging parts for me, and I find the >>>documentation unclear on these points. >> >>It's quite spread over several manpages, so it's a bit confusing / >>difficult at times. >> >>>This port has been tested on an amd64 platform. Unfortunately, I do not have >>>access to compatible machines with other architectures to conduct further >>>testing. If anyone is able to test the port on different platforms, I would >>>greatly appreciate it. I am currently using a virtual machine, as the only >>>computer I own with sufficient resources to compile this software in a >>>reasonable time frame is not compatible with OpenBSD. >>> >>>Initially, I considered naming the port net/monero-cli since there is an >>>official GUI wallet. However, I decided to leave it as net/monero, as I >>>don't think the GUI wallet will be ported in the future due to the >>>dependencies that would also need to be ported. If I ever attempt to port a >>>GUI wallet, I would try with Feather Wallet, as it seems the most feasible >>>option given that it uses QT. >> >>So, properly about David port, it's mostly fine, but >> >>- it's uncommon to have things listed both in RDEPs and BDEPs, and in >> this particular case all of the are actually LDEPs > >I agree. > >>- you don't need the do-install target, cmake manages it fine > >The modules used in OpenBSD ports handle so many tasks for you that, >at first, you might not realize you don't need to add instructions for >certain tasks because the module already takes care of them. The >package building infrastructure in OpenBSD is highly refined. > >>- you don't need gmake > >According to the original project documentation, you do need it. > We can leave gmake out of it, thank goodness. >>- I ran into a compilation error because hidapi was installed > >Thanks, I didn't realize that. It might be because I built the >software using the "no-dev" CMake option. > >> >>About byteskeptical port: >> >>- small detail, but REVISION isn't needed for an import >>- ONLY_FOR_ARCHS is for things that actually don't build in the other >> arches, not for things not tested in different arches >>- upstream tarballs are preferred over GitHub-generated tarballs >>- also the RDEPs and BDEPs that should be LDEPs and the do-install >> target >>- I run into some compilation error with this one too, but I don't want >> to recompile it yet again. I believe it was a different one than the >> David, but the issue is also present in here: if hidapi is detected >> at configure-time, then the port fails compilation >>- the patches for replacing {sprintf,strcat} with {snprintf,strncat} are >> better suited for upstream, which benefits all the users I will work on those patches and submit them upstream when I get them fully tested. Thank you for the feedback. >> >>My tarball does the following: >> >>- adds hidapi and miniupnp deps, as they are automatically picked up and >> there isn't a sensible way to disable them, which means issue for >> bulks (hidapi requires patching a source file) >> - I do have a hardware wallet to try it out, but I haven't managed to >> do so yet and is low in my priority list. I don't see much point in >> having neither this nor the individual binaries as FLAVORs: OpenBSD >> packages tend to be bulky and prefers something that works mostly >> as expected for 80% of the users over saving space. > >I think it would be beneficial to test this feature to ensure the port >provides 100% of the software's functionality. > Took me longer than expected but I've tested monero-wallet-cli, monero-wallet-rpc, monero-blockchain-stats, monero-blockchain-export, monero-blockchain-import, monero-gen-ssl-cert, and monerod. Everything seems to be operating as expected though I'm running into issues creating a new copy of the blockchain. Whether it's using monerod or monero-blockchain-import when it gets to v12 (randomx introduction) I'm seg-faulting on an opt-code failure. Attaching a tail of the ktrace as the full file is 161GB along with a backtrace from gdb for similar reasons the .core file is 86GB. Seems like a resource exhaustion that shouldn't be happening given it's only suppose to take a few GB of memory and the current system has more than enough memory and disk space available. This was the original intent behind the {snprintf,strncat} patches as I suspect something is not being freed. Where you able to complete a full sync Lucas? >>- uses devel/readline instead of base's, as the latter results in broken >> display (requires patching cmake files) >>- disables liblzma, which is autodetected while looking for libunwind >> (requires patching cmake files) >>- disables the docs: I think they're only dev docs and nothing for >> user-facing programs, and saves a dep on doxygen which tends to break >> things >>- dropped pkg/monerod.login. I'm currently at 38% of a full sync, >> haven't run into issues with the defaults yet. > >I think it would be great if you could provide updates on how it >performs once the blockchain is fully synchronized. > >>- don't install the example monerod.conf: it's 3 lines and the defaults >> work >>- OpenBSD tends to prefer /var/prog to /var/lib/prog, so the home for >> _monerod is /var/monero (also means that the default location of the >> blockchain is /var/monero/.bitmonero) >>- I picked up UID 627 for _monerod. zarafa was replaced by kopano in >> 2017, so I think it should be safe to reuse it now. >>- couldn't make libunwind work and pointing it to > >How did you detect all these problems? What recommendations can you >give me to properly debug a port? I imagine it’s because you're a >programmer, while I’m merely a system administrator. > >> /usr/include/c++/v1/libunwind.h and libc++abi.so isn't enough: > Still working on this as well. >Does anyone know why libunwind is not available on OpenBSD? I am >referring to the portable version, not the one included in Clang. > >https://github.com/libunwind/libunwind > >> >>ld: warning: easylogging++.cc(easylogging++.cc.o:(el::base::utils::File::buildStrippedFilename(char const*, char*, std::__1::basic_string, std::__1::allocator> const&, unsigned long)) in archive external/easylogging++/libeasylogging.a): warning: strcat() is almost always misused, please use strlcat() >>ld: error: undefined symbol: el::base::debug::StackTrace::generateNew() >>>>>referenced by stack_trace.cpp >>>>> stack_trace.cpp.o:(tools::log_stack_trace(char const*)) in archive src/common/libcommon.a >> >>ld: error: undefined symbol: el::base::debug::operator<<(std::__1::basic_ostream>&, el::base::debug::StackTrace const&) >>>>>referenced by stack_trace.cpp >>>>> stack_trace.cpp.o:(tools::log_stack_trace(char const*)) in archive src/common/libcommon.a >> >> but StackTrace::generateNew is behind a complex ifdef and I don't >> feel we're missing out on much without it. I think we may have to tackle this sooner rather than later. I'm also including an updated patch file for CMakeLists_txt to remove the unsupported -z option for ld flags. >> >>That's more than enough for a mail. Port attached. Bye. >> >> Lucas > >Thank you for your help! Hopefully this software can be added to the >OpenBSD ports tree before the next stable release. > Thank you both for the effort on this. >> >> >>diff refs/heads/master 9676c6688a222bd27c6b3fee2f9b2bee4297eea1 >>commit - 3e15eae7e00ea78227ed46a1a7153ce4d102ead1 >>commit + 9676c6688a222bd27c6b3fee2f9b2bee4297eea1 >>blob - 5104415dc091f405d9c26b9c73d49c1d2e7be77e >>blob + 7ba9d8a70c4595b0489e10b1b6ccadcf749d2846 >>--- infrastructure/db/user.list >>+++ infrastructure/db/user.list >>@@ -135,7 +135,7 @@ id user group port >> 624 _noip _noip net/no-ip >> 625 _varnish _varnish www/varnish >> 626 _pound _pound www/pound >>-#627 _zarafa _zarafa mail/zarafa/zarafa >>+627 _monerod _monerod net/monero >> 628 _kamailio _kamailio telephony/kamailio >> 629 _avahi _avahi net/avahi >> 630 _victorialogs _victorialogs sysutils/victorialogs > GDB: #0 0x00000ad3b72b9690 in __vfwprintf (fp=0xad3b769db60, fmt0=0xad390d6c440, ap=0x4b27b35434033bbe) at /usr/src/lib/libc/stdio/vfwprintf.c:517 517 GETASTER(width); (gdb) bt full #0 0x00000ad3b72b9690 in __vfwprintf (fp=0xad3b769db60, fmt0=0xad390d6c440, ap=0x4b27b35434033bbe) at /usr/src/lib/libc/stdio/vfwprintf.c:517 hold = Variable "hold" is not available. (gdb) info frame Stack level 0, frame at 0xad390d6c4e0: rip = 0xad3b72b9690 in __vfwprintf (/usr/src/lib/libc/stdio/vfwprintf.c:517); saved rip 0xad18f0ed3ac called by frame at 0xad390d6c510 source language minimal. Arglist at 0xad390d6c0a8, args: fp=0xad3b769db60, fmt0=0xad390d6c440, ap=0x4b27b35434033bbe Locals at 0xad390d6c0a8, Previous frame's sp is 0xad390d6c4e0 Saved registers: rbx at 0xad390d6c4a0, rbp at 0xad390d6c4d0, r11 at 0xad390d6c4c8, r12 at 0xad390d6c4a8, r13 at 0xad390d6c4b0, r14 at 0xad390d6c4b8, r15 at 0xad390d6c4c0, rip at 0xad390d6c4d8 Current language: auto; currently minimal (gdb) info threads 10 process 134355 0x00000ad3b72d020b in __gethex_D2A (sp=Unhandled dwarf expression opcode 0xa3 ) at /usr/src/lib/libc/gdtoa/gethex.c:255 9 process 165048 0x00000ad3b72d020b in __gethex_D2A (sp=Unhandled dwarf expression opcode 0xa3 ) at /usr/src/lib/libc/gdtoa/gethex.c:255 8 process 339520 0x00000ad3b72d020b in __gethex_D2A (sp=Unhandled dwarf expression opcode 0xa3 ) at /usr/src/lib/libc/gdtoa/gethex.c:255 7 process 157681 0x00000ad3b72d020b in __gethex_D2A (sp=Unhandled dwarf expression opcode 0xa3 ) at /usr/src/lib/libc/gdtoa/gethex.c:255 6 process 551469 0x00000ad3b72d020b in __gethex_D2A (sp=Unhandled dwarf expression opcode 0xa3 ) at /usr/src/lib/libc/gdtoa/gethex.c:255 5 process 308586 0x00000ad3b72d020b in __gethex_D2A (sp=Unhandled dwarf expression opcode 0xa3 ) at /usr/src/lib/libc/gdtoa/gethex.c:255 4 process 431403 0x00000ad3b72d020b in __gethex_D2A (sp=Unhandled dwarf expression opcode 0xa3 ) at /usr/src/lib/libc/gdtoa/gethex.c:255 3 process 387150 0x00000ad3b72d020b in __gethex_D2A (sp=Unhandled dwarf expression opcode 0xa3 ) at /usr/src/lib/libc/gdtoa/gethex.c:255 2 process 579830 0x00000ad18ef24b50 in mdb_page_search_root () from /usr/local/bin/monero-blockchain-import * 1 process 561364 0x00000ad3b72b9690 in __vfwprintf (fp=0xad3b769db60, fmt0=0xad390d6c440, ap=0x4b27b35434033bbe) at /usr/src/lib/libc/stdio/vfwprintf.c:517 Ktrace: 37284 monero-blockchain-impor RET write 4 37284 monero-blockchain-impor CALL mmap(0,0x81000,0x3,0x5002,-1,0) 37284 monero-blockchain-impor RET mmap 11903783845888/0xad390cec000 37284 monero-blockchain-impor CALL mprotect(0xad390cec000,0x1000,0) 37284 monero-blockchain-impor RET mprotect 0 37284 monero-blockchain-impor CALL __tfork(0x7aaeb2f7f6b0,24) 37284 monero-blockchain-impor STRU struct __tfork { tcb=0xad3c666ca00, tid=0xad3c666ca30, stac k=0xad390d6c640 } 37284 monero-blockchain-impor RET __tfork 561364/0x890d4 37284 monero-blockchain-impor CALL munmap(0xad39d516000,0x2000) 37284 monero-blockchain-impor RET munmap 0 37284 monero-blockchain-impor RET __tfork 0 37284 monero-blockchain-impor CALL munmap(0xad454de3000,0x2000) 37284 monero-blockchain-impor RET munmap 0 37284 monero-blockchain-impor CALL munmap(0xad3e3854000,0x2000) 37284 monero-blockchain-impor RET munmap 0 37284 monero-blockchain-impor CALL munmap(0xad476742000,0x1000) 37284 monero-blockchain-impor RET munmap 0 37284 monero-blockchain-impor CALL munmap(0xad45f54b000,0x2000) 37284 monero-blockchain-impor RET munmap 0 37284 monero-blockchain-impor CALL munmap(0xad3e1e07000,0x2000) 37284 monero-blockchain-impor RET munmap 0 37284 monero-blockchain-impor CALL munmap(0xad3aba4d000,0x1000) 37284 monero-blockchain-impor RET munmap 0 37284 monero-blockchain-impor CALL futex(0xad440501b80,0x82,1,0,0) 37284 monero-blockchain-impor RET futex 1 37284 monero-blockchain-impor CALL munmap(0xad3e6831000,0x1000) 37284 monero-blockchain-impor RET futex 0 37284 monero-blockchain-impor RET munmap 0 37284 monero-blockchain-impor CALL futex(0xad440501b80,0x81,165,0,0) 37284 monero-blockchain-impor CALL mmap(0,0xa000,0x3,0x1002,-1,0) 37284 monero-blockchain-impor RET mmap 11906676158464/0xad43d33f000 37284 monero-blockchain-impor CALL mprotect(0xad43d348000,0x1000,0) 37284 monero-blockchain-impor RET mprotect 0 37284 monero-blockchain-impor CALL munmap(0xad4442b1000,0x1000) 37284 monero-blockchain-impor RET munmap 0 37284 monero-blockchain-impor CALL munmap(0xad3d7ac5000,0x1000) 37284 monero-blockchain-impor RET munmap 0 37284 monero-blockchain-impor CALL mmap(0,0x2000,0x3,0x1002,-1,0) 37284 monero-blockchain-impor RET mmap 11906354941952/0xad42a0e9000 37284 monero-blockchain-impor CALL mprotect(0xad42a0ea000,0x1000,0) 37284 monero-blockchain-impor RET mprotect 0 37284 monero-blockchain-impor CALL munmap(0xad42a0e9000,0x2000) 37284 monero-blockchain-impor RET munmap 0 37284 monero-blockchain-impor CALL futex(0xad3f2275660,0x82,2147483647,0,0) 37284 monero-blockchain-impor RET futex 0 37284 monero-blockchain-impor CALL mmap(0,0x13000,0x3,0x1002,-1,0) 37284 monero-blockchain-impor RET mmap 11907574718464/0xad472c2e000 37284 monero-blockchain-impor PSIG SIGSEGV SIG_DFL code=SEGV_ACCERR addr=0xad18f0ef8c0 trapno=6 37284 monero-blockchain-impor NAMI "monero-blockchain-impor.core" -- All desire is the desire to be desired by the subject presumed to know. COMMENT = secure, private, untraceable cryptocurrency V = 0.18.3.4 DISTNAME = monero-source-v${V} PKGNAME = monero-${V} CATEGORIES = net HOMEPAGE = https://getmonero.org/ MAINTAINER = Lucas Gabriel Vuotto SITES = https://downloads.getmonero.org/cli/source/ EXTRACT_SUFX = .tar.bz2 # BSD3, MIT PERMIT_PACKAGE = Yes WANTLIB += ${COMPILER_LIBCXX} boost_chrono-mt boost_date_time-mt WANTLIB += boost_filesystem-mt boost_locale-mt boost_program_options-mt WANTLIB += boost_regex-mt boost_serialization-mt boost_system-mt WANTLIB += boost_thread-mt c crypto ereadline execinfo hidapi-libusb WANTLIB += m sodium ssl unbound zmq # C++14 COMPILER = base-clang ports-gcc MODULES = devel/cmake BUILD_DEPENDS = devel/protobuf \ net/miniupnp/miniupnpc # Uses a bundled lmdb. LIB_DEPENDS = comms/libhidapi \ devel/boost \ devel/readline \ net/libunbound \ net/zeromq TEST_DEPENDS= lang/python \ www/py-requests CONFIGURE_ARGS = -DBUILD_DOCUMENTATION=OFF \ -DMANUAL_SUBMODULES=ON \ -DUSE_DEVICE_TREZOR=ON .include Don't use -Ofast. Index: CMakeLists.txt --- CMakeLists.txt.orig +++ CMakeLists.txt @@ -338,7 +338,7 @@ endif() if(WIN32 OR ARM OR PPC64LE OR PPC64 OR PPC) set(OPT_FLAGS_RELEASE "-O2") else() - set(OPT_FLAGS_RELEASE "-Ofast") + set(OPT_FLAGS_RELEASE "") endif() # BUILD_TAG is used to select the build type to check for a new version @@ -867,15 +867,15 @@ else() add_linker_flag_if_supported("-pie" LD_SECURITY_FLAGS) endif() endif() - add_linker_flag_if_supported(-Wl,-z,relro LD_SECURITY_FLAGS) - add_linker_flag_if_supported(-Wl,-z,now LD_SECURITY_FLAGS) - add_linker_flag_if_supported(-Wl,-z,noexecstack noexecstack_SUPPORTED) + add_linker_flag_if_supported(-Wl,relro LD_SECURITY_FLAGS) + add_linker_flag_if_supported(-Wl,now LD_SECURITY_FLAGS) + add_linker_flag_if_supported(-Wl,noexecstack noexecstack_SUPPORTED) if (noexecstack_SUPPORTED) - set(LD_SECURITY_FLAGS "${LD_SECURITY_FLAGS} -Wl,-z,noexecstack") + set(LD_SECURITY_FLAGS "${LD_SECURITY_FLAGS} -Wl,noexecstack") endif() - add_linker_flag_if_supported(-Wl,-z,noexecheap noexecheap_SUPPORTED) + add_linker_flag_if_supported(-Wl,noexecheap noexecheap_SUPPORTED) if (noexecheap_SUPPORTED) - set(LD_SECURITY_FLAGS "${LD_SECURITY_FLAGS} -Wl,-z,noexecheap") + set(LD_SECURITY_FLAGS "${LD_SECURITY_FLAGS} -Wl,noexecheap") endif() if(BACKCOMPAT)