<ha.alamin@gmail.com> writes:

> My bad, looks like those references do have something to do with it.
>
> I made some more changes to the cross-build script, and tried again,
> and the result was that the sysroot shows up in the linker's -L args
> when compiling with that GHC, then stops showing up after removing
> those references from the package.conf.d files for ghc-internal and
> running ghc-pkg recache, then shows up with a new library directory if I
> manually add it to the conf file and recache again.

To be honest, I'm focusing on putting together a monster of a hack
environment on arm64 by cross-compiling just enough stuff. I suspect
polishing the cross-build flow is too much trouble for one use that we
need to happen. The next one is likely to be RISCV whenever this is.

Some notes for anybody brave enough to repro this at home. We need some
basic tools. Given the cross-compiler, we can leverage the native amd64
cabal to given us the basics.

$ GHC=$HOME/s/ghc/_build/bindist/ghc-9.10.1-aarch64-unknown-openbsd/bin/aarch64-unknown-openbsd-ghc
$ cabal get happy-1.20.0
$ cd happy-1.20.0
$ cabal build -w $GHC
$ scp /home/greg/s/happy-1.20.0/dist-newstyle/build/aarch64-openbsd/ghc-9.10.1/happy-1.20.0/x/happy/build/happy/happy exy:bin/
$ cd ../
$ cabal get alex-3.2.6
$ cd alex-3.2.6
$ cabal build -w $GHC
$ scp /home/greg/s/alex-3.2.6/dist-newstyle/build/aarch64-openbsd/ghc-9.10.1/alex-3.2.6/x/alex/build/alex/alex exy:bin/
...

I also built the tools from ghc source tree by requesting them from our:
hadrian/build ... -j stage2:exe:ghc-{bin,pkg} stage2:exe:hsc2hs

I then scp'd the files over to exy and hand-placed them into the
semi-broken bindist. The ~/bin/ is also populated with these

~/bin/ghc-pkg:

#!/bin/sh

exec /home/greg/ghc/lib/aarch64-unknown-openbsd-ghc-9.10.1/bin/aarch64-unknown-openbsd-ghc-pkg --global-package-db /home/greg/ghc/lib/aarch64-unknown-openbsd-ghc-9.10.1/lib/package.conf.d "$@"

~/bin/ghc

#!/bin/sh

exec  \
  ~/ghc/bin/aarch64-unknown-openbsd-ghc \
  -pgmc=clang -pgml=clang -pgma=clang -optc=--sysroot=/ \
  -optl=--sysroot=/ -optl=-L/usr/local/lib -optl=-Wl,--no-execute-only \
  "$@"

The most trouble was hadrian because it really is inconvenient to build
and is on the critical path.

I had to unpin the index and disable text optimizations which don't
cross-build:

diff --git a/hadrian/cabal.project b/hadrian/cabal.project
index 6666af00e1..0fd9fb8b06 100644
--- a/hadrian/cabal.project
+++ b/hadrian/cabal.project
@@ -4,9 +4,12 @@ packages: ./
 
 -- This essentially freezes the build plan for hadrian
 -- It would be wise to keep this up to date with the state set in ci.sh
-index-state: 2023-10-05T11:38:51Z
+-- index-state: 2023-10-05T11:38:51Z
 
 -- N.B. Compile with -O0 since this is not a performance-critical executable
 -- and the Cabal takes nearly twice as long to build with -O1. See #16817.
 package Cabal
   optimization: False
+
+package text
+  flags: +pure-haskell

I hacked the hadrian build script somewhat mercilessly and probably
unnecessarily, but here it is:

diff --git a/hadrian/build-cabal b/hadrian/build-cabal
index 7d16f3e6b2..0a012af806 100755
--- a/hadrian/build-cabal
+++ b/hadrian/build-cabal
@@ -15,17 +15,13 @@ if ! [ -f "$PROJ" ]; then
     exit 2
 fi
 
-if ! type "$CABAL" > /dev/null; then
-    echo "Please make sure 'cabal' is in your PATH"
-    exit 2
-fi
 
 CABVERSTR=$("$CABAL" --numeric-version)
 CABVER=( ${CABVERSTR//./ } )
 
 if [ "${CABVER[0]}" -gt 2 -o "${CABVER[0]}" -eq 2 -a "${CABVER[1]}" -ge 2 ];
 then
-    "$CABAL" --project-file="$PROJ" new-build "${CABFLAGS[@]}" -j exe:hadrian
+    "$CABAL" -v --with-ghc-pkg=$HOME/s/ghc/_build/bindist/ghc-9.10.1-aarch64-unknown-openbsd/bin/aarch64-unknown-openbsd-ghc-pkg --project-file="$PROJ" new-build "${CABFLAGS[@]}" -j exe:hadrian
     # use new-exec instead of new-run to make sure that the build-tools (alex & happy) are in PATH
     "$CABAL" --project-file="$PROJ" new-exec  "${CABFLAGS[@]}"    hadrian -- \
         --directory "$PWD" \


The command to build the arm64 hadrian binary was:

$ GHC=$GHC hadrian/build-cabal

With all this arm64 goodness prebuilt, I can get my normal off-ports
build script running:

#!/bin/ksh

set -eu

export MAKE=gmake AUTOCONF_VERSION=2.71 AUTOMAKE_VERSION=1.16

./boot

GHC_CC_OPTS='-Wl,--no-execute-only -Qunused-arguments -Wl,-z,nobtcfi'

CONF_GCC_LINKER_OPTS_STAGE1="${GHC_CC_OPTS}" \
CONF_GCC_LINKER_OPTS_STAGE2="${GHC_CC_OPTS}" \
CONF_CC_OPTS_STAGE2="${GHC_CC_OPTS}" \
./configure --with-ffi-includes=/usr/local/include \
	    --with-ffi-libraries=/usr/local/lib \
 	    --with-gmp-includes=/usr/local/include \
 	    --with-gmp-libraries=/usr/local/lib \
 	    --with-iconv-includes=/usr/local/include \
 	    --with-iconv-libraries=/usr/local/lib \
 	    --with-system-libffi

ulimit -d $((10<<20))

~/bin/hadrian \
    '*.rts.ghc.c.opts+=-optc-fno-ret-protector' \
    '*.*.ghc.link.opts+=-optl-Qunused-arguments' \
    '*.*.ghc.link.opts+=-L/usr/local/lib' \
    '*.*.ghc.link.opts+=-optl-Wl,--no-execute-only' \
    --docs=none \
    --flavour=validate+no_profiled_libs \
    -j test

It will take a bit before my slow-ish x13s churns through the whole
build, but I've seen it go part way through Stage1, which means that the
monster cross is good enough of a boot compiler to produce a Stage0.

With luck bindist will just work and I can focus on building ghc 9.8.3
bootstrap for the ports. This won't be an entirely trivial exercise, but
should be much easier than the stuff you've accomplished!

> It's possible the changes to the ksh cross-build script were needed in
> tandem with removing the references, but I doubt it. I think it's more
> likely I made a mistake last time and removed the references from the
> extracted tarball instead of the install prefix after $ gmake install.
>
> However, I'm unable to embed the -Wl,--no-execute-only into the linker
> arguments with stage1.*.ghc.c.opts (nor do I get a grep match for the
> -fno-ret-protector flag in the compiler or linker arguments anyway, so
> I must have misunderstood how it's producing working aarch64 binaries
> after that change).

We'll see pretty soon if my magic incantation a few lines above is good
enough.

Thanks
Greg