From: José Maldonado <josemald89@gmail.com>
Subject: Re: Enable VA-API in graphics/ffmpeg
To: Landry Breuil <landry@openbsd.org>
Cc: ports@openbsd.org
Date: Fri, 26 Jul 2024 20:49:49 -0400

El vie, 26 jul 2024 a la(s) 8:56 a.m., Landry Breuil
(landry@openbsd.org) escribió:
>
> > There are numerous issues that will need to be fixed before va-api in
> > firefox will actually work. Some of them are pledge/unveil related,
> > others are firefox hardcoding specific library versions for dlopen
> > without fallback.
>
> thanks tb for looking into that :)
>
> > With the symlinks below and various additions to pledge and unveil,
> > I get sound to play in what looks like hardware decoded video.
> > Unfortunately, the player remains dark...
>
> the symlinks shouldnt be needed in all cases, because in the code i can
> find that wraps ffmpeg and tries to load the libva libs
> (https://searchfox.org/mozilla-central/source/dom/media/platforms/ffmpeg/FFmpegLibWrapper.cpp#366)
> and the libgbm libs
> (https://searchfox.org/mozilla-central/source/third_party/gbm/libgbm/mozgbm.cpp#31),
> (and maybe others ? havent seen libxaw in the source code)
> it uses nspr's PR_LoadLibraryWithFlags() which is patched to drop what's
> after the .so (cf
> https://github.com/openbsd/ports/blob/master/devel/nspr/patches/patch-nspr_pr_src_linking_prlink_c#L32)
>

In my case, I am trying Firefox 128 from packages, and ffmpeg with
VAAPI patches and obtain VAAPI hwaccel with this changes

Modify /etc/firefox/unveil.rdd

# shm files
/tmp rwc
#Vaapi
/dev/dri rw
/usr/lib r
/usr/X11R6/lib r
/usr/local/lib r
~/.Xauthority r

Modify /etc/firefox/pledge.rdd
stdio
rpath # gtk tries to access /usr/local/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
tmppath
recvfd
sendfd
unix # getsockopt
# VAAPI
wpath
cpath
drm
prot_exec

Modify /etc/firefox/unveil.utility
/usr/lib r
/usr/local/lib r
/tmp rwc
# Vaapi test
/usr/X11R6/lib r

And symlinks for libgbm and libdrm, for fix hard code libs in Firefox.

Execute Firefox with:

MOZ_LOG="FFmpegVideo:5,Dmabuf:5,OpenBSDSandbox:5" MOZ_GFX_DEBUG=1 firefox

Don´t show error and ffmpeg+vaapi work fine, attach partial log:

https://pastebin.com/q77ESyXy

> what can/should be tried instead is preloading those libs before calling
> unveil/pledge, at the same place where it is done for ffmpeg libs cf
> https://searchfox.org/mozilla-central/source/dom/media/platforms/ffmpeg/ffvpx/FFVPXRuntimeLinker.cpp#44
> and
> https://searchfox.org/mozilla-central/source/dom/media/ipc/RDDProcessImpl.cpp#31
>
> with that, maybe no changes to unveil/pledge config is needed (or less)
> trying various combination of pr_loadlibrary flags might be needed too,
> so that it doesnt try to load again a lib that is already loaded, but
> without the same flags, where in that case it bypasses the dlopen cache
> and fails to load the lib because unveil.
>
> cf https://bugzilla.mozilla.org/show_bug.cgi?id=1856301 &
> https://hg.mozilla.org/mozilla-central/rev/f6d59e68442d4e8fc895d59de77277f823c50fe2
> for the previous similar issue wrt dlopen cache.
>
> (sorry, wont be able to play much with that in the coming days)
>
> Landry
>

On the other hand, there is one additional positive point in creating
the libgbm and libdrm symlinks (or patch hardcode name for this libs
in Firefox): enabling general DMAbuf support for Firefox which is
broken due to the library versioning not matching.

Enabling DMAbuf support can improve WebGL performance in certain
conditions and opens the way for using the full Wayland implementation
of Firefox, for those who want to experiment and develop support for
it.

If I'm not mistaken this leaves us with two things:

1.- Apply the patches you mentioned to pre-load libva and VAAPI
support without many changes in unveil/pledge, but I don't think this
enables full DRM and DMAbuf support for Firefox.

2.- Apply a patch to change the library lookup to match OpenBSD
versioning or directly make the symlinks and pledge/unveil settings so
that everything works as it is.


-- 
"Dios en su Cielo, todo bien en la Tierra"
***********************************************