I have had a pytorch port (with Vulkan) in the works last year, but 
there were so many small things to cover so I got tired of it.  I did 
succeed in running some inference on my Radeon cards, but I could never 
care enough about cleaning it all up for a port, and now I have forgot 
most of the details :-(.  I am still very interested in ML, and do a lot 
of experiments, but nothing for production use.  I too have succeeded 
with llama.cpp, but do most of my experiments with candle (the Rust 
pytorch wannabe), which sadly lacks Vulkan, which seems to be the 
easiest wy to get GPU support on OpenBSD.  CPU works great though.  
Currenty I actually build POCs of using Rust/vulkano for building small 
neural nets, just to see if I can learn enough of Vulkan, to actually 
add support for it to candle.

However all this is vaporware, and I cannot be trusted to finish any of 
it.  Maybe I can dig out specific bits of someone else is curious, but 
it will not be in very good shape.  This is all stuff I deal with on the 
little spare time I have.

/Niklas

On 2024-07-17 17:28, José Maldonado wrote:
> El mié, 17 jul 2024 a la(s) 7:04 a.m., Frank Baster
> (frankbaster@protonmail.com) escribió:
>> Is there any interest in machine learning ports for PyTorch / JAX / TensorFlow? I had a quick look at mlpack but couldn't find support for recurrent neural networks.
>>
>> PyTorch looks like the most do-able.
> This is possible, but the performance is a big issue here.
>
> For example, you can build PyTorch using Vulkan Backend
> (https://pytorch.org/tutorials/prototype/vulkan_workflow.html - in
> OpenBSD Vulkan support is very good) but Vulkan Backend is slow when
> you compare with specific AI HW accel options.
>
> In my case I use Llama.cpp with Vulkan support enable on AMD RX 580
> for curiosity, and work fine is very usable.
>
> Give a read for this: Kompute (https://kompute.cc/). This is a general
> purpose GPU compute framework for AI using Vulkan Kompute.
>
>