I'm running the 32b version on a home server (Ryzen 7) using llama.cpp, which uses Vulkan for hardware acceleration on the iGPU. For hobbyist inference, getti

I'm running the 32b version on a home server (Ryzen 7) using llama.cpp, which uses Vulkan for hardware acceleration on the iGPU. For hobbyist inference, getting a iGPU with lots of system ram is probably better than getting a dedicated Nvidia gpu.