20 ms·
Depending on what you want to do, you already can. llama.cpp and other inference servers work fine on the kernel driver.
by suprjami 1y ago
Depending on what you want to do, you already can.
llama.cpp and other inference servers work fine on the kernel driver.
- yencabulator 1y agoWhere "fine" unfortunately still means "don't push it too hard on a busy desktop system or your graphical session might crash". Make sure to keep enough RAM free or you start seeing GPU resets, the stack can't cope with transient errors :-(