3 ms·
If you’re on AMD, use llamacpp with the vulkan backend. For the 35B I get around 115 tps without mtp and 150-170tps with it. Also the pcie slot you use affects
by mhjkl 2mo ago
If you’re on AMD, use llamacpp with the vulkan backend. For the 35B I get around 115 tps without mtp and 150-170tps with it. Also the pcie slot you use affects it
- ThunderSizzle 2mo agoI have issues with the vulkan backend. It seems to cause over all GUI sluggishness, but maybe that was actually Firefox. I'll give it another go.