5 ms·
It depends on what you are doing. A lot of people who want to do local inference want to do it using much larger models than what can be fit onto a RTX3090, and
by Tuna-Fish 1y ago
It depends on what you are doing. A lot of people who want to do local inference want to do it using much larger models than what can be fit onto a RTX3090, and Strix Halo is such a hit because it gives you reasonable (not great, but good enough to not be outright painful) performance with 128GB of memory.
- geerlingguy 1y agoAlso, Vulkan is great, and much more stable. Plus tends to work great for new, and even very old, graphics cards.
- deleted 1y ago[deleted]