4 ms·
I got a "gaming" PC for LLM inference with an RTX 3060. I could have gotten more VRAM for my buck with AMD, but didn't because at the time alot of inference nee
by aussieguy1234 2y ago
I got a "gaming" PC for LLM inference with an RTX 3060. I could have gotten more VRAM for my buck with AMD, but didn't because at the time alot of inference needed CUDA.
As soon AMD is as good as Nvidia for inference, I'll switch over.
But I've read on here that their hardware engineers aren't even given enough hardware to test with...