3 ms·
If you're willing to go the AMD route, the AMD Radeon Pro R9700 definitely looks interesting for the price compared to NVidia.
by tracker1 4mo ago
If you're willing to go the AMD route, the AMD Radeon Pro R9700 definitely looks interesting for the price compared to NVidia.
- felooboolooomba 4mo agoCan we also run LLMs on Radeon?
- lloyd-christmas 4mo agoI run qwen 27B:Q4 @ 130k context at 50 t/s on a single R9700, and have a 7900XT that runs mellum 12B:Q8 as its subagent. R9700s do really well at low wattage and underclocking as well. It's designed to run at 300W, mine is throttled at 210W, and only had an 8% slowdown. If I had somewhere else to put my desktop in my house, I'd bump it up to 240W and there would be zero perf degradation.