3 ms·
Ask HN: What can I do with a single R9700?
Hi HN,
I’m considering getting an R9700, budget only for one unit currently. Is there anything useful I can do with local LLMs within its 32G VRAM? I’m also thinking to try and learn building a simple inference engine. Any thoughts?
- spottedmarley 2mo agoFor learning it's all you need. Pick a good small model, I would recommend Qwen3.5 there are models ranging from 0.8b params (will run on a smart phone) all the way up to 397b params .. I run the 122b param model daily with 128GB vram and get 20+tk/sec. regardless of the model, you can learn all about how to host and harness a model with any size
- scott01 2mo agoWith this amount of VRAM, do you happen to be using a Strix Halo or a Mac?
- spottedmarley 2mo agoIt's a Dell Pro Max GB10. I have two that I can bridge together to run larger models across the 256GB pool but I only bridge them when I am experimenting with larger models and they will run fine but at lower tk/s with really only marginal reasoning gains, so I tend to stick to medium models (122b) on one GB10 and I use the other to serve gen models for music, image and video gen.
- rdx-amw 2mo agoPlenty - you could do both with an R9700. Unless it's CUDA only, a fair amount of software now supports AMD ROCm. And if you're already wanting to write an inference engine, 32G will be more than fine. I've had a R9700 and it'd run the lower quants (4-5 bit?) of Qwen 3.5 27B reasonably attached to Hermes Agent. Not quite a 3090, but still respectable speeds.
- FloatArtifact 2mo agoYou say that as it's in the past, have you moved on to something else?
- rdx-amw 2mo agoNo, the R9700 I have got fried by a power supply fault, and GPU replacements are not always trivial. Of the two affordable options for good 32G compute, the R9700 would be the best of the two (compared to the Arc Pro B70).
- scott01 2mo agoInteresting, how did the fault happen for you? Was the PSU underpowered? I have a 850W unit, so you think it’ll be enough?
- rdx-amw 2mo agoAccidental - miswired PCIe power such that it went back into a SATA/molex modular port. Somehow managed to not break anything else though, but an experience not to repeat.
- jake_and_fatman 2mo ago[dead]
- moustache_hn 2mo ago[flagged]
- andrevalleee 2mo ago[flagged]
- Daxo 2mo ago[flagged]