4 ms·

Thankfully, there are still people willing to jump on the R9700 bandwagon and get a vLLM fork working. If you have an RDNA4 card check out https://hub.docker.c

Thankfully, there are still people willing to jump on the R9700 bandwagon and get a vLLM fork working. If you have an RDNA4 card check out https://hub.docker.com/r/stilldeadcode/vllm-radiance https://hub.docker.com/r/stilldeadcode/vllm-radiance

intothemild 27d ago
Deadcode is currently working on INT4 right now on his R4D kernel. The MXFP4 fork is excellent too. Its my daily driver right now. https://codeberg.org/ggz14/radiance-vllm-mxfp4 https://codeberg.org/ggz14/radiance-vllm-mxfp4 Also has PARO quant support there too (early stage) Also speedups in both repos for 4x R9700s
- karmakaze 27d ago
  Thanks! Didn't expect to see this here. Exactly what I needed to run Qwen3.8-27B-Quark-AWQ-MXFP4-native.gguf as well as other experiments on one or 2x R9700's (I hope).
- nicce 27d ago
  That MXFP4 is an excellent project. But I have difficulties on reading that README. Is it intentionally generated like that with LLMs?
  - intothemild 27d ago
    Feed the setup and run scripts to your LLM.
- karmakaze 27d ago
  The way it does tensor splitting without all-reduce cost over PCIe bus wasn't something I thought was possible. What kind of performance are you getting with 4x R9700s--what do you do with all the VRAM (batching, concurrent requests, etc)?
  - intothemild 27d ago
    personally? i have 2x gpus.. but i get bursts of ~200tok/s generation, and around 4500-5000tok/s prefil Yeah the R4D Kernel rules imho.
    - karmakaze 27d ago
      Similar here peak ~250 and down to ~120 as it gets close to 128k (which is where I set DSH compaction) though it can readily do 256k. I just got DeepSeek Harness (DSH) set up with 2x R9700 and it's rather mind blowing that these can do actual work and quickly. Up until now I've always been evaluating and searching for better hardware/model/tweaks. This is much more than I even hoped for and considered getting extra 3090/4090. Now I can stop looking/tweaking and start using it for all the different things I've yet to discover it's good for. I do plan to also try/use Hermes and Pi. DSH is annoying that every plugin install/remove requires a restart--given that "everything's a plugin".

intothemild 27d ago

Deadcode is currently working on INT4 right now on his R4D kernel. The MXFP4 fork is excellent too. Its my daily driver right now. https://codeberg.org/ggz14/radiance-vllm-mxfp4 https://codeberg.org/ggz14/radiance-vllm-mxfp4 Also has PARO quant support there too (early stage) Also speedups in both repos for 4x R9700s

karmakaze 27d ago
Thanks! Didn't expect to see this here. Exactly what I needed to run Qwen3.8-27B-Quark-AWQ-MXFP4-native.gguf as well as other experiments on one or 2x R9700's (I hope).
nicce 27d ago
That MXFP4 is an excellent project. But I have difficulties on reading that README. Is it intentionally generated like that with LLMs?
- intothemild 27d ago
  Feed the setup and run scripts to your LLM.
karmakaze 27d ago
The way it does tensor splitting without all-reduce cost over PCIe bus wasn't something I thought was possible. What kind of performance are you getting with 4x R9700s--what do you do with all the VRAM (batching, concurrent requests, etc)?
- intothemild 27d ago
  personally? i have 2x gpus.. but i get bursts of ~200tok/s generation, and around 4500-5000tok/s prefil Yeah the R4D Kernel rules imho.
  - karmakaze 27d ago
    Similar here peak ~250 and down to ~120 as it gets close to 128k (which is where I set DSH compaction) though it can readily do 256k. I just got DeepSeek Harness (DSH) set up with 2x R9700 and it's rather mind blowing that these can do actual work and quickly. Up until now I've always been evaluating and searching for better hardware/model/tweaks. This is much more than I even hoped for and considered getting extra 3090/4090. Now I can stop looking/tweaking and start using it for all the different things I've yet to discover it's good for. I do plan to also try/use Hermes and Pi. DSH is annoying that every plugin install/remove requires a restart--given that "everything's a plugin".