3 ms·
What type of hardware, GPU are you utilizing? I have the same model self hosted at home
by lobster45 3mo ago
What type of hardware, GPU are you utilizing? I have the same model self hosted at home
- yolo-auto 3mo ago1x MI300x , rocm/sglang , highly optimized image, FP8, FP16 Cache (this fixes looping issues with qwen3.6-35b ~100tok/s average