4 ms·
Oh I didn't expect this to be on HN haha - but yes for our new benchmarks for Qwen3.5, we devised a slightly different approach for quantization which we plan t
by danielhanchen 8mo ago
Oh I didn't expect this to be on HN haha - but yes for our new benchmarks for Qwen3.5, we devised a slightly different approach for quantization which we plan to roll out to all new models from now on!
- nnx 8mo agoCan you describe what is this slightly different approach and why it should work on all models?
- hedora 8mo agoNice! Your stuff ran LLMs extremely well on < $500 boxes (24-32GB ram) with iGPUS before this update. I’m eager to try it out, especially if 16GB is viable now.
- gundmc 8mo agoThe 5080 is 16GB VRAM, not system memory. I don't think you can get 24-32GB VRAM in a $500 box