2 ms·
Did you set that up following a guide or anything you could share?
by turtle4 8mo ago
Did you set that up following a guide or anything you could share?
- mathrawka 8mo agonot OP, but I got it running on my 4090 (and RAM) by following this guide: https://unsloth.ai/docs/models/qwen3-coder-next https://unsloth.ai/docs/models/qwen3-coder-next I see around 30 t/s
- PeterStuer 8mo agoEasiest way I know is to just use LMStudio. Just download and press play :). Optional, but recommended, increase the context length to 262144 if you have the DRAM available. It will definitely get slower as your interaction prolongs, but (at least for me) still tolerable speed.