3 ms·
I'm still in awe that a local 3090 gpu was able to run the qwen3 coder instruct 30b-a3b exl3 q6 and... Was able to create a sample page, tried starting a serve
by eurekin 1y ago
I'm still in awe that a local 3090 gpu was able to run the qwen3 coder instruct 30b-a3b exl3 q6 and...
Was able to create a sample page, tried starting a server, recognising a leftover server was running, killing it (and forced a prompt for my permission), retrying and finding out it's ip for me to open in the browser.
This isn't a demo anymore. That's actually very useful help for interns/juniors already.
- mrheosuper 1y agoHow did you do your setup ?, right now the only way i know how to run LLM is through LM studio.
- eurekin 1y agoUsing tabbyApi and exLlamav2.