4 ms·
The $1500 Intel B70 with 32GB of VRAM can run this model at max context with good performance, btw. If you don't want to drop $5-10k for running DeepSeek this i
by Almondsetat 2mo ago
The $1500 Intel B70 with 32GB of VRAM can run this model at max context with good performance, btw. If you don't want to drop $5-10k for running DeepSeek this is your best budget option for local refactor/small scale dev help
- bogzz 2mo agoOh, can it work with the /v1/completions/ auto-complete endpoint?
- Almondsetat 2mo agoSorry, I wrote autocompletion by force of habit. I simply meant it can complete code you have already created a structure for, which personally is very nice
- bogzz 2mo agoI thought so, but thanks for the clarification. I am a little bit disappointed that local autocompletion models have been left by the wayside in favor of models post-trained for agentic coding. Both Codestral and Qwen-2.5-coder are more than a year old at this point, but local auto-complete seems to me to be such a great usecase.
- gered 2mo agoThe latest Qwen models (including 3.8 27B) do still support FIM-style in-editor code auto-completion if that's what you're looking for. I wouldn't want to use a large dense model like 27B for such a task (since FIM-style auto-completion really works best with low-latency responses), but it works.
- LeBit 2mo agoI understand the B70 is a bargain vs AMD and especially nVidia offerings, but to me it feels like I would be buying something that would feel too limited in less than a year. 48G would be much more confortable. And I know the 96G nVidia cards are selling for over 10k$. The future can’t arrive fast enough!
- Almondsetat 2mo ago32GB is perfect for models around 30B parameters. Since qwen has really hit the spot with their 27B dense models, I think it's a good bet. Also, 32GB is enough for other tasks such as image/video generation and loading multiple smaller specialized models
- kamranjon 2mo agoYou can buy two b60s for $1300 right now (650 each) if you want a total of 48gb. Intel recently raised the price on all of their gpu's except the b60 series, so they are currently the best deal per gb I think.
- Almondsetat 2mo agoThey have terrible cross connectivity, it's not wise to share a model between them
- segmondy 2mo agoYou don't need $10k to run DeepSeek, I run it on a $1000 system.
- 758488 2mo agoCould you elaborate please? Genuinely interested
- desterothx 2mo agoat that price, probably loaded into ddr5 ram, a 3 bit or less quantization, and 3 tps
- kennywinker 2mo agoGo on… what is this $1000 system?
- jared0x90 2mo ago4x CMP-170HX he purchased with his time machine last month
- aappleby 2mo agoI have a B70, what llama options are you using and what performance are you seeing?
- apitman 2mo agoI wonder how this would stack up against 4x RTX 3060, assuming you have the physical room for them.
- Almondsetat 2mo agoSince the 3060s would have to communicate constantly using pcie, I think I'd go with intel
- apitman 2mo agoCurrently software is king when it comes to inference performance though.
- Almondsetat 2mo agoHaving the entire model in the vram without having to transfer anything is even more king, though