3 ms·

Been using DeepSeek Coder 33B Q8 on my work laptop for a bit now. I like it, but am still finding myself going to GPT-4's API for the more nuanced things. They

Been using DeepSeek Coder 33B Q8 on my work laptop for a bit now. I like it, but am still finding myself going to GPT-4's API for the more nuanced things. They just released a v1.5 (https://huggingface.co/deepseek-ai/deepseek-coder-7b-instruct-v1.5 https://huggingface.co/deepseek-ai/deepseek-coder-7b-instruc...), but for some reason, they reduced the context length from ~16k to ~4k.

busfahrer 3y ago
Interesting, what are your laptop specs? I imagine the 33B Q8 to need a bit of memory - are you sharing VRAM and RAM?
- M4v3R 3y ago
  ARM based Macbooks with 48 GB of RAM or more are capable of running 33B models just fine, thanks to their "unified memory" architecture (single memory subsystem for RAM & VRAM).
- _boffin_ 3y ago
  M2 Max 64GB MBP 16"