3 ms·

Can't wait for the DwarfStar quants - I have been using DeepSeek v4 flash (preview) as my main coding agent for months now (running on my 128gb mbp) - it seems

Can't wait for the DwarfStar quants - I have been using DeepSeek v4 flash (preview) as my main coding agent for months now (running on my 128gb mbp) - it seems this model outperforms GLM 5.2 on nearly every metric. Thanks for sharing the news, I was refreshing huggingface but gave up thinking it likely would take some more time.

theturtletalks 2mo ago
What kind of tps are you getting?
- kamranjon 2mo ago
  Generally get 20-25tps - prefill is pretty good around 400-450tps. I have been using compaction at around 100k tokens but mostly just cause it was the default in pi coding agent - might see if i can expand it a bit.
  - Lwerewolf 2mo ago
    I've had it run to ~400k when debugging "obscure" (to it) sequences. Wouldn't recommend more.
vmt-man 2mo ago
are you working in earplugs? :)) even with 128 gigs of ram it must be super noisy.
- hetspookjee 2mo ago
  the noise is pretty doable, and so is the heat, I find. I read these messages before I had the machine and expected far worse. The tok/s is for me the dealbreaker as I prefer to have multiple sessions. For nightly runs i do like it a lot, or to be a node in a mesh.
- kamranjon 2mo ago
  I actually run it as a server - so most of the time I don't have to listen to it right next to me - it's just sitting in another room in my house - but I often am traveling with it and will have it sitting right next to my coding laptop and yea the fan runs non-stop - it's not obnoxious so i can pretty easily tune it out - also airpods/noise canceling headphones help!
- FuckButtons 2mo ago
  There’s fan noise, but the acoustic engineers at Apple have done a very good job of making it be pretty much not noticeable to someone who has noise sensitivity (me).
- ycui7 2mo ago
  my AC is noiser than my GPU server.
deleted 2mo ago
[deleted]
knuppar 2mo ago
same. been running it in a dgx spark and it slaps
tough 2mo ago
I've put Opus to it and it says it will take 3-4h to do the process to the new weights. hoping it works!
ycui7 2mo ago
you don’t need new weight. try vllm-moet from github. it will autogenerate 2-bit plane.

theturtletalks 2mo ago

What kind of tps are you getting?

kamranjon 2mo ago
Generally get 20-25tps - prefill is pretty good around 400-450tps. I have been using compaction at around 100k tokens but mostly just cause it was the default in pi coding agent - might see if i can expand it a bit.
- Lwerewolf 2mo ago
  I've had it run to ~400k when debugging "obscure" (to it) sequences. Wouldn't recommend more.

vmt-man 2mo ago

are you working in earplugs? :)) even with 128 gigs of ram it must be super noisy.

hetspookjee 2mo ago
the noise is pretty doable, and so is the heat, I find. I read these messages before I had the machine and expected far worse. The tok/s is for me the dealbreaker as I prefer to have multiple sessions. For nightly runs i do like it a lot, or to be a node in a mesh.
kamranjon 2mo ago
I actually run it as a server - so most of the time I don't have to listen to it right next to me - it's just sitting in another room in my house - but I often am traveling with it and will have it sitting right next to my coding laptop and yea the fan runs non-stop - it's not obnoxious so i can pretty easily tune it out - also airpods/noise canceling headphones help!
FuckButtons 2mo ago
There’s fan noise, but the acoustic engineers at Apple have done a very good job of making it be pretty much not noticeable to someone who has noise sensitivity (me).
ycui7 2mo ago
my AC is noiser than my GPU server.

deleted 2mo ago

[deleted]

knuppar 2mo ago

same. been running it in a dgx spark and it slaps

tough 2mo ago

I've put Opus to it and it says it will take 3-4h to do the process to the new weights. hoping it works!

ycui7 2mo ago

you don’t need new weight. try vllm-moet from github. it will autogenerate 2-bit plane.