mmastrac

5 ms·

31.
by mmastrac 2mo ago
If you can explain how to make compaction not suck in opencode or Kilo Code, I would love to know how. I am really trying to make use of other harnesses but that's a major sticking point.
32.
by mmastrac 2mo ago
Yes, sorry 5.3 flash.
33.
by mmastrac 2mo ago
NVFP4 ~20-30tps (MTP + vision, no dflash2).
34.
by mmastrac 2mo ago
I previously posted that DS4Flash was _good_ but not _great_ on two DGX Sparks, but I have to say that GLM-5.3 is pretty amazing. It's been able to tackle all the random hard problems I've thrown at it and it has the intuition tha
35.
by mmastrac 2mo ago
Sparks + cables + 10g SFP+ came out to ~$21,500 CAD
36.
by mmastrac 2mo ago
I think I need one now that I have four - reduce is ring-oriented and still works I believe, but IIRC you only get 200gbps if you use _one_ of two connectx ports.
37.
by mmastrac 2mo ago
What harness? I've had similar results as _Implicated_ said above - it's not done well in any of the tests I've tried with it. I currently have it hung off DS4Flash as a pseudo-vision tool and subagent only because of this.
38.
by mmastrac 2mo ago
The cables are ~USD $50 from AliExpress although I'm not sure what the tariff situation is for Americans (I think I paid $75 all-in CAD for them)
39.
by mmastrac 2mo ago
I already have a recycled QNAP-now-TrueNAS that has 10G connections into the fleet so the 1TB doesn't bother me at all. I did some rough math and I don't think I'll ever need to load weights off NFS for what I'm doing so
40.
by mmastrac 2mo ago
For me it's entirely because I have a bunch of projects with my own personal data that would be tough to do with openrouter/claude or any other cloud. For example, I have a small posix-shell-based LLM harness that can SSH into my
41.
by mmastrac 2mo ago
I bought 4x Asus GX10 with the 1TB option. I don't understand why, but it's the only model in the whole lineup that isn't priced insanely. They were briefly on sale with a $200-off coupon, but they show up on warehouse deals
42.
by mmastrac 2mo ago
Weights on HF here: https://huggingface.co/zai-org/GLM-5.3-Flash I decided to take the plunge and get myself four sparks at a decent price (and bought the QSFP cables from AliExpress because they are literally 1/2
43.
by mmastrac 2mo ago
I'm currently on sabbatical looking for volunteer work. I always thought IPFS was an interesting project, might be happy to pitch in some of my now-free bandwidth.
44.
by mmastrac 2mo ago
rsync tends to be a bit tougher on smaller CPUs. scp is generally better for those cases if you don't mind hashing afterwards to confirm everything made it without bit flips (in theory it shouldn't happen, but I've definitely
45.
by mmastrac 2mo ago
I found a crash in zsh and Fable refused to work after that.
46.
by mmastrac 2mo ago
Wild how infrequently this happens. There have only been a handful of compiler crashes I've been hit by (outside of nighties), and I don't recall the last one that has happened since near the 1.0 days. Crater runs are probably one
47.
by mmastrac 2mo ago
I've started giving these instructions and I think I've been much more successful in generating clear output: Comment blocks are <= 7 words, function names <= 4 words. User-facing message strings should be <= 10 words. Us
48.
by mmastrac 2mo ago
That's mostly correct, but I believe the INT8 support on M5 changes the equation pretty drastically. I also believe that diffusion models generally output _better_ results if they have shaping for code (they often will miss delimiter m
49.
by mmastrac 2mo ago
You can save steps when diffusing from a pre-drafted canvas instead of seeding with random noise! If you have a drafter that is good enough, steps drop by ~50% or more, and tok/s can definitely increase. I'd be curious to see your
50.
by mmastrac 2mo ago
I re-implemented this one for macOS over the last couple of months: https://github.com/mmastrac/diffgemma I like the model a lot and it's fairly good at reasoning. You can also really bend it to your needs. It&#x2
51.
Qwen3.8-2.4T(huggingface.co)
84 points by mmastrac 2mo ago|12 comments
52.
by mmastrac 2mo ago
Congrats on this release. The WASM implementation is really cool. This is a surprisingly good fit for a lot of cases, and I totally want to try turning this into a helper assistant for an application. Please, though, take a pass at humanizi
53.
by mmastrac 2mo ago
I've been pondering a smaller quantization and experts swapping for this! Happy to take on PRs if you want to experiment as well.
54.
by mmastrac 2mo ago
I've been working on a fresh, AI assisted port of DiffusionGemma from scratch and it takes a significant amount of time to deslop. I've spend a nonzero amount of time on refactoring and comment-vomit cleanup. https://gi
55.
by mmastrac 3mo ago
I suspect this means that there's just a ~10% inefficiency in token to information mapping.
56.
by mmastrac 3mo ago
Which one? Nemotron Diffusion? It's impossible to say for sure, but I have a fairly deep library of metal kernels that _might_ cover some of the nvidia model's architecture.
57.
by mmastrac 3mo ago
Awesome. I may need to finally bite the bullet and upgrade my macOS to test out the MPP approach you've taken. I've got a number of tiled-load kernels, and a top-k attention kernel that you might find interesting.
58.
by mmastrac 3mo ago
TBH, I think there's some truth to that. I spent _ages_ tuning the kernels to match the tested FLOP count of my M3's processor. I only have an M3 though and wasn't able to push int8 very far on it, but I think there's a
59.
by mmastrac 3mo ago
I have a project that's almost ready to run DiffusionGemma as well. The two project might potentially work well together. I'm getting ~20tok/s on a 36GB M3 and there's strong possibility we might be able to crib faster k
60.
by mmastrac 3mo ago
The founding fathers would be aghast at what America became. Hard to imagine modern America passing some of those constitutional amendments that older America passed wayyy back.

More ›