mmastrac

4 ms·

1.
by mmastrac 7d ago
Nice. I saw some git-on-durable-objects projects but I'd so much prefer a straight object storage option.
2.
by mmastrac 7d ago
I'd be interested to see if using DiffusionGemma-as-Jev helps as you can feed the image directly into the model and it'll make decisions based on the image embeddings.
3.
by mmastrac 8d ago
I suspect you could get interesting results, but DiffusionGemma has a lot of knowledge that may be challenging to train into the smaller models. The advantage of pulling a fully-trained diffusion model off the shelf is that it already knows
4.
by mmastrac 8d ago
I believe you get better answers by diffusing each question together, but the PR's server gives you finer control over that. If each question is independent, you can get better parallelism.
5.
Near-real-time vision using DiffusionGemma in Jev mode(twitter.com)
2 points by mmastrac 8d ago|0 comments
6.
by mmastrac 8d ago
If you want to try a _legit_ Jev implementation that matches (at least in my evals), the vLLM patch to turn DiffusionGemma into Jev is available. On my DGX Spark I get very similar latency numbers, and it matches my evals + or - a few point
7.
by mmastrac 8d ago
That's a rough place to land on a spark. It seems unlikely to be memory bandwidth at this model size, but maybe just lack of tuned kernels? The chip is missing some CUDA features but with tuning you should be able to hit way more than
8.
by mmastrac 9d ago
Send me an email- it's not public just yet
9.
by mmastrac 9d ago
I've been working with an automatic incremental context compactor enabled and it's been surprisingly helpful. It was particularly effective with DS41f - I think I was running at an effective session length of 5M, with the model ru
10.
by mmastrac 10d ago
Any diffusion model is potentially a Jev in disguise: https://github.com/vllm-project/vllm/pull/57250 Runs ~0.2s per decision on my DGX Spark. 10/10 programming language detection 9/10 human l
11.
vLLM: Jev-like mode for the DiffusionGemma model(github.com)
1 points by mmastrac 10d ago|0 comments
12.
by mmastrac 10d ago
Is this a Markov/Diffusion model with some sort of external Engram memory? If so, this could be extremely interesting.
13.
by mmastrac 12d ago
Andon Cafe: Revenue: 13 933 kr Token Cost: 14 882 kr .. and a bastardization of the old Twitter joke: "I told our CEO that our cafe keeps hitting its token spending limit but not making money. So I asked where he gets the tokens, and h
14.
by mmastrac 13d ago
It's "load bearing".
15.
by mmastrac 14d ago
magnet:?xt=urn:btih:67177297E84766DFBF1C9EAAC6CF44B6F40BF3D1 I haven't checked it out myself but I've heard that some posts were removed from this magnet.
16.
by mmastrac 14d ago
This is one of the best references on this topic I've seen - run cycles and foot locking are annoyingly lacking in documentation outside a few papers.
17.
by mmastrac 16d ago
I'm using DSH with my local models (4x sparks running GLM53F, trying them on DS41F this morning). DSH is better than opencode IMO. It's a little barebones out of the box but I guess that's the point. I had to have an agent ad
18.
Rust is tier-1 language at Microsoft(rustfoundation.org)
726 points by mmastrac 16d ago|517 comments
19.
by mmastrac 16d ago
I ran the preview model around 2,126,605,070 tokens for $22.04 USD for the last couple of days. Kind of shocked. It did a decent job refactoring https://github.com/mmastrac/diffgemma to create a CUDA support backbone,
20.
by mmastrac 17d ago
I've been trying out the 4.1 flash preview for some bulk tasks: it did a pretty good job refactoring a bunch of .metal kernels to .cu. It needed less steering than Opus on refactoring, IMO, and writes better comments. It failed to port
21.
by mmastrac 18d ago
https://github.com/mmastrac/diffgemma#custom-quantization has some instructions on (naively) quantizing the upstream model as well.
22.
by mmastrac 18d ago
It should be possible to run on Mac via https://github.com/mmastrac/diffgemma , but I'm at rustconf right now and I can't download weights on hotel wifi easily.
23.
by mmastrac 22d ago
Honestly, not sure. I use Firefox Mobile pretty heavily during the day and I suspect it's part of the drain. I'm not too concerned by it and I just try to replace the battery every ~1-1.5 years.
24.
by mmastrac 22d ago
I've been running mine for ~4ish I think, on my third battery and might run out my fourth because it's feeling a _little_ sluggish at times. I am waiting for the next FP model where I can use a USB-C/dp external display and t
25.
by mmastrac 23d ago
The comparisons with other models here are odd.. the other models change depending on the task. It would be far more useful to at least compare against the more recent open models (DS4Flash/GLM53Flash/Qwen38).
26.
by mmastrac 24d ago
Any idea what size this is?
27.
by mmastrac 26d ago
Do you have a source for more info? That seems odd to me - LLMs are particularly resilient to bitflips, so if anything the demand would pull on DDR5 in general, not on ECC specifically.
28.
by mmastrac 26d ago
What's missing from these discussions is the real weakness of diffusion models: you can have two positions where two tokens need to be coordinated, but both spots are teetering between two valid states. They don't always collapse
29.
by mmastrac 26d ago
Is there a lookup that gives you a title to blob mapping?
30.
by mmastrac 27d ago
Oh nice. I have a little PCIe/VGA one that I use for truenas and it's a bit flaky but I love that it's basically free from a PCIe perspective (just toss it off _any_ root). These are super useful and there are even m.2 varian

More ›