colingauvin

4 ms·

31.
by colingauvin 1mo ago
Quality of tokens matters almost as much. 30 is worth it on a meaningfully better model, though annoying. 50 is pretty much unnoticeable (good old 60 FPS). 80-100 is heaven (aka 120 FPS - once you get used to it, it does suck to go back). I
32.
by colingauvin 2mo ago
I can't recommend LocalAI enough. I am thoroughly enjoying it in all aspects. Especially the memory management and ability to load and boot models to and from memory as needed, right from the clients that use them.
33.
by colingauvin 2mo ago
Finally switched over to entirely local AI models (Qwen 3.8 Flash Next) and...for the first time in a year, I didn't know Claude was down. What a phenomenal feeling.
34.
by colingauvin 2mo ago
In terms of pure tokens per dollar, absolutely not worth it. That said, when I bought my pair of Sparks, the best model I could run on it was GPT OSS 120B. That has an AA score of 24. Today, the best model I can run on them is GLM 5.3 Flash
35.
by colingauvin 2mo ago
PWAs suck and when I have to use one I immediately go find something else.
36.
by colingauvin 2mo ago
Nvidia literally releases quantization tools.
37.
by colingauvin 2mo ago
A single Spark alone is not worth the price. You are paying $1000 just for networking equipment you aren't using. At 2x it starts to maybe become worth it if you don't want to deal with Apple. Outside of the newest Macs, I can&#x2
38.
by colingauvin 2mo ago
Your point isn't lost on me, but a few other considerations: 1) Rates are theoretically discounted for GLM 5.3 Flash right now, by 50%. 2) Hardware costs have continued ascending with no sign of letting off, so it's unlikely that
39.
by colingauvin 2mo ago
Who the hell cares when I can run it myself?
40.
by colingauvin 2mo ago
That's 160GB-ish for Q4...how is 256 insufficient?
41.
by colingauvin 2mo ago
I don't know anything about blood pressure, really. But I do know that where this guy's writing touches my own expertise (infectious diseases, pharmacology) - he is often very very wrong.
42.
by colingauvin 2mo ago
I'm supposed to believe that DeepSeek distilled a 300B-1T param model with 150,000 requests? Lol.
43.
by colingauvin 2mo ago
I'm saying that it is unclear that without distillation this wouldn't still be happening. There is a massive narrative that no one but OpenAI, Anthropic, and Google can make a model without distilling. But there's basically n
44.
by colingauvin 2mo ago
Where is the actual evidence of distillation? I keep seeing this repeated ad nauseam but I must have somehow missed the evidence.
45.
by colingauvin 2mo ago
In general the more severe a virus the less likely it is to spread into a pandemic. This is for a couple reasons: 1) really sick people are less likely to go spread it, 2) really sick people are easier to identify and isolate and 3) the tim
46.
by colingauvin 2mo ago
I can't even ask it how to make toast without zeroing out all its memories of me.
47.
by colingauvin 2mo ago
It has figured it out through the software that I work on, data that I analyze, questions I ask, etc. I guess I can't prove it's because I'm a biochemist, but that's the only thing that makes any sense. Either way it won
48.
by colingauvin 2mo ago
I am because Claude has figured out I am a biochemist and therefore even asking Fable what the weather is gets me bumped back to Opus, sometimes even Opus 4.8 instead of 5. Kimi? GLM 5.3? DeepSeek? No such problem. I can literally open a ne
49.
by colingauvin 2mo ago
Which implementation details? I very much already have this, today.
50.
by colingauvin 2mo ago
You can also just do read only access through most mcp servers for functionally no difference
51.
by colingauvin 2mo ago
That really is the load bearing seam, and it's worth stating plainly. I realized I was spending most of my tokens arguing with Opus and trying to get it to let go of stupid, lazy, obviously incorrect premonitions. I wound up canceling
52.
by colingauvin 2mo ago
It's unreal how bad the initial rollout was between HTTP/streaming and stdio, bearer auth and OAuth. Virtually every client/MCP server pair had a different portion of that matrix implemented.
53.
by colingauvin 2mo ago
shhhhhhhhh
54.
by colingauvin 2mo ago
As someone living in Montana, I agree. It's awful up here. I don't recommend it at all. Stay in California folks!
55.
by colingauvin 2mo ago
I absolutely positively believe that consciousness is tuning into something else. Some specific arrangement of information and neurons allows it to be received.
56.
by colingauvin 2mo ago
I live in Montana and routinely hunt the Musselshell river valley/Meagher county area. It's maybe my favorite place in the world. So few folks have been there, but spend some time there and you might just find yourself feeling dep
57.
by colingauvin 2mo ago
I hate cargo and npm. Why do we keep settling on arbitrary code execution in our build process?
58.
by colingauvin 2mo ago
I think it would run out of context before it could hack that much stuff.
59.
by colingauvin 2mo ago
3 DGX Sparks with 400 Gb interconnects in a loop.
60.
by colingauvin 2mo ago
Ah thanks for the background. Well, time to learn how to quant things down!

More ›