colingauvin

5 ms·

61.
by colingauvin 2mo ago
397 is just too big for two Sparks even at NVFP4. Wish they had made this just a tiny bit smaller.
62.
by colingauvin 2mo ago
All the people saying "it's not a driver, it didn't write a driver, it's wrapping the Linux one" are missing the point: no one cares - the printer prints. It didn't before, and now it does.
63.
by colingauvin 2mo ago
I pre-ordered the very first Framework 13. I was so excited that I loaned it to Level1Techs to make a video of it, and at one point early on it also failed - everyone I communicated with had their name printed on that motherboard, all the w
64.
by colingauvin 2mo ago
I enjoyed the writeup on the optics. Very similar to the contrast transfer function in my field (cryo-EM). Except arguably harder because this is an anisotropic technique with a much less coherent source. In this case, refractive index is a
65.
by colingauvin 2mo ago
Tied for #1 by agentic index (with Opus 5).
66.
by colingauvin 2mo ago
At least by API usage, they aren't yet lower cost than subscriptions. Not sure about GLM's subscription plans though.
67.
by colingauvin 2mo ago
Not with my credit!
68.
by colingauvin 2mo ago
...do I take out a double mortgage to buy a 4 Spark cluster?
69.
by colingauvin 2mo ago
This should be trivially demonstrable if it actually works. Cosine similarity is not a good metric, however, for attention compression.
70.
by colingauvin 2mo ago
It's 3rd at 58.
71.
by colingauvin 2mo ago
It's 7th (!!!) overall on the agentic index, above Terra.
72.
by colingauvin 2mo ago
Ignoring the impracticality of trying to deploy this, codon bias exists for a reason, it's a translation-regulation mechanism and you'd wind up having to do a massive amount of protein engineering to not mess up everything.
73.
by colingauvin 2mo ago
Yeah, for someone like me in a lot of ways it's a dream come true. I have domain expertise in my field, I have a ton of computer knowledge due to building HPC clusters and working in Linux every day for years, I have lots of problems c
74.
by colingauvin 2mo ago
The real value right now is in figuring out how to generate robust data cheaply and quickly. I'd wager that the effect of a good model on marginal data is small, but the effect of a marginal model on great data is probably quite large.
75.
by colingauvin 2mo ago
Yes, and it's relatively good for on-rails data collection and data processing pipelines that would have previously needed occasional human intervention. Especially now that I can run something like DeepSeek v4 Flash on a couple of RTX
76.
by colingauvin 2mo ago
I have no love for Amodei (or any of the other AI magnates), but I am concerned by the allure of handwaving away all his points due to his hubris and arrogance. Make no mistake, even this post is mired in it, but the fundamentals of what he
77.
by colingauvin 2mo ago
In some ways, but it's tough to say if that's my ADHD or not. It's far too easy to leave one branch of reasoning now and jump to another whenever progress gets difficult. Though in some areas where I can sustain interest, AI
78.
by colingauvin 2mo ago
I'm a structural biologist at a mid-sized biotech. I use AI tools daily. They make accomplishing the same things I was able to accomplish before quite a lot faster and easier. They don't help me magically accomplish new things tha
79.
by colingauvin 2mo ago
No, I'm saying AI has not introduced any fundamentally new capabilities to anyone in this space. Making a virus that grows in a cell is one thing. Making a virus that is an existential risk to humans is altogether another and would req
80.
by colingauvin 2mo ago
No, a human could do (and have, many many many times, see like half of molecular biology) this as well. If anything this will help with vaccine development. Edit: I'm a PhD biochemist that has done a fair bit of virus modification. Lov
81.
by colingauvin 2mo ago
For DS4 Flash, with 2x Sparks, I am getting 35-85 TPS in single stream, fresh context after quite a bit of RoCe config and the DSpark MTP, on vLLM with Ray and tensor parallel = 2. For multi-stream, it tops out all stream at well north of 1
82.
by colingauvin 2mo ago
For Flash there are some excellent Q2/Q4 hybrids. I know that model was QAT so it handles Q4 better but the meta on quantization seems to be shifting a little bit to be more intelligent about what exactly gets quantized.
83.
by colingauvin 2mo ago
>Run the same exact prompt 100x in a single step test (one prompt, one response), hash the full responses, and you’ll see 10-50+ unique responses. The higher the unique the worse your prompt; focus on the system. A highly tuned system pr
84.
by colingauvin 2mo ago
I sort of do. For scheduled things I am using Goose and subagents per task. So I wake up at 6 am to a briefing that was composed by a ton of different agents. That works well in a narrow programmatic setting, but it breaks down in certain n
85.
by colingauvin 2mo ago
I've been building this for the last 6 months or so. I've basically got it working. The model is not the issue, the infra is. Keeping everything in context just isn't possible and LLMs, even Fable, don't mode switch well
86.
by colingauvin 2mo ago
...are they going to come to my house and delete my local copy? Open weights are a one-way ratchet.
87.
by colingauvin 2mo ago
16 DGX Sparks can run K3 at a reasonable TPS. So that's $64k. 2 DGX Sparks can run DS4 at 1 million context with 50 TPS so that's $8k. 1 A4500 can run 35A3B. Those are about $1200 new. 27B actually takes more hardware to run than
88.
by colingauvin 2mo ago
Claude 4.5/4.6 - absolutely agree. Fable 5? From my (limited) testing, also reasonable to interact with. Opus 4.7/4.8/5? Absolutely smug and antagonistic and preachy. I'm constantly fighting with it to stop fighting me a
89.
by colingauvin 2mo ago
DS4 Flash Q2/Q4 mixed quant fits on a DGX Spark (a $4000 device which is not particularly unheard of expense for Apple customers), and is indistinguishable for me from Opus for my personal daily use/assistant benchmarks[0]. [0] ht
90.
by colingauvin 2mo ago
Prefill is survivable if you cache well. But what kills me is the context. Qwen 27 needs a ton of room for KV Cache. I guess not an issue on a 128 GB Halo or Spark, but if you are running of consumer/prosumer GPUs it's miserable t

More ›