gcr

6 ms·

91.
by gcr 5mo ago
interesting! I might give omlx another chance, thank you
92.
by gcr 5mo ago
Unsloth’s guides on getting various models running are great starting-off points for the “practicioner’s side” of things. Note that they include settings for llama-cpp, ollama, and other runtimes in addition to their own “unsloth studio” (t
93.
by gcr 5mo ago
If your hardware fits K_M but not K_XL, should you prefer going down to a lower quantization’s XL or sticking to the higher quant’s Q_M?
94.
by gcr 5mo ago
on my M1 Max, MTP consistently lowers my performance! I’ve tried both llama-cpp’s recently landed MTP support (cloned and built Tuesday) as well as one of the other forks a few weeks ago. Suspect nobody’s done a comparison on hardware like
95.
by gcr 5mo ago
Pardon. You had mentioned open weight models so I assumed you meant self-hosted
96.
by gcr 5mo ago
Agreed. Prefill kills me for local model work. The model reads much faster than it writes, but I'd love to get a sense for how fast the model can read large source conversations.
97.
by gcr 5mo ago
Depends. ROCm is pretty well-supported for example. Non-NVIDIA backends tend to get less support and new features land slower, or features that are expected to improve performance wind up hurting it instead. That sort of thing. For basic “t
98.
by gcr 5mo ago
Why is this answer notable to you? Wouldn’t you be more surprised / concerned if the model refused to to criticize Google?
99.
by gcr 5mo ago
how could running the qwen GGUF phone home? that would require cooperation with the inference backend (llama-cpp), or some kind of model exploit. It’d be far easier to pay the agent harness devs or supply-chain some plugin or something, tha
100.
by gcr 5mo ago
Look for an open release from the Qwen team in the coming weeks. They like to showcase their proprietary models first, which score higher on benchmarks anyway due to model size.
101.
by gcr 5mo ago
There are two flavors of Qwen 3.6: - A 27B "dense" model - A 35B "Mixture of Experts" model, which activates only 3B parameters for each token. For your hardware, I strongly recommend `unsloth/Qwen3.6-35B-A3B-GGUF:Q
102.
by gcr 5mo ago
here's a simple setup to get you started on an Apple M1 Max from 2021 with 32GB VRAM. it will download 20GB of models to `~/.cache/huggingface/hub`, which you can delete when you're done. /Users/gcr&#x
103.
by gcr 5mo ago
What’s the price point for getting into that sweet spot? I’m on an M1 Max with 32GB VRAM, so I’m looking forward to the 27B or 35B-A3B models. Is dropping $5k for an RTX 6000 or a DGX Spark really the best option?
104.
by gcr 5mo ago
how do you know your harness design isn’t just overfitting on your test set?
105.
by gcr 5mo ago
> Railway owns our vendor choices, and we ultimately own this one. Your customers don't care whether the failure was Google or Railway; they see your product. Your uptime is our responsibility, and we'll keep delivering on it.
106.
by gcr 5mo ago
They’re talking past each other. For some, “high quality” is a comment about implementation elegance. For others, “high quality” is about duct-taping crude implementations together to fashion a kickass user experience. To most, quality prob
107.
by gcr 5mo ago
Exactly! Thats a complimentary problem for new grads
108.
by gcr 5mo ago
Your last sentence is factually not true. Friends across five different companies report that LLM adoption is now a key metric in their performance review.
109.
by gcr 5mo ago
For college graduates, LLM tech is an existential threat to their livelihoods by making it so much harder to start a career without connections or pedigree.
110.
by gcr 5mo ago
canirun doesn’t work well for MoE models. Qwen’s 35B A3B model works amazing on my M1 Max 32GB, but canirun thinks it barely runs.
111.
by gcr 5mo ago
There’s some weak evidence against this actually. Harness design makes a huge difference for tiny local models for example: https://itayinbarr.substack.com/p/honey-i-shrunk-the-coding-... I have only my own personal ex
112.
by gcr 5mo ago
DwarfStar4 is a small LLM inference runtime that can run DeepSeek 4. The blog post implies that it currently requires 96GB of VRAM. For others who are lacking context :-)
113.
by gcr 5mo ago
When I hear "claude code one-shotted X" and X is a novel problem, I mentally substituted "the agentic harness that I tried one-shotted X," since that's what they're saying. Getting any smart model to take a loo
114.
by gcr 5mo ago
That's what the comment is saying. In normal repositories, version control acts as a record of the momentum of the direction the product was taking. If it's just "_old" and "_new," the developer has to read and
115.
by gcr 5mo ago
Google powerwashes your corp Chromebook when they let you go. A friend was composing an email on the train when their screen went black and the device reset itself to factory settings. They even send the “you’re being fired” email to their
116.
by gcr 5mo ago
You're not understanding the difference between API billing and subscirption billing. Subscription usage refills every five hours. API billing doesn't.
117.
by gcr 5mo ago
Ya but that monthly usage doesn't refill every five hours or every week the way your subscription usage does. Once you're through the $200, you're done until next month.
118.
by gcr 5mo ago
As a fellow zed user, I fear this could affect us (though i'm not 100% confident). Zed uses their own adapter to implement claude's ACP support, and it's likely using `claude -p` (the agent SDK) under the hood. Hoping the Zed
119.
by gcr 5mo ago
A better link: https://support.claude.com/en/articles/15036540-use-the-clau...
120.
by gcr 5mo ago
This is HP’s current philosophy towards consumer desktop inkjet and laser printing, and customers universally hate it. No thanks!

More ›