hedgehog

12 ms·

151.
by hedgehog 4mo ago
Work duration is also not that valuable of a measure, you're usually better off defining the process yourself in code and having that delegate chunks of work to the models. The only real issue there is that it's harder to take adv
152.
by hedgehog 4mo ago
For scale though if three or four chips that size can replicate a Qwen 27B experience that'll be quite useful.
153.
by hedgehog 4mo ago
iTunes and iPhoto both. Given how good the tools are getting, and how much existing sample code is available, it seems likely someone will do a good job of reincarnating them in the near future. Apple broke the apps I used most on the Mac a
154.
by hedgehog 4mo ago
Do you have plans to generate a buildable version of the sources, and do you know the original implementation language (C?).
155.
by hedgehog 4mo ago
I want to like this but it reads like Claude output, how much scrutiny did this get for accuracy?
156.
by hedgehog 4mo ago
The scale of the data and the size of the models don't change the underlying issue, the whole construction of these models is to start with a maximum likelihood language sampler (pre-training) and then massage it into a maximum utility
157.
by hedgehog 4mo ago
Ryzen 395 is what I'm using, anything with 128GB+ of RAM accessible to the GPU should work fine for a 4 bit version of the model (so Spark or Mac Studio should be ok too).
158.
by hedgehog 4mo ago
Same chip, with a 6 bit 35B and 8 bit KV cache I see about 500 prefill and 55 decode at 30k into the context window. MiniMax seemed a bit lower token rate but much, much less prone to 40k tokens of monologue before generating an answer. A p
159.
by hedgehog 4mo ago
The 6-bit versions + 8-bit KV cache seems to save a good bit of memory without a significant loss of quality. The Qwen 35B is pretty fast in my testing, but MiniMax M2.7 230B is in some ways faster (way fewer tokens to arrive at an answer)
160.
by hedgehog 4mo ago
Yes. Divide execution of a change into separate responsibilities. Designate the main chat as the "orchestrator", Opus. You designate a goal, then tell it to grind until it gets there using the following sub-agents in sequence: 1.
161.
by hedgehog 5mo ago
It does, my experience has been that it adds code complexity, deployment complexity, and performance problems. There are some observability benefits, but other ways to solve that. It's possible there are workloads that fit it but not a
162.
by hedgehog 5mo ago
How granular is the control over the internal process? In my experiments I've had some success modeling the work to be done as a DAG of typed artifacts with a combination of code + LLM doing decomposition, transforms, synthesis, and fi
163.
by hedgehog 5mo ago
You know that, and I know that, but for someone who started working more recently the difference between CORBA and punch cards might be a little blurry because they're both so far back they've never seen either. It's like kid
164.
by hedgehog 5mo ago
I went from one dev machine to two at my desk so I connected them via 25GBe. With about 2.8GBps TCP throughput and RDMA available I don't have to think too much about task placement or cross-traffic. (specific hardware: Mellanox Connec
165.
by hedgehog 5mo ago
Does that mean it can work 20% faster?
166.
by hedgehog 5mo ago
I've done some tool-assisted ports (including without original source), the work you already did is probably 1/4 of the way to a web-hosted Rust BattleTris.
167.
by hedgehog 5mo ago
That's enough resources to build on something like the Olmo 3 recipe but with a mix prioritizing their own data and post-training for their own tasks. If they build their own embedding model, index everything in the library, and train
168.
by hedgehog 5mo ago
This was fixed in 26.5 as well as 15.7.7 etc. https://app.opencve.io/cve/CVE-2026-28952
169.
by hedgehog 5mo ago
Pauses are a problem with heap size and structure, not allocation rate, because the pause is caused by GC code that is O(heap size). Making garbage slower reduces the frequency but not severity. This is an issue with most GCs to some degree
170.
by hedgehog 5mo ago
Possibly in your specific application, usually there are a handful of options far less painful than a rewrite. For the original issue of GC pauses, a narrow change is to move problem data to non-pointer-carrying types, or the bigger hammer
171.
by hedgehog 5mo ago
I've run into GC pauses, I think in many (most?) cases there is some class of bulky data that you can either move into slices of pointer-free structs (so the GC doesn't scan them) or off-heap entirely. The workload where GC is slo
172.
by hedgehog 5mo ago
You pay for cache hits on every turn and even with the newest architectures longer context is slower/more energy intensive. Constructing concise turns that reuse prefix and stop when the new context is no longer useful help, as does pu
173.
by hedgehog 5mo ago
The data I've seen is stuff like the KL Divergence comparisons that Unsloth does which show something but not clearly whether there's an observable or significant difference in task performance.
174.
by hedgehog 5mo ago
The model just predicts text, Claude Code etc parse the output and do the actual file creation (or run shell commands that do it). If you have Claude Code installed look in ~/.claude/projects/... and you can see the transcrip
175.
by hedgehog 5mo ago
That's a very insightful observation. How could you explain that using the analogy of a pancake breakfast?
176.
by hedgehog 5mo ago
Have you done comparisons with 4 bit and seen a noticeable difference for coding tasks?
177.
by hedgehog 5mo ago
That pelican looks like it's in Miami for a crypto conference.
178.
by hedgehog 5mo ago
There's clearly a range of outcomes, from the people pushing the envelope in system performance (look at the state of networking ASICs and the code to drive them) to people adopting hyper-complex frameworks for cloud orchestration and
179.
by hedgehog 5mo ago
Similarly I haven't seen Qwen 27B as remotely competitive with Opus, at least Q4 hooked up to Claude Code. What harness are you using?
180.
by hedgehog 5mo ago
If you do that someone still needs to make sure the details make sense which, from experience, sometimes they will and sometimes they won't. When I open tickets using automation I often back into the ticket from a running implementatio

More ›