zozbot234

6 ms·

31.
by zozbot234 1mo ago
How does this relate to the more recent work on the M4 ANE found at https://maderix.github.io/articles/ ? Does the M4 and later ANE expose any additional capabilities, or is it just a higher-performance iteration of th
32.
by zozbot234 1mo ago
Because his letter misuses the word "misalignment" for what's very clearly a capability gap. To anyone familiar with that sort of language, his prose is directly implying that evil superintelligent AIs are deliberately wri
33.
by zozbot234 1mo ago
> AI is now capable of constructions so complex that no human or human team can unpack. How can we possibly know this when we haven't even seriously started on the endeavor of actively reverse engineering these AI-generated proofs?
34.
by zozbot234 1mo ago
> Companies who run locally, are perfectly able to buy a few H200/B200 and get a setup that run a model that almost rivals Opus 5.0 in their office. I agree with your broader point about Flash being about speed not total model size,
35.
by zozbot234 1mo ago
Actually this ought to run quite well with SSD streaming. The MoE expert sparsity seems to be similar to DSv4 Pro (hence exceptionally sparse) but with far fewer total and activated params. The added engram params can reside on disk as we
36.
by zozbot234 1mo ago
"Movement is the main thing" is precisely why pursuing compute-in-RAM makes some sort of sense to begin with. But DRAM fabrication processes are quite specialized and do not perform well with pure compute logic. The overall prof
37.
by zozbot234 2mo ago
If AI compute is a transformative technology compared to industrialization (that's a huge "if", essentially positing a singularity-like outcome), that $1T-$2T/yr at current prices might be a tiny fraction of future GDP
38.
by zozbot234 2mo ago
> Anthropic and OpenAI frequently “reset” customer limits to allow them to use even more resources at no additional cost. Surely this applies to fixed-price subscriptions, not per-token spend? Large enterprises (the "very very resou
39.
by zozbot234 2mo ago
> Small models are rapidly growing in capability, require less compute to train and serve According to Jevons' paradox a reduction in resource requirements (improved resource efficiency for the same payoff) leads to an increase in
40.
by zozbot234 2mo ago
The typical bottleneck to wider batching on consumer hardware is memory capacity for the KV-cache, not compute (even unified memory/iGPU-based platforms have enough compute to allow for some batching, and SSD offloading changes the s
41.
by zozbot234 2mo ago
The raw margins on proprietary model inference are rumored to be quite high though (they have to successfully defray the entire investment into model training and datacenter capacity for inference, which is massive enough). The API cost you
42.
by zozbot234 2mo ago
> I believe this report has confused Opus (which is known to be around 5T) and Fable. 5T for Opus feels quite high though. DeepSeek V4 Pro is a mere 1.6T and often described as a match with Opus in overall quality. Even the largest open
43.
by zozbot234 2mo ago
Are these directly comparable, though? Doesn't Windows default to using free space on the system drive to grow the swap file in case of memory pressure? Linux has you set up swap files/partitions explicitly.
44.
by zozbot234 2mo ago
If you have model layers/experts that reside in CPU RAM, it's generally better to do that part of inference on the CPU than pay the cost of shipping them over to the GPU. LLM inference is generally bottlenecked by data-movement,
45.
by zozbot234 2mo ago
You're talking about multi-session vs. single-session throughput. A single user can easily leverage multiple sessions via e.g. subagent swarms, especially on a lower-end setup where any single session is going to be quite slow. Satura
46.
by zozbot234 2mo ago
Inherited wealth is not self-sustaining at all: if you spend it down, it disappears. Most inheritors of large windfalls will not manage to pass down that wealth in turn; it takes a rare ingrained cultural attitude to make it genuinely &q
47.
by zozbot234 2mo ago
The frontier labs will do well if they pivot their offering towards more capable, larger-scale models that are inherently harder to both train and deploy for commodity suppliers. Their existing investments in gigawatt-scale datacenters are
48.
by zozbot234 2mo ago
If you want to effectively acquire a broad understanding of the human condition as it has been understood throughout history, the great books of the classical tradition (including, yes, myths and fables that talk about e.g. the sun being dr
49.
by zozbot234 2mo ago
Answering 2,500 independent questions is an embarrassingly parallel workload, all it needs is scale out. It would be more meaningful to know how much time was required for a single complete answer to a difficult HLE question.
50.
by zozbot234 2mo ago
> 99.9%+ of the tech worker population will never be able to build their own servers to run future frontier models. A single local user can run frontier models slowly on a 24/7 basis, which drops hardware requirements by orders of m
51.
by zozbot234 2mo ago
Performance might still end up being bounded by data transfer speed if SSD streaming is heavily used to make up for limited RAM. By comparison, it doesn't take many parallel-batched sessions to make LLM decode compute-bound on typical
52.
by zozbot234 2mo ago
Is that an up-to-date quote? That's not a bad price for 16 TiB of RAM. Once you add the CPU cores and the storage, it's actually an amazing deal.
53.
by zozbot234 2mo ago
> The actual risk that the author does not even broach upon for investors... the thing that will actually torpedo this massive investment are the open source open weight chinese models that commoditize the entire endeavor. I disagree tha
54.
by zozbot234 2mo ago
> I suspect the economics favor centralized servers, if you only look at the aggregated cost to serve X number of users' tokens. The economics of real-time, low-latency inference of very large near-SOTA models will heavily favor
55.
by zozbot234 2mo ago
Read disturb ought to be quite rare, especially on a fresh drive that was written only once or a handful of times (WORM-like usage). Practically, it's not likely to be an issue even with very heavy read workloads.
56.
by zozbot234 2mo ago
Yup, I hate to engage in anything that looks like a "shallow dismissal" but the project documentation seems to outright contradict itself wrt. whether it's running the model at genuinely native precision (though the claimed 3
57.
by zozbot234 2mo ago
The native weights are 4-bit for the sparse experts, and they quantize to ~80GB with limited degradation in real-world performance. That's a viable target for 64GB with SSD streaming, though it will be slower than keeping the whole th
58.
by zozbot234 2mo ago
Reading between the lines, it's fairly clear that Mythos Preview only got its reported results in offensive cyber thanks to a highly specialized harness and humongous amounts of test-time compute. We also don't know what kind of p
59.
by zozbot234 2mo ago
This is not entirely true: non-constructive proofs can be fully understood within intuitionistic logic as proofs of negative statements, viz. statements about what isn't constructively true or doesn't constructively exist.
60.
by zozbot234 3mo ago
Only the sparse experts are 4-bit in native precision, and those take up ~25GB of active params. The dense parameters' native footprint is ~115GB. So in order to infer that natively on a 16GB machine you'd need to reload around

More ›