dannyw

5 ms·

61.
by dannyw 2mo ago
As a counterpoint, my homelab/home-LLM hardware has appreciated in value by about 60% since I bought it. Of course, it's not real unless I sell, and the value will eventually go down, but so far I have significant paper profits. A
62.
by dannyw 2mo ago
ARC Prize has retested Luna after the discount and validated identical performance. (Also, quantization isn't inherently bad or damaging when done properly, e.g. QAT). These APIs are used heavily by enterprises at scale; with lots of p
63.
by dannyw 2mo ago
You might already know this, but a large part of test-time compute / 'overthinking' is just letting the model do more passes, and refine its activation residuals more. For example, even if you make thinking tokens literally j
64.
by dannyw 2mo ago
Another self-inflicted own courtesy of US government policy. While I think China would always get to hardware self-sufficiency eventually, all export controls have done is (1) accelerate China's development, and (2) divert revenue that
65.
by dannyw 2mo ago
Looks like the M5 Ultra Studio wait times are going to increase again. Already at 10-12 weeks, I wonder how long it'll go?
66.
by dannyw 2mo ago
Or 3, they find some bug/regression in their pipeline; maybe they didn't quant parts of a model properly, maybe their inference engine had a bug, maybe some pinned MoE expert wasn't pinned, etc... That's very plausible t
67.
by dannyw 2mo ago
"Black-Box On-Policy Distillation of Large Language Models", Microsoft Research, https://aka.ms/GAD-project > 'GAD consistently surpasses standard sequence-level distillation, delivering superior generaliz
68.
by dannyw 2mo ago
to be fair, OMP/Pi is also just a better harness. e.g. https://www.databricks.com/blog/benchmarking-coding-agents-d...
69.
by dannyw 2mo ago
With the amount of Claudish on the internet now, and in source code repositories (how many Claudish README.mds have you seen?), you don't have to make a single API call to end up with a model that talks like Claude. And critically, lik
70.
by dannyw 2mo ago
Better than a DGX Spark running Linux? Absolutely not (I own both). Real-world matmul is about 4x higher on a DGX Spark than a M3 Ultra.
71.
by dannyw 2mo ago
256GB is enough for DSv4 Flash, expect maybe ~30tg/s, and a lot better profile. I'm using Flash heavily, and I would describe it as nearly as intelligent as Sonnet-class in agentic coding, but more usable. Less world knowledge of
72.
by dannyw 2mo ago
The product marketing managers write for the target audience; of course they know lots of people buying these are the local LLM crowd. I actually wouldn't want a serious model shipping 'on disk'. Models release so often, it&#
73.
by dannyw 2mo ago
A lot more upgradable too. Those empty DRAM slots can get populated one day!
74.
by dannyw 2mo ago
Oops, thank you, corrected.
75.
by dannyw 2mo ago
AMD Strix Halo unified memory still requires allocating X to GPU, x to CPU at UEFI. So not as flexible as apple's unified memory.
76.
by dannyw 2mo ago
My card (RTX Pro 6000) is always doing something all the time from my queue; like some synthetic dataset generation up next. I still actively use runpods and openrouter for scaled stuff, I was spending a bit and then did the maths, and inve
77.
by dannyw 2mo ago
They probably literally don't have enough NAND to go around. 768GB of memory (48GB x 16) is enough for nearly 100 iPhone 17s; that's $80k (corrected) of iPhones at MSRP, although likely much lower margins than these high-RAM boxes
78.
by dannyw 2mo ago
I dunno, Apple's been throwing many bones to their customers who use their Macs for AI stuff; like Apple working with, and signing TinyGPU's NVIDIA eGPU drivers. Plus introducing features like RDMA over thunderbolt, which is criti
79.
by dannyw 2mo ago
M5 is excellent, they’ve finally gotten their own tensor cores. Good for inference; however if you like to train, data format support and effective performance is limited (M5 Pro). Some hardware features are not exposed or extremely slow. Y
80.
by dannyw 2mo ago
If you like to experiment with training / finetuning / etc on LLMs, these are actually incredibly ‘cheap’. 1.2TB/s memory bandwidth unlocks a lot with 256GB unified, and agentic AI is pretty good at optimising performance. Fo
81.
by dannyw 2mo ago
Build quality of the Neo is extremely good, I love the keyboard — it’s more tactile and reminds me of early 2010s MacBooks. I’ll be selling my M4 MBA soon, I genuinely use the Neo more. Huge difference in typing experience. Great repairabil
82.
by dannyw 2mo ago
The DRAM market is cyclical. I don’t think anyone truly knows when, but it will happen. Fab capacity is being bought online; there’s just lead time. Noticeably greater intelligence is being achieved at the same number of parameters (see: Qw
83.
by dannyw 2mo ago
I'm not sure if I'm following. Apple is capable of creating a lookup table, or an atomic database read query.
84.
by dannyw 2mo ago
Unfortunately you can only do that for yourself, your recipients almost certainly aren't doing the same. It doesn't address the "gmail deliverability" issue.
85.
by dannyw 2mo ago
I think GP's point (which I agree with) is that the functionality is almost essential. When it's not part of the spec, then you end up with differing off-spec implementations, and again, centralization risk. This would be like the
86.
by dannyw 2mo ago
Apple’s devices generally have no or very underpowered thermal solutions; but it also partly regulates battery life. (Seriously, they rely on air cooling for Neo and MBA — these chips hit 105deg easily and throttle, intentionally) Adding a
87.
by dannyw 2mo ago
Shipyard has been one of the main maintainers because they have previously been an aggregator of funding — think consulting firm, that gets grants and hires people. Protocol Labs is moving and offering to pay (as far as I understand) the sa
88.
by dannyw 2mo ago
For the "hide my email" feature, Apple currently and has always used "@icloud.com". This masks you amongst all iCloud users. They were planning to change it to a custom domain, which would allow sites to easily filter ou
89.
by dannyw 2mo ago
> What's much more likely is that your US AI provider is promising not to train on your data but is doing so anyway. There's genuinely no evidence of this for OpenAI and Anthropic. It's impossible to disprove, but I don&#x
90.
by dannyw 2mo ago
Great reminder to be using multiple providers (i.e. Anthropic 1P, Bedrock, Vertex AI, Azure) with auto-fallback. Bedrock has been fine throughout today; just pick two providers and you have much more stable Claude :) Also, sometimes older m

More ›