bigyabai

29 ms·

691.
by bigyabai 4mo ago
The M5 Max's GPU is slower than a single RTX 3090, though. Both machines will be stuck in the 30-50GB model range, but the 3090 would have faster token prefill and faster decode speeds (614GB/s on M5 Max vs 936GB/s on 1x 3090
692.
by bigyabai 4mo ago
Larry Sanger also founded a Wikipedia alternative, then shut it down, then founded a blockchain-based Wikipedia alternative, then shut that down too. Third time's the charm?
693.
by bigyabai 4mo ago
If you're fine overpaying for a throttling computer, you could buy 40-series cards and underclock them to the same TDP of a Mac Studio. You'd probably get faster prefill speeds, as well as better drivers for accelerated transcode
694.
by bigyabai 4mo ago
The M7 Max and M7 Ultra will likely prefill-bottlenecked at 100GB+ scale inference. Layered 6090s would not be.
695.
by bigyabai 4mo ago
To be fair to Jolla, their brand predates the concept of "digital sovereignty" as we know it today. They were making Android alternatives even before China or Apple did.
696.
by bigyabai 4mo ago
> We don't let passengers fly on unvetted jumbo jets either and we prevent them from flying when they have problems. This is Mr. Fart's Favorite Colors all over again. Our "vetting" process is not any more useful th
697.
by bigyabai 4mo ago
> M1 released 6 years ago, but AMD/Intel still can't get close to ARM cores in IPC. > teams from Intel and AMD absolutely HAD to know the truth. These people are professionals that acknowledge IPC is a stupid metric. If you
698.
by bigyabai 4mo ago
> That would be the kind of thing you could sell as many as you could make. People said that about the M1 Ultra Mac Pro, a few years before it was discontinued. I don't think there are many HPC customers looking at Apple hardware.
699.
by bigyabai 4mo ago
Because Apple doesn't sell socketed memory, but instead integrated SOC products that usually stimulate growth in their much more profitable services segment?
700.
by bigyabai 4mo ago
> I have zero doubt it'll reach mainstream adoption Be careful. I said this 10 years ago about OpenCL, and I've ate crow ever since.
701.
by bigyabai 4mo ago
Competitive for 16-24GB dGPUs, but for 100gb+ inference workloads it's going to be a decode bottleneck. For smaller models it'd be fine, but the same goes for the smaller GPUs. In particular though, the fatal bottleneck is the wea
702.
by bigyabai 4mo ago
Without a "zero regulation" option, you're just begging people to select one of your troll answers.
703.
by bigyabai 4mo ago
The integrated GPU. Not enough compute onboard to handle prefill for 100gb+ models, and the decode is constrained by memory bandwidth that's lower than most dGPUs that price. Apple would be in a much stronger spot right now if they did
704.
by bigyabai 4mo ago
They very well could have. Apple was the only company poised to take on CUDA with OpenCL, and they got pantsed so hard by the HPC industry that the Mac Pro got discontinued entirely . Apple could have added a couple trillions to their valu
705.
by bigyabai 4mo ago
How are people getting so upset over this? OpenAI asked for regulation to validate their scaremongering. Now they've allied with the government, just like they said they would. It's 2026 now, you can't pin your hopes and drea
706.
by bigyabai 4mo ago
> And in the end, everyone is worse of. I don't know, it really depends on the product. If we were all still paying for proprietary UNIX in the current year, then it's likely that these startups would never exist in the first p
707.
by bigyabai 4mo ago
> From GLM 4.7 flash GLM 4.7 Flash is a 30b model that was far behind SOTA at launch, and I know that because I pay for z.ai inference and have run the model locally. Qwen and Deepseek V4 Flash have the same issue, and beg the questi
708.
by bigyabai 4mo ago
Opencode has a toggle for planning and edit modes, you might like that.
709.
by bigyabai 4mo ago
They aren't. Apple Silicon is unusable for interactive prefill and decode speeds in agentic workflows and SOTA LLMs.
710.
by bigyabai 4mo ago
The US and allies should have banned Chinese tech when they cloned STM-32s without ARM licenses and refused to pay royalties for the IP. You're a day late and a dollar short, if this isn't an ironic shitpost intended to foment nat
711.
by bigyabai 4mo ago
We have to get real, here - most people are not replacing GPT or Claude with local inference, even on M5. If you can afford to do that (RAM shortage or not), then you are in the minority of customers. Alleviating the memory constraint would
712.
by bigyabai 4mo ago
The flip side is that the pot is now boiling. Windows and macOS are both replete with advertisements and service upsell, which is something that nontechnical and technical users both pick up on. It's been expanding the discussion of al
713.
by bigyabai 4mo ago
Message recieved, porting my webapp to Electron ASAP.
714.
by bigyabai 4mo ago
I'm just genuinely convinced that Apple's AI flywheel is going in reverse. Their killed their golden goose with OpenCL, which had a genuine shot at dethroning CUDA if Apple took it seriously. It had industry-wide buy in and multip
715.
by bigyabai 4mo ago
Me too! The problem is that people don't love having 128gb of DDR5 held back with a laptop-grade iGPU. It puts up strictly non-interactive speed for LLMs of that size. When you layer those same models across 128gb of dGPUs, then you
716.
by bigyabai 4mo ago
Personally, I doubt it. Apple hamstrung themselves with unified SOC memory, there are cheap dGPUs that smoke the M5's prefill speeds and even have faster decode too. Apple is running up against the limitations of putting a mobile integ
717.
by bigyabai 4mo ago
Nvidia wasn't going to buy them. Unless Mojo intended to compete toe-to-toe in the hardware space, they were destined to get bought out by a hardware underdog at some point or another. This is where an industry-spanning consortium woul
718.
by bigyabai 4mo ago
It's almost as if the majority of high-quality, permissively licensed online prose exhibits a liberal bias. Who could have possibly seen this coming!?
719.
by bigyabai 4mo ago
Yep, Valve is working with Nvidia to create an installer with the proprietary Nvidia drivers bundled in. FWIW, Nvidia's Wayland experience has been very usable with post-550-Series drivers. If you have an Nvidia GPU that supports more
720.
by bigyabai 4mo ago
The tradeoff is that the M3 Ultra's GPU loses to laptop GPUs in compute benchmarks. All of that bandwidth is wasted idling for token prefill. For inference workloads, it makes a lot more sense to optimize for prefill/ttft before m

More ›