3 ms·
In a lot of Apple's hardware, the fastest way to inference is to ignore the built-in Neural hardware and use Metal Compute Shaders. Suffice to say that simply s
by smoldesu 3y ago
In a lot of Apple's hardware, the fastest way to inference is to ignore the built-in Neural hardware and use Metal Compute Shaders. Suffice to say that simply shipping dedicated silicon is not equivalent to accelerating everything well, or supporting it upstream.
- intrasight 3y agoSMOB (simply matter of bytes) as they say. Will they finally invest in AI software? My take from this announcement is that they will. Dedicated silicon with a dedicated hardware stack shouldn't be so hard for Apple. They can throw considerable resource at the problem.
- smoldesu 3y agoIt would be great to see, if they pull it off. I suspect that Apple's desire for control and curation (as well as limited liability) will prevent them from making an effective "one size fits all" model, though. Considering that local AI isn't a paid service or a system-seller, it makes more sense for Apple to double-down on software support rather than trying to Sherlock the world with their own model. Time will tell what the future holds, but I sincerely believe it's too late for Apple to ride the AI bubble. The iPhone is their last stand; they need to deliver a competitive AI experience, ideally one that's open and supports older hardware models. Their work is cut out for them.
- intrasight 3y agoI think they'll be training-agnostic (will use whomever's hardware gets it done) but will optimize their silicon and hardware stack for running models locally. This is VERY EARLY days of use-facing AI (besides the Nvidia bubble, which is late-days)
- smoldesu 3y agoI definitely agree that Apple will optimize their SOC, but IMO their best solution is to just copy Nvidia's homework. Dedicated inferencing hardware is power-hungry and rarely any better than the beefed-up GPU or even a RISC CPU core at a decent clock speed. Integrating AI at the GPU-level lets you get dual-purpose functionality out of the same silicon; it's what Nvidia has built towards since they started shipping CUDA. > besides the Nvidia bubble, which is late-days The Nvidia bubble might just be beginning, if we keep relying on phrases like "training agnostic" to pay our dividends.