3 ms·
Regarding Tensor cores, it does have them as part of the 32 core Neural Engine. Apple considers AI/ML a consumer feature, all the way down to the iPhone hardwar
by zamadatix 3y ago
Regarding Tensor cores, it does have them as part of the 32 core Neural Engine. Apple considers AI/ML a consumer feature, all the way down to the iPhone hardware. At the same time, this isn't a data-center supercluster. It's still just a mid sized workstation.
- earthnail 3y agoThe Neural engine is for inference, not training.
- sillysaurusx 3y agoThere’s no difference between inference and training. Training is simply ~4x more expensive inference. The submitted article also talks about training models.
- bufo 3y agoThere is a difference. We train with large batch sizes these days. The ANE silicon size is tiny and can't do the large matrix multiplications for big LLMs with or without a batch size higher than 1. Meaning that it cannot saturate the RAM bandwidth and that you're better using off the much bigger GPU on the Apple die.