6 ms·
> FERRET is trained on 8 A100 GPUs So Apple uses NVidia internally. Not surprising, but doesn't bode well for A Series. Dogfooding. [edit] I meant M series, A
by tambourine_man 3y ago
> FERRET is trained on 8 A100 GPUs
So Apple uses NVidia internally. Not surprising, but doesn't bode well for A Series. Dogfooding.
[edit] I meant M series, Apple Silicon
- hhh 3y agoWhy would they dogfood Apple Silicon for training models? Seems like a waste of developer time to me.
- tambourine_man 3y agoApple doesn’t even sell NVidia cards on their Mac Pros. Are they training it on Linux? I think Apple would strive to be great at all computing related tasks. “Oh, Macs are not good for that, you should get a PC” should make them sad and worried. AI/LLM is the new hot thing. If people are using Windows or Linux, you’re loosing momentum, hearts and minds… and sales, obviously.
- Gorgor 3y agoBut no one is training these kinds of models on their personal device. You need compute clusters for that. And they will probably run Linux. I'd be surprised if Microsoft trains their large models in anything else than Linux clusters.
- tambourine_man 3y agoApple used to sell servers. I don’t thing they should settle for “just use Linux” in such and important field.
- MBCook 3y agoWhy does the OS matter for training models? Apple would want to train models as fast as they could. Nvidia provides an off the shelf solution they can just buy and use for a very reasonable price and sell on the second hand market. If they wanted to use their own hardware they would either need more of it, which would cost a lot and divert production from sellable devices; or they would need to make special chips with much bigger neural engines, which would cost even more. Also Apple uses public clouds for service stuff. They may not even own any hardware and just be renting it from AWS/Azure/GCP for training.
- tambourine_man 3y agoI feel like I’ve answered the issues you raised in this thread already.
- hhh 3y ago> Used to Exactly, over a decade ago...
- SpaceManNabs 3y ago> But no one is training these kinds of models on their personal device on-device transfer learning/fine tuning is def a thing for privacy and data federation reasons. Part of the reason why model distillation was so hot a few years ago.
- hermannj314 3y agoIf a train with a GM diesel engine delivers raw materials to a Ford factory for making F150s, you would conclude that consumers whould start driving trains to work? Is that your argument?
- tambourine_man 3y agoNot at all, just that the engineers at Ford would be more proud if the train used their own diesel engine. And that this kind of thing affects public perception. “Ford is not for heavy duty”
- causal 3y agoDon't think that follows. I doubt all their cloud services run on Apple hardware. They make consumer devices.
- nicolas_17 3y agoAFAIK all their cloud services run on x86 hardware with Linux, including Xcode Cloud (which runs macOS in a QEMU VM, in a way that only Apple can technically and legally do).
- threeseed 3y agoIt has always been x86 Linux going back to the NeXT days. Today they have large scale Kubernetes ands some legacy Mesos clusters.
- bjtitus 3y agoThis They ceded the data center environment years ago.
- nicolas_17 3y agoApple doesn't even support NVidia cards on their Mac Pros. The technical details are above my head, but the way Apple M* chips handle PCIe make them incompatible with GPUs and other accelerator cards. Whether you use macOS or Linux.
- tjohns 3y agoApple doesn't even make rack-mount server hardware. It's not that surprising. Apple makes very capable, efficient devices for end users and content producers. End users do not normally need to train new models.
- microtonal 3y agoThey do make rack mount hardware: https://www.apple.com/shop/buy-mac/mac-pro/rack https://www.apple.com/shop/buy-mac/mac-pro/rack Don't know if that qualifies as server :).
- tambourine_man 3y agoThey used to sell the best developer’s machines. Before Docker, at least. Don’t developers need to train models? And if they are doing so on Linux or Windows, wouldn’t that make it easier for them to better or exclusively support the end product on the one they’re more used to? You want to be the platform where the excitement is happening.
- deleted 3y ago[deleted]
- threeseed 3y ago> Are they training it on Linux Yes. Apple has always run their servers on Linux e.g. App Store, iTunes. And training isn't right now an end user activity but something reserved for server farms.
- shepherdjerred 3y ago> I think Apple would strive to be great at all computing related tasks. “Oh, Macs are not good for that, you should get a PC” should make them sad and worried. What percent of Apple's customers train models? Does it even crack 1%? Apple already fails for many types of computing, e.g. any workflow that requires Windows or AAA gaming.
- tambourine_man 3y agoRight, and they are trying to attract games and gamers. “Macs are bad at gaming” shouldn’t be something they settle with.
- sxg 3y agoBy "A series" are you referring to the Nvidia A100 or the Apple A-series iPhone/iPad chips? If the latter, I don't think you can draw that conclusion. Training has memory and processor requirements that are very different from inference. You don't need iPhones and iPads to train models—you need them to run models. These are two very different things.
- tambourine_man 3y agoApple Silicon, sorry for the ambiguity. Apple sells Macs too. That’s where I’d hope they would train their models.
- woke_neolib 3y agoApple apparently uses Google Cloud, so it's that or TPUs!
- tambourine_man 3y agoThey use many clouds. But LLM should be their core business and they usually don’t outsource that.
- blackoil 3y agoThat's not how s/w or any development work. Even if there is a team working on M4 SuperUltra++ which competes with H100, in meanwhile, s/w team will continue to use what is available which may be Intel/AMD PCs with Nvidia GPUs or GoogleCloud/Azure/AWS.
- tambourine_man 3y agoSure, but there’s also the possibility that they aren’t developing the SuperUltra++
- gooob 3y agoyeah, aren't the new M3 chips supposed to be really good for ML training?
- cryogenicfire 3y agoI feel like Apple is only testing the waters with AI right now, but perhaps if they get involved enough they'll spend money on their own compute infrastructure? Nvidia is kind of the king at GPU compute right now, and developing comparable hardware is no small or cheap task, but I think Apple is in a very good position to be able to make it work---if they decide to invest in it. But honestly, as far as corporate feud goes, I feel like companies will happily suck it up if it makes some process cheaper and/or easier
- tambourine_man 3y agoI think you’re completely correct, but if they were caught off guard by the AI train, they shouldn’t be testing the waters now. It should be treated as an existential threat.
- cryogenicfire 3y agoI've always wondered why Apple is never in the conversation of modern AI, like until now it almost feels like they've been just watching from the sidelines without taking part in all the commotion Maybe they can just afford to observe before making major decisions on the direction of the company... For a company like Apple, I feel like they won't lose their customer-base just because they are taking the AI race slowly, in fact Apple has often been late to introducing very common feature
- cryogenicfire 3y agoI hit reply before I finished typing XD ... Anyways, my point being that Apple will gladly introduce a polished product a couple of years after everyone else has already done it, and their target audience will still applaud their work and give them money. Apple for some reason simply _can_ afford to test the water
- fragmede 3y agoYou can edit your comments shortly after you make them.
- hmottestad 3y agoApple had a falling out with Nvidia a number of years ago. I believe they were using chips from Nvidia in their MacBook Pros, the first to come with both integrated and discrete graphics, but the solder between the chips and the motherboard kept cracking and a rather large number of MacBooks needed to be repaired. https://techcrunch.com/2008/12/09/scientists-nvidia-put-faulty-solder-in-new-macbook-pros/amp/ https://techcrunch.com/2008/12/09/scientists-nvidia-put-faul...
- dcchambers 3y agoAs long as the inference can be done locally on their chips I don't at all think it's a big deal to train models on Nvidia/other hardware. Are all the iCloud servers running on Apple silicon? I assumed they were running on standard rack mounted hardware.
- tambourine_man 3y agoI think Apple considers cloud infrastructure a necessary evil and a commodity. AI isn’t, yet at least, and I don’t think they can afford to treat it as such.
- deleted 3y ago[deleted]