4 ms·
Didn’t Apple say they’re using their own hardware for serving some of the AI workloads? They dubbed it ‘Private Cloud Compute’. Not sure how much of a vote of c
by ra7 2y ago
Didn’t Apple say they’re using their own hardware for serving some of the AI workloads? They dubbed it ‘Private Cloud Compute’. Not sure how much of a vote of confidence it is for Nvidia.
- whimsicalism 2y agonot for gpt4o workloads they aren't going to
- jsheard 2y agoPlus even if Apple is using their own chips for inferencing, they're still driving more demand for training, which Nvidia still has locked down pretty tight.
- ra7 2y agoApple said they’re using their own silicon for training. Edit: unless I misunderstood and they meant only inference.
- talldayo 2y agoThey trained GPT-4o on Apple Silicon? I find that hard to believe, surely they only mean that some models were trained with Apple Silicon.
- ra7 2y agoNot GPT-4o, their own models that power some (most?) of the “Apple Intelligence” stuff.
- jsheard 2y agoInteresting, I thought Apple Silicon mainly excelled at inferencing. Though I suppose the economics of it are unique for Apple themselves since they can fill racks full of barebones Apple Silicon boards without having to pay their own retail markup for complete assembled systems like everyone else does.
- whimsicalism 2y agowithout more details hard to say, but i seriously doubt they trained any significantly large LM on their own hardware people on HN routinely seem to overestimate Apple's capabilities e: in fact, iirc just last month Apple released a paper unveiling their 'OpenElm' language models and they were all trained on nvidia hardware
- ra7 2y agoWhich is only a subset of requests Apple devices will serve and only with explicit user permission. That’s going to shrink over time as Apple continue to advance their own models and silicon.
- stetrain 2y agoRight, but are those going to run on Apple-owned hardware at all? It seems like Apple will first prioritize their models running on-device, then their models running on Apple Silicon servers, and then bail out to ChatGPT API calls specifically for Siri requests that they think can be better answered by ChatGPT. I'm sure OpenAI will need to beef up their hardware to handle these requests - even as filtered down as they are - coming from all of the Apple users that will now be prompting calls to ChatGPT.
- whimsicalism 2y agothey're going to be using nvidia (or maybe AMD if they ever catch up) to train these models anyways
- kolinko 2y agonot necessarily so, in terms of tflops per $ (of apple’s cost of gpus, nit consumer), and tflops per watt their apple silicon is comparable if not better
- whimsicalism 2y agoflops/$ is simply not all (or even most) that matters when it comes to training LLMs.... Apple releases LLM research - all of their models are trained on nvidia.
- talldayo 2y ago> and tflops per watt their apple silicon is comparable if not better If Apple currently ships a single product with better AI performance-per-watt than Blackwell, I will eat my hat.
- lxgr 2y agoThey're even explicitly saying: > These models run on servers powered by Apple silicon [...] That doesn't mean that there are no Nvidia GPUs in these servers, of course.
- bbatsell 2y agoThey say user data remains in the Secure Enclave at all times, which Nvidia GPUs would not be able to access. I am quite certain that their private cloud inference runs only Apple silicon chips. (The pre-WWDC rumors were that they built custom clusters using M2 Ultras.)
- talldayo 2y agoNot that it matters anyways, since Apple refuses to sign Nvidia GPU drivers for MacOS in the first place. So if they own any Nvidia hardware themselves, then they also own more third-party hardware to support it.
- 7speter 2y agoMaybe this is way too science fiction, but what are the chances Apple's GPU/AI engine designs on Apple Silicon were a testbed for full sized, dedicated GPU dies that could compete with Nvidia's power in their own data centers?
- talldayo 2y agoVery low? I guess anything is possible, but the M1 through M4 GPUs weren't really anything to write home about. It more closely resembles AMD's raster-focused GPU compute in my opinion, which is certainly not a bad thing for mobile hardware. Nvidia's GPUs are complex. They have a lot of dedicated, multipurpose acceleration hardware inside of them, and then they use CUDA to tie all those pieces together. Apple's GPUs are kinda the opposite way; they're extremely simple and optimized for low-power raster compute. Which isn't bad at all, for mobile! It just gimps them design-wise when they go up against purpose-built accelerators. If we see Apple do custom Apple Silicon for the datacenter, it will be a pretty radically new design. The first thing they need is good networking; a full-size Nvidia cluster will use Mellanox Infiniband to connect dozens of servers at Tb/s speeds. So Apple would need a similar connectivity solution, at least to compete. The GPU would need to be bigger and probably higher-wattage, and the CPU should really emphasize core count over single-threaded performance. If they play their cards right there, they would have an Apple Silicon competitor to the Grace superchip and GB200 GPU.