3 ms·

not for gpt4o workloads they aren't going to

jsheard 2y ago
Plus even if Apple is using their own chips for inferencing, they're still driving more demand for training, which Nvidia still has locked down pretty tight.
- ra7 2y ago
  Apple said they’re using their own silicon for training. Edit: unless I misunderstood and they meant only inference.
  - talldayo 2y ago
    They trained GPT-4o on Apple Silicon? I find that hard to believe, surely they only mean that some models were trained with Apple Silicon.
    - ra7 2y ago
      Not GPT-4o, their own models that power some (most?) of the “Apple Intelligence” stuff.
  - jsheard 2y ago
    Interesting, I thought Apple Silicon mainly excelled at inferencing. Though I suppose the economics of it are unique for Apple themselves since they can fill racks full of barebones Apple Silicon boards without having to pay their own retail markup for complete assembled systems like everyone else does.
  - whimsicalism 2y ago
    without more details hard to say, but i seriously doubt they trained any significantly large LM on their own hardware people on HN routinely seem to overestimate Apple's capabilities e: in fact, iirc just last month Apple released a paper unveiling their 'OpenElm' language models and they were all trained on nvidia hardware
ra7 2y ago
Which is only a subset of requests Apple devices will serve and only with explicit user permission. That’s going to shrink over time as Apple continue to advance their own models and silicon.
stetrain 2y ago
Right, but are those going to run on Apple-owned hardware at all? It seems like Apple will first prioritize their models running on-device, then their models running on Apple Silicon servers, and then bail out to ChatGPT API calls specifically for Siri requests that they think can be better answered by ChatGPT. I'm sure OpenAI will need to beef up their hardware to handle these requests - even as filtered down as they are - coming from all of the Apple users that will now be prompting calls to ChatGPT.
- whimsicalism 2y ago
  they're going to be using nvidia (or maybe AMD if they ever catch up) to train these models anyways
  - kolinko 2y ago
    not necessarily so, in terms of tflops per $ (of apple’s cost of gpus, nit consumer), and tflops per watt their apple silicon is comparable if not better
    - whimsicalism 2y ago
      flops/$ is simply not all (or even most) that matters when it comes to training LLMs.... Apple releases LLM research - all of their models are trained on nvidia.
    - talldayo 2y ago
      > and tflops per watt their apple silicon is comparable if not better If Apple currently ships a single product with better AI performance-per-watt than Blackwell, I will eat my hat.

Plus even if Apple is using their own chips for inferencing, they're still driving more demand for training, which Nvidia still has locked down pretty tight.

ra7 2y ago
Apple said they’re using their own silicon for training. Edit: unless I misunderstood and they meant only inference.
- talldayo 2y ago
  They trained GPT-4o on Apple Silicon? I find that hard to believe, surely they only mean that some models were trained with Apple Silicon.
  - ra7 2y ago
    Not GPT-4o, their own models that power some (most?) of the “Apple Intelligence” stuff.
- jsheard 2y ago
  Interesting, I thought Apple Silicon mainly excelled at inferencing. Though I suppose the economics of it are unique for Apple themselves since they can fill racks full of barebones Apple Silicon boards without having to pay their own retail markup for complete assembled systems like everyone else does.
- whimsicalism 2y ago
  without more details hard to say, but i seriously doubt they trained any significantly large LM on their own hardware people on HN routinely seem to overestimate Apple's capabilities e: in fact, iirc just last month Apple released a paper unveiling their 'OpenElm' language models and they were all trained on nvidia hardware

ra7 2y ago

Which is only a subset of requests Apple devices will serve and only with explicit user permission. That’s going to shrink over time as Apple continue to advance their own models and silicon.

stetrain 2y ago

Right, but are those going to run on Apple-owned hardware at all? It seems like Apple will first prioritize their models running on-device, then their models running on Apple Silicon servers, and then bail out to ChatGPT API calls specifically for Siri requests that they think can be better answered by ChatGPT. I'm sure OpenAI will need to beef up their hardware to handle these requests - even as filtered down as they are - coming from all of the Apple users that will now be prompting calls to ChatGPT.

whimsicalism 2y ago
they're going to be using nvidia (or maybe AMD if they ever catch up) to train these models anyways
- kolinko 2y ago
  not necessarily so, in terms of tflops per $ (of apple’s cost of gpus, nit consumer), and tflops per watt their apple silicon is comparable if not better
  - whimsicalism 2y ago
    flops/$ is simply not all (or even most) that matters when it comes to training LLMs.... Apple releases LLM research - all of their models are trained on nvidia.
  - talldayo 2y ago
    > and tflops per watt their apple silicon is comparable if not better If Apple currently ships a single product with better AI performance-per-watt than Blackwell, I will eat my hat.