4 ms·

We had a workshop 6 months ago and while I've always been sceptical of OpenAI,etc's silly AGI/ASI claims, the investments have shown the way to a lot of new tec

We had a workshop 6 months ago and while I've always been sceptical of OpenAI,etc's silly AGI/ASI claims, the investments have shown the way to a lot of new technology and has opened up a genie that won't be put back into the bottle. Now extrapolating in line with how Sun servers around year 2000 cost a fortune and can be emulated by a 5$ VPS today, Apple is seeing that they can maybe grab the local LLM workloads if they act now with their integrated chip development. But to grab that, they need developers to rely less on CUDA via Python or have other proper hardware support for those environments, and that won't happen without the hardware being there first and the machines being able to be built with enough memory (refreshing to see Apple support 128gb even if it'll probably bleed you dry).

fny 8mo ago
I feel like the push by devs towards Metal compatibility has been 10x than AMD. I assume that's because the majority of us run MacBooks.
- davidmurdoch 8mo ago
  Who is "us" in this case? Majority of devs that took the stack overflow survey use Windows: https://survey.stackoverflow.co/2025/technology/#1-computer-operating-systems https://survey.stackoverflow.co/2025/technology/#1-computer-...
  - deleted 8mo ago
    [deleted]
  - pdpi 8mo ago
    I think it's reasonable to say that the people responding to surveys on Stack Overflow aren't the same people who work on pushing the state of the art in local LLM deployment. (which doesn't prove that that crowd is Apple-centric, of course)
    - davidmurdoch 8mo ago
      Perhaps. Though Windows has been the majority share even when stack overflow was at it's peak, and before.
      - petercooper 8mo ago
        It's not the whole answer, but SO came from the .NET world and focused on it first so it had a disproportionately MS heavy audience for some time. GitHub had the same issue the other way around. Ruby was one of GitHub's top five languages for its first decade for similar reasons.
  - AdamN 8mo ago
    That's the broad developer community. 90%+ of the engineers at Big Tech and the technorati startups are on MacOS with 5% on Linux and the other 5% on Windows.
    - davidmurdoch 8mo ago
      Source?
      - saagarjha 8mo ago
        Working there?
      - AdamN 8mo ago
        Working in three countries, working in big tech and startups, talking to people.
    - re-thc 8mo ago
      > 90%+ of the engineers at Big Tech and the technorati startups The US 1s? Is that why we have Deepseek and then other non-US open source LLMs catching up rapidly? World view please. The developer community is not US only.
      - seanmcdirmid 8mo ago
        You’ll see a lot of MacBooks in Beijing’s zhongguangcun where all the tech companies are, but they also have a lot of students there as well, so who knows. You need to go out to the suburbs where Lenovo has offices to stop seeing them. I know Apple is common in Western Europe having lived there for two years (but that was 20 years ago, I lived in China for 9 years after that). It wouldn’t surprise me if the deepseek people were primarily using Mac’s. Maybe Alibaba might be using PCs? I’m not sure.
        AdamN 8mo ago
        I would also expect that the Deepseek devs are using MacBook. If not they may be using Linux - Windows is possible of course but not likely imho. I have no knowledge about that area though so would be interesting to here any primary sources or anecdotes.
  - JCharante 8mo ago
    Majority of devs are in the global south I presume
- whizzter 8mo ago
  I think that might be partly because on regular PC's you can just go and buy an NVidia card insteaf of fuzzing around with software issues, and for those on laptops they probably hope that something like Zluda will solve it via software shims or MS backed ML api's. Basically, too many choices to "focus on" makes non a winner except the incumbent.
- pjmlp 8mo ago
  Which majority? I certainly only use Macs when being project assigned, then there are plenty of developers out there whose job has nothing to do with what Apple offers. Also while Metal is a very cool API, I rather play with Vulkan, CUDA and DirectX, as do the large majority of game developers.
  - whizzter 8mo ago
    Honestly though, gamedevs really are among the biggest Windows stalwarts due to SDK's and older 3d software. Only groups of developers more tied to Windows that I can think of are probably embedded people tied due to weird hardware SDK's and Windows Active Directory dependent enterprise people. Outside of that almost everyone hip seems to want a Mac.
    - pjmlp 8mo ago
      80% of the desktop market has to have their applications developed by someone, at least until software replicators replace them. Everyone hip alright, or at least those that would dream to earn a salary big enough to afford Apple taxes. Remember there are world regions where developers barely make 1 000 euros per month.
    - fishcrackers 8mo ago
      [dead]
- well_ackshually 8mo ago
  The only "push" towards Metal compatibility there's been has been complaints on github issues. Not only has none of the work been done, absolutely nobody in their right mind wants to work on Metal compatibility. Replacing proprietary with proprietary is absolutely nobody's weekend project. or paid project.
  - hnb2137 8mo ago
    If coding by AI was truly solved then it would be done with AI, right?
freeone3000 8mo ago
Torch mlp support on my local macbook outperforms CUDA T4 on Colab.
pjmlp 8mo ago
Except CUDA feels really cozy, because like Microsoft, NVidia understands the Developers, Developers, Developers mantra. People always overlook that CUDA is a polyglot ecosystem, the IDE and graphical debugging experience where one can even single step on GPU code, the libraries ecosystem. And as of last year, NVidia has started to take Python seriously and now with cuTile based JIT, it is possible to write CUDA kernels in pure Python, not having Python generate C++ code that other tools than ingest. They are getting ahead of Modular, with Python.

fny 8mo ago

I feel like the push by devs towards Metal compatibility has been 10x than AMD. I assume that's because the majority of us run MacBooks.

davidmurdoch 8mo ago
Who is "us" in this case? Majority of devs that took the stack overflow survey use Windows: https://survey.stackoverflow.co/2025/technology/#1-computer-operating-systems https://survey.stackoverflow.co/2025/technology/#1-computer-...
- deleted 8mo ago
  [deleted]
- pdpi 8mo ago
  I think it's reasonable to say that the people responding to surveys on Stack Overflow aren't the same people who work on pushing the state of the art in local LLM deployment. (which doesn't prove that that crowd is Apple-centric, of course)
  - davidmurdoch 8mo ago
    Perhaps. Though Windows has been the majority share even when stack overflow was at it's peak, and before.
    - petercooper 8mo ago
      It's not the whole answer, but SO came from the .NET world and focused on it first so it had a disproportionately MS heavy audience for some time. GitHub had the same issue the other way around. Ruby was one of GitHub's top five languages for its first decade for similar reasons.
- AdamN 8mo ago
  That's the broad developer community. 90%+ of the engineers at Big Tech and the technorati startups are on MacOS with 5% on Linux and the other 5% on Windows.
  - davidmurdoch 8mo ago
    Source?
    - saagarjha 8mo ago
      Working there?
    - AdamN 8mo ago
      Working in three countries, working in big tech and startups, talking to people.
  - re-thc 8mo ago
    > 90%+ of the engineers at Big Tech and the technorati startups The US 1s? Is that why we have Deepseek and then other non-US open source LLMs catching up rapidly? World view please. The developer community is not US only.
    - seanmcdirmid 8mo ago
      You’ll see a lot of MacBooks in Beijing’s zhongguangcun where all the tech companies are, but they also have a lot of students there as well, so who knows. You need to go out to the suburbs where Lenovo has offices to stop seeing them. I know Apple is common in Western Europe having lived there for two years (but that was 20 years ago, I lived in China for 9 years after that). It wouldn’t surprise me if the deepseek people were primarily using Mac’s. Maybe Alibaba might be using PCs? I’m not sure.
      - AdamN 8mo ago
        I would also expect that the Deepseek devs are using MacBook. If not they may be using Linux - Windows is possible of course but not likely imho. I have no knowledge about that area though so would be interesting to here any primary sources or anecdotes.
- JCharante 8mo ago
  Majority of devs are in the global south I presume
whizzter 8mo ago
I think that might be partly because on regular PC's you can just go and buy an NVidia card insteaf of fuzzing around with software issues, and for those on laptops they probably hope that something like Zluda will solve it via software shims or MS backed ML api's. Basically, too many choices to "focus on" makes non a winner except the incumbent.
pjmlp 8mo ago
Which majority? I certainly only use Macs when being project assigned, then there are plenty of developers out there whose job has nothing to do with what Apple offers. Also while Metal is a very cool API, I rather play with Vulkan, CUDA and DirectX, as do the large majority of game developers.
- whizzter 8mo ago
  Honestly though, gamedevs really are among the biggest Windows stalwarts due to SDK's and older 3d software. Only groups of developers more tied to Windows that I can think of are probably embedded people tied due to weird hardware SDK's and Windows Active Directory dependent enterprise people. Outside of that almost everyone hip seems to want a Mac.
  - pjmlp 8mo ago
    80% of the desktop market has to have their applications developed by someone, at least until software replicators replace them. Everyone hip alright, or at least those that would dream to earn a salary big enough to afford Apple taxes. Remember there are world regions where developers barely make 1 000 euros per month.
  - fishcrackers 8mo ago
    [dead]
well_ackshually 8mo ago
The only "push" towards Metal compatibility there's been has been complaints on github issues. Not only has none of the work been done, absolutely nobody in their right mind wants to work on Metal compatibility. Replacing proprietary with proprietary is absolutely nobody's weekend project. or paid project.
- hnb2137 8mo ago
  If coding by AI was truly solved then it would be done with AI, right?

freeone3000 8mo ago

Torch mlp support on my local macbook outperforms CUDA T4 on Colab.

pjmlp 8mo ago

Except CUDA feels really cozy, because like Microsoft, NVidia understands the Developers, Developers, Developers mantra. People always overlook that CUDA is a polyglot ecosystem, the IDE and graphical debugging experience where one can even single step on GPU code, the libraries ecosystem. And as of last year, NVidia has started to take Python seriously and now with cuTile based JIT, it is possible to write CUDA kernels in pure Python, not having Python generate C++ code that other tools than ingest. They are getting ahead of Modular, with Python.