11 ms·

Dissecting the Apple M1 GPU, Part IV

379 points by caution 5y ago

loop0 5y ago
That’s an impressive amount of progress already. Congrats on the work. I’m waiting apple’s next silicon to jump to arm, and hopefully by then I’m going to be able to run linux on it.
- tambourine_man 5y ago
  Natively, it's gonna take a long time, I'd wager. Virtualization works pretty well today, however.
  - floatboth 5y ago
    Pretty much all the things except the GPU are working in Corellium's quick'n'dirty "demo" Linux port. And you see how amazingly quickly the GPU driver is coming along… So not that long of a time after all (I'm surprised too).
    - tambourine_man 5y ago
      I'm positively surprised as well. But working well for day to day use… I don't know. But it's not my field, just more or less educated guess.
Engineering-MD 5y ago
So, given the current trend in progress, when will we have truly usable linux on M1?
- mekster 5y ago
  Would server vendors start putting out M1 in data centers once Linux truly runs on it? Doesn't opening up driver specification also let M1 start ruling in the server spaces?
  - blinkingled 5y ago
    Servers/Datacenters aren't just about being able to run Linux though. How are they going to do firmware updates? Apple can break stuff as there is no contract there like there is with x86 server vendors who test all their updates to make sure Windows and Linux runs. Besides there's the whole issue of configuration/systems management, remote access and bulk chip availability.
    - ultrarunner 5y ago
      I would love to see the shakeup with more efficient chips pressuring Intel to innovate in the datacenter space. Being able to decrease our hosting bill would reflect well on me, and it just seems like the way the space will eventually progress. However, I run a hackintosh at home. I personally love it; I had access to top of the line hardware years before Apple offered it, with an OS that I understand and work well in, etc. It is absolutely not the way to go for any kind of must-work production. It is brittle. Apple is in the enviable position of being the only consumer of its APIs, and they barely publish documentation on the public stuff, much less internal-only provisions. Unless Apple explicitly supports (read: $$$$) a datacenter application, I can only see burning buildings (from all the hair on fire) where stable datacenters should be.
      - blinkingled 5y ago
        Yeah Intel and AMD both need competition from power efficient chips that perform equally well and offer the same level of support for Windows/Linux. And as you said, it's not going to be Apple - they have a consumer focus and culture and it's far fetched to imagine they will do all the stuff necessary for enterprise adoption of their ARM chips. In the cloud space Amazon has Graviton but there's not much hype about its performance and you run into compat issues a lot for normal cloud workloads at least in the Enterprise. Nvidia has a real shot here with ARM purchase to fill the gaps but I doubt that's something they have as a focus.
    - deleted 5y ago
      [deleted]
  - 5y ago
Jasper_ 5y ago
> Well-written apps generally require primitive restart, and it’s almost free for apps that don’t need it. I'm really surprised that primitive restart is forced on in Metal, but they also have a hardware bit for it! Primitive restart makes your whole pipeline slower, because you can't just chunk your index list when building your vertex packets to send to the work distributors; the possibility of a primitive restart index means you have to linearly scan for it. Though I'm not a hardware guy -- maybe the degenerate tris thing makes it just as annoying in practice. I'd guess that the limitation to force primitive restart on in Metal came from another IHV limitation (maybe old PVR chips?) That said, you really shouldn't be using tristrips in 2021 anyway, they have poor locality, and are hard to optimize. Index buffers solve the problems that tristrips wanted to solve to begin with. IME, a well-written app should just ignore tristrips / restartable topologies, and just use straight tri-lists.
- atq2119 5y ago
  In theory you could do parallel scan tricks to implement primitive restart at full speed, but obviously it's going to cost you and it complicates the index fetch immensely since it effectively ends up being split into two parts. Most likely, Apple simply doesn't build GPUs big enough to run into that problem, but it makes you wonder what they're doing on their hardware with AMD GPUs.
varispeed 5y ago
Hopefully one day companies like Apple will be forced to disclose documentation to the devices they sell so that consumers can make the full use of them and won't have to waste time on reverse engineering.
- enos_feedler 5y ago
  that would put a lot of obligation on the vendor to honor some kind of contract they don't need to do today. They can change anything they want with zero cost today.
  - ddingus 5y ago
    Zero cost to them perhaps. However, those costs often prove quite high for others. Right now increasing numbers of things are, "works until it doesn't" and that imposes risks and costs to everyone. Perhaps those get high enough to warrant legislative solutions.
  - enos_feedler 5y ago
    hah, bring up a good point but get downvoted because it doesn't jive.
  - gpm 5y ago
    They could change what they want afterwards too presumably, they'd just need to publish the new documentation along side it.
    - ece 5y ago
      This is what Intel/AMD have done for multiple generations of their GPU hardware now for a decade or more. Hasn't stopped them from innovating. Nvidia has PTX with a proprietary steam assembler underneath which can retarget any hardware changes. Either way, Apple can offer more documentation than it currently does.
  - NoSorryCannot 5y ago
    Does documentation necessarily create such an obligation? That would jeopardize most open-source projects, most of which explicitly promise no warranty.
eointierney 5y ago
So well written, so engaging, such love in the labour. "We’re looking for conspicuous gaps in our understanding of the hardware, like looking for black holes by observing the absence of light." Gorgeous
- simondotau 5y ago
  Seconded. Despite being a genuinely challenging and complicated process with countless moving parts, it was written to be as approachable as possible. They didn't feel a need to "prove" how hard the work is by writing from deep within the weeds.
- GeekyBear 5y ago
  I have also enjoyed reading these progress reports.
deleted 5y ago
[deleted]
dapids 5y ago
Word of warning, stay away from the metal shader bitcode (AIR), or they are gonna sue ya, they did a couple academics for publishing on it.
- Jasper_ 5y ago
  That whole stack is conveniently avoided in this research, because they're going from NIR straight to the GPU machine code, without touching AIR.
  - dapids 5y ago
    Ah, very clever, will have see what the future brings. I would assume apple would not be too pleased though, with either approach.
    - notaplumber 5y ago
      Why would Apple go through the effort of adding support for booting alternative OS's on M1 Macs and then stand in the way of implementing drivers for the hardware? That doesn't make a lot of sense, IMHO. Especially considering Apple could have gone the exact same path as iOS devices, which is a heavily locked-down platform that shares a lot in common.
      - dapids 5y ago
        What? Who? Apple? Please point me towards apple directly upstreaming code to Linux to support M1. Your comment makes no sense, IMHO.
        coldtea 5y ago
        What are you talking about? Nobody here said about Apple "directly upstreaming code to Linux to support M1". Parent said "adding support for booting alternative OS's on M1 Macs", which Apple did do.
theunamedguy 5y ago
A big shoutout to Alyssa here. For those of you who don't know, she is doing this work as a college student.
- atq2119 5y ago
  College students are just as smart as graduates and they tend to have much more time on their hands. So yeah, this is cool work, kudos and everything, but the fact that a college student is doing it is the least surprising part of it if you really think about it.
- viktorcode 5y ago
  She's brilliant. Here's hoping she'll leave her mark on software industry.
KirillPanov 5y ago
Alyssa is a hero for her work on Panfrost, which gave us open-source 3D graphics on ARM Mali GPUs. I am eternally grateful for this; the Samsung Kevin I use is the only blobless laptop currently in production, and now it has 3D accelerated graphics thanks to her work! But part of me is sad to see her working on such closed hardware now. Does anybody think Linux on the M1 will ever be able to touch the internal SSD? Apple has been locking that down with proprietary controllers and signed firmware since their Intel days (i.e. T1 chip). Are people really going to drag around their shiny new macbooks with an external USB-C dongle hanging off of it because that's required in order to run Linux? I worry that the endgame here is Linux becoming Just Another MacOS App. Apple is quite happy for Linux to be a MacOS app running in their VM. Elite developers will buy Macbooks and run Linux in a VM because "we'll have Linux on the bare metal soon, it's just temporary" and that will just keep getting pushed back and pushed back and pushed back...
- 58028641 5y ago
  Hector Martin believes it will be possible. https://twitter.com/marcan42/status/1383964656058781703 https://twitter.com/marcan42/status/1383964656058781703
  - marcan_42 5y ago
    I don't "believe" it will be possible; I am absolutely, 100% positively certain it is possible.
- mjg59 5y ago
  Apple don't block access to their NVMe controllers at all. They do appear to have a, well, interesting approach to spec compliance, but Linux is now entirely capable of handling the SSD on all x86 Apple hardware. In the M1 case the NVMe controller isn't exposed via PCI so the existing driver won't work, but there's already in-kernel abstraction between the actual NVMe code and the PCI interface, so adding an alternative shouldn't be a problem. The M1 systems depend on some number of blobs, but the amount of non-free code required to boot one looks like it'll end up being less than a typical x86 system requires.
marsven_422 5y ago
*He
deleted 5y ago
[deleted]
_ph_ 5y ago
Great work and even more surprising: a great writeup (writing that as someone who always struggles to document my own work). This really gives me hope to have an usable, native Linux on the M1 machines. I really wished Apple would see, how much benefit this would bring to their platform. Their new hardware is really exciting, the first ARM on the desktop, which doesn't just compete, but in many aspects beat current x86 chips. A lot of the Linux and tech enthusiast crowd would love to jump onto Apple Silicon. And while they might not bring huge profits by themselves, these are the people who do come up with great new technologies. Better give them a home on Apple devices. It doesn't need a complete and formal documentation of every aspect, just supporting those projects with a little bit of information would go a long way. A single engineer who would answer questions by those developers might be sufficient. So come on, Apple, do it! :)
- dannyw 5y ago
  This is a company that actively fights right to repair and implements software DRM to lock out non-Apple authorised replacements. Don't hold your breath. Apple's stance by actions is the opposite.
  - _ph_ 5y ago
    I am not holding my breath, just stressing how nice it would be and how beneficial it could be for Apple themselves.
  - w0utert 5y ago
    >> This is a company that actively fights right to repair and implements software DRM to lock out non-Apple authorised replacements. But they do all these things for obvious reasons. Reasons you and I may not agree with or be happy about, but still obvious reasons. In the case of repair/replacement it's just because they want you to use expensive replacement parts, they want to lure you into their Apple stores, and they don't want any liability/accountability for repairs with 'unofficial' parts. I don't see how providing specifications about how their GPU's work so someone can make a Linux driver out of it hurts their commercial interests or liability though. Yes people may screw up their system if they install Linux on a Mac and it doesn't boot anymore, but as long as you can still take it into an Apple store and they can restore it to MacOS, why would Apple actively fight the extremely small minority of people who want to do that? And even if more people (developers/enthusiasts) would buy M1 hardware and immediately slap Linux on it, why would they care about that? They still made the sale, and these people will still walk around with a machine with a big fat Apple logo on it? They previously spent a lot of effort accomodating people who wanted to run Windows on Macs using Boot Camp, so why would they be worried about people running Linux on M1 macs? Edit: I can imagine Apple want to protect their IP and hence don't want to disclose anything about out it, period. Much like NVidia and most other GPU manufacturers do. But if AMD and Intel can be OSS-friendly, Apple could be too, apparently IP protection does not have to be a deal-breaker.

379 points by caution 5y ago

That’s an impressive amount of progress already. Congrats on the work. I’m waiting apple’s next silicon to jump to arm, and hopefully by then I’m going to be able to run linux on it.

tambourine_man 5y ago
Natively, it's gonna take a long time, I'd wager. Virtualization works pretty well today, however.
- floatboth 5y ago
  Pretty much all the things except the GPU are working in Corellium's quick'n'dirty "demo" Linux port. And you see how amazingly quickly the GPU driver is coming along… So not that long of a time after all (I'm surprised too).
  - tambourine_man 5y ago
    I'm positively surprised as well. But working well for day to day use… I don't know. But it's not my field, just more or less educated guess.

Engineering-MD 5y ago

So, given the current trend in progress, when will we have truly usable linux on M1?

mekster 5y ago
Would server vendors start putting out M1 in data centers once Linux truly runs on it? Doesn't opening up driver specification also let M1 start ruling in the server spaces?
- blinkingled 5y ago
  Servers/Datacenters aren't just about being able to run Linux though. How are they going to do firmware updates? Apple can break stuff as there is no contract there like there is with x86 server vendors who test all their updates to make sure Windows and Linux runs. Besides there's the whole issue of configuration/systems management, remote access and bulk chip availability.
  - ultrarunner 5y ago
    I would love to see the shakeup with more efficient chips pressuring Intel to innovate in the datacenter space. Being able to decrease our hosting bill would reflect well on me, and it just seems like the way the space will eventually progress. However, I run a hackintosh at home. I personally love it; I had access to top of the line hardware years before Apple offered it, with an OS that I understand and work well in, etc. It is absolutely not the way to go for any kind of must-work production. It is brittle. Apple is in the enviable position of being the only consumer of its APIs, and they barely publish documentation on the public stuff, much less internal-only provisions. Unless Apple explicitly supports (read: $$$$) a datacenter application, I can only see burning buildings (from all the hair on fire) where stable datacenters should be.
    - blinkingled 5y ago
      Yeah Intel and AMD both need competition from power efficient chips that perform equally well and offer the same level of support for Windows/Linux. And as you said, it's not going to be Apple - they have a consumer focus and culture and it's far fetched to imagine they will do all the stuff necessary for enterprise adoption of their ARM chips. In the cloud space Amazon has Graviton but there's not much hype about its performance and you run into compat issues a lot for normal cloud workloads at least in the Enterprise. Nvidia has a real shot here with ARM purchase to fill the gaps but I doubt that's something they have as a focus.
  - deleted 5y ago
    [deleted]
- 5y ago

Jasper_ 5y ago

> Well-written apps generally require primitive restart, and it’s almost free for apps that don’t need it. I'm really surprised that primitive restart is forced on in Metal, but they also have a hardware bit for it! Primitive restart makes your whole pipeline slower, because you can't just chunk your index list when building your vertex packets to send to the work distributors; the possibility of a primitive restart index means you have to linearly scan for it. Though I'm not a hardware guy -- maybe the degenerate tris thing makes it just as annoying in practice. I'd guess that the limitation to force primitive restart on in Metal came from another IHV limitation (maybe old PVR chips?) That said, you really shouldn't be using tristrips in 2021 anyway, they have poor locality, and are hard to optimize. Index buffers solve the problems that tristrips wanted to solve to begin with. IME, a well-written app should just ignore tristrips / restartable topologies, and just use straight tri-lists.

atq2119 5y ago
In theory you could do parallel scan tricks to implement primitive restart at full speed, but obviously it's going to cost you and it complicates the index fetch immensely since it effectively ends up being split into two parts. Most likely, Apple simply doesn't build GPUs big enough to run into that problem, but it makes you wonder what they're doing on their hardware with AMD GPUs.

varispeed 5y ago

Hopefully one day companies like Apple will be forced to disclose documentation to the devices they sell so that consumers can make the full use of them and won't have to waste time on reverse engineering.

enos_feedler 5y ago
that would put a lot of obligation on the vendor to honor some kind of contract they don't need to do today. They can change anything they want with zero cost today.
- ddingus 5y ago
  Zero cost to them perhaps. However, those costs often prove quite high for others. Right now increasing numbers of things are, "works until it doesn't" and that imposes risks and costs to everyone. Perhaps those get high enough to warrant legislative solutions.
- enos_feedler 5y ago
  hah, bring up a good point but get downvoted because it doesn't jive.
- gpm 5y ago
  They could change what they want afterwards too presumably, they'd just need to publish the new documentation along side it.
  - ece 5y ago
    This is what Intel/AMD have done for multiple generations of their GPU hardware now for a decade or more. Hasn't stopped them from innovating. Nvidia has PTX with a proprietary steam assembler underneath which can retarget any hardware changes. Either way, Apple can offer more documentation than it currently does.
- NoSorryCannot 5y ago
  Does documentation necessarily create such an obligation? That would jeopardize most open-source projects, most of which explicitly promise no warranty.

eointierney 5y ago

So well written, so engaging, such love in the labour. "We’re looking for conspicuous gaps in our understanding of the hardware, like looking for black holes by observing the absence of light." Gorgeous

simondotau 5y ago
Seconded. Despite being a genuinely challenging and complicated process with countless moving parts, it was written to be as approachable as possible. They didn't feel a need to "prove" how hard the work is by writing from deep within the weeds.
GeekyBear 5y ago
I have also enjoyed reading these progress reports.

deleted 5y ago

[deleted]

dapids 5y ago

Word of warning, stay away from the metal shader bitcode (AIR), or they are gonna sue ya, they did a couple academics for publishing on it.

Jasper_ 5y ago
That whole stack is conveniently avoided in this research, because they're going from NIR straight to the GPU machine code, without touching AIR.
- dapids 5y ago
  Ah, very clever, will have see what the future brings. I would assume apple would not be too pleased though, with either approach.
  - notaplumber 5y ago
    Why would Apple go through the effort of adding support for booting alternative OS's on M1 Macs and then stand in the way of implementing drivers for the hardware? That doesn't make a lot of sense, IMHO. Especially considering Apple could have gone the exact same path as iOS devices, which is a heavily locked-down platform that shares a lot in common.
    - dapids 5y ago
      What? Who? Apple? Please point me towards apple directly upstreaming code to Linux to support M1. Your comment makes no sense, IMHO.
      - coldtea 5y ago
        What are you talking about? Nobody here said about Apple "directly upstreaming code to Linux to support M1". Parent said "adding support for booting alternative OS's on M1 Macs", which Apple did do.

theunamedguy 5y ago

A big shoutout to Alyssa here. For those of you who don't know, she is doing this work as a college student.

atq2119 5y ago
College students are just as smart as graduates and they tend to have much more time on their hands. So yeah, this is cool work, kudos and everything, but the fact that a college student is doing it is the least surprising part of it if you really think about it.
viktorcode 5y ago
She's brilliant. Here's hoping she'll leave her mark on software industry.

KirillPanov 5y ago

Alyssa is a hero for her work on Panfrost, which gave us open-source 3D graphics on ARM Mali GPUs. I am eternally grateful for this; the Samsung Kevin I use is the only blobless laptop currently in production, and now it has 3D accelerated graphics thanks to her work! But part of me is sad to see her working on such closed hardware now. Does anybody think Linux on the M1 will ever be able to touch the internal SSD? Apple has been locking that down with proprietary controllers and signed firmware since their Intel days (i.e. T1 chip). Are people really going to drag around their shiny new macbooks with an external USB-C dongle hanging off of it because that's required in order to run Linux? I worry that the endgame here is Linux becoming Just Another MacOS App. Apple is quite happy for Linux to be a MacOS app running in their VM. Elite developers will buy Macbooks and run Linux in a VM because "we'll have Linux on the bare metal soon, it's just temporary" and that will just keep getting pushed back and pushed back and pushed back...

58028641 5y ago
Hector Martin believes it will be possible. https://twitter.com/marcan42/status/1383964656058781703 https://twitter.com/marcan42/status/1383964656058781703
- marcan_42 5y ago
  I don't "believe" it will be possible; I am absolutely, 100% positively certain it is possible.
mjg59 5y ago
Apple don't block access to their NVMe controllers at all. They do appear to have a, well, interesting approach to spec compliance, but Linux is now entirely capable of handling the SSD on all x86 Apple hardware. In the M1 case the NVMe controller isn't exposed via PCI so the existing driver won't work, but there's already in-kernel abstraction between the actual NVMe code and the PCI interface, so adding an alternative shouldn't be a problem. The M1 systems depend on some number of blobs, but the amount of non-free code required to boot one looks like it'll end up being less than a typical x86 system requires.

marsven_422 5y ago

*He

deleted 5y ago

[deleted]

_ph_ 5y ago

Great work and even more surprising: a great writeup (writing that as someone who always struggles to document my own work). This really gives me hope to have an usable, native Linux on the M1 machines. I really wished Apple would see, how much benefit this would bring to their platform. Their new hardware is really exciting, the first ARM on the desktop, which doesn't just compete, but in many aspects beat current x86 chips. A lot of the Linux and tech enthusiast crowd would love to jump onto Apple Silicon. And while they might not bring huge profits by themselves, these are the people who do come up with great new technologies. Better give them a home on Apple devices. It doesn't need a complete and formal documentation of every aspect, just supporting those projects with a little bit of information would go a long way. A single engineer who would answer questions by those developers might be sufficient. So come on, Apple, do it! :)

dannyw 5y ago
This is a company that actively fights right to repair and implements software DRM to lock out non-Apple authorised replacements. Don't hold your breath. Apple's stance by actions is the opposite.
- _ph_ 5y ago
  I am not holding my breath, just stressing how nice it would be and how beneficial it could be for Apple themselves.
- w0utert 5y ago
  >> This is a company that actively fights right to repair and implements software DRM to lock out non-Apple authorised replacements. But they do all these things for obvious reasons. Reasons you and I may not agree with or be happy about, but still obvious reasons. In the case of repair/replacement it's just because they want you to use expensive replacement parts, they want to lure you into their Apple stores, and they don't want any liability/accountability for repairs with 'unofficial' parts. I don't see how providing specifications about how their GPU's work so someone can make a Linux driver out of it hurts their commercial interests or liability though. Yes people may screw up their system if they install Linux on a Mac and it doesn't boot anymore, but as long as you can still take it into an Apple store and they can restore it to MacOS, why would Apple actively fight the extremely small minority of people who want to do that? And even if more people (developers/enthusiasts) would buy M1 hardware and immediately slap Linux on it, why would they care about that? They still made the sale, and these people will still walk around with a machine with a big fat Apple logo on it? They previously spent a lot of effort accomodating people who wanted to run Windows on Macs using Boot Camp, so why would they be worried about people running Linux on M1 macs? Edit: I can imagine Apple want to protect their IP and hence don't want to disclose anything about out it, period. Much like NVidia and most other GPU manufacturers do. But if AMD and Intel can be OSS-friendly, Apple could be too, apparently IP protection does not have to be a deal-breaker.