4 ms·

It's not Mach IPC, it's a bespoke thing only for the main DCP endpoint (they have a whole separate IPC framework for sub-drivers that have more sanely fully mig

It's not Mach IPC, it's a bespoke thing only for the main DCP endpoint (they have a whole separate IPC framework for sub-drivers that have more sanely fully migrated to the DCP). It also has nothing to do with Intel, since these display controllers are only in Apple SoCs. They do have a non-DCP driver they're still shipping, that runs all the same code entirely within the macOS kext. I'm not sure exactly what it's for.

derefr 5y ago
Re: this bespoke protocol, could it be something that emulates the same abstraction as Mach IPC (without being the same protocol), such that the kernel API of this protocol exposes functions similar-enough to Mach IPC functions to the driver that it would basically be a matter of an #ifdef to switch between the two? The tagged-64bit-messages thing sounds very reminiscent, is why I’m thinking along those lines. > They do have a non-DCP driver they're still shipping, that runs all the same code entirely within the macOS kext. I'm not sure exactly what it's for. Presumably, they didn’t always have the coprocessor as part of the design, especially at the prototype phase. Imagine what type of workstations were built to prototype the Apple Silicon release of macOS — they probably didn’t have any Apple-specific coprocessors at first (and then they were likely gradually added for testing by sticking them on a PCIe card or something.)
- marcan_42 5y ago
  It's much simpler than Mach IPC, and at the same time a strange beast in that they literally chopped the driver down the middle. The same objects exist on both sides, and certain methods call over to the other side. This is accomplished by having thunks on one side that marshal the arguments into a buffer and call the RPC function, and then the other side un-marshals and calls the real method on the object, then return data goes back in the opposite direction. I assume those thunks are autogenerated from some sort of IDL definition they have (they mostly use C++ templates on the argument types to perform the marshaling). As for the non-DCP driver, the odd thing is that it's specifically for the same chip generation (H13, i.e. A14 and M1), not an older one or any weird prototype thing. No weird "workstations" were built to prototype the Apple Silicon release of macOS; Apple Silicon has existed for many years now, coprocessors like these included, in iPhones and iPads. The M1 is an iPad chip, the A14X, that they beefed up just enough to be able to stick it in laptops, and then marketing rebranded it as M1. DCP specifically is relatively recent, though, I think it only showed up in a relatively recent Ax silicon generation.
  - derefr 5y ago
    > DCP specifically is relatively recent, though, I think it only showed up in a relatively recent Ax silicon generation. This was more my point. I find it unlikely that Apple were doing most of the testing of macOS-on-ARM (which probably occurred for years prior to the M1 announcement, and prior to the A14X being created) directly using the iOS device architecture. Doing that wouldn’t have allowed them to develop AS support for regular PCIe devices attached through Thunderbolt, for example, since there’s nothing like a PCIe lane in that architecture. Instead, to test things like that, I suspect Apple would have needed some kind of testbench that allowed them to run macOS on an ARM CPU, while attaching arbitrary existing peripherals into said ARM CPU’s address space, without having to fab a new one-off board for it every time they tweaked the proposed architecture. I would guess that their approach, then, would have been very similar to the approach used in prototype bring-up in the game-console industry: - Use a regular Intel machine as a host / hypervisor (in Apple’s case, probably one of the internal mATX-testbench-layout Intel Mac Pros) - Put whatever recent-generation ARM CPU they made, onto a PCIe card - Build a bespoke hypervisor to drive that CPU, which presents to the CPU a virtualized chipset matching the current proposed architecture (e.g. a DCP or not, a Thunderbolt controller, etc.) - Have the hypervisor configure the host’s IOMMU to present both virtual peripherals (for bring-up), and arbitrary host peripherals, to the CPU It’s not like Apple are unfamiliar with “SoCs used to accelerate a host-run emulator”; decades ago, they put an Apple II on an accelerator card and drove it through a host hypervisor just like this :)
    - marcan_42 5y ago
      > I find it unlikely that Apple were doing most of the testing of macOS-on-ARM (which probably occurred for years prior to the M1 announcement, and prior to the A14X being created) directly using the iOS device architecture. Of course they would be doing it like that, since the macOS kernel was already ported to ARM for iOS. They even handed out developer kits that literally used an iPad chip. https://en.wikipedia.org/wiki/Developer_Transition_Kit_(2020) https://en.wikipedia.org/wiki/Developer_Transition_Kit_(2020... macOS on ARM is very clearly a descendant of the way iOS works. It's just macOS userspace on top of an ARM XNU kernel, which was already a thing. The way the boot process works, etc. is clearly iOS plus their new fancy Boot Policy stuff for supporting multiple OSes and custom kernels. > Doing that wouldn’t have allowed them to develop AS support for regular PCIe devices attached through Thunderbolt, for example, since there’s nothing like a PCIe lane in that architecture. iPhones and iPads use PCIe! How do you think WiFi/storage are attached? PCIe isn't only a desktop thing, it's in most phones these days, the Nintendo Switch, Raspberry Pi 4, etc. Most modern ARM embedded systems use PCIe for something. You'd be hard pressed to find a high-end ARM SoC without PCIe lanes. They didn't have Thunderbolt, but since Apple rolled their own bespoke controller too, there is absolutely nothing to be gained by bringing that up on Intel first. Instead they probably did the same thing everyone does: bring it up on FPGAs. Possibly as PCIe add-ons for existing iPad chips, possibly hosting a whole (reduced) SoC; both approaches would probably be mixed at different design stages. I'm sure they also had at least one unreleased silicon spin/design before the M1, during this project. You never quite get these things right the first time. No idea if that would've made it close to tape-out or not, but I'm sure there was something quite a bit different from the M1 in design at some point. > Use a regular Intel machine as a host > Put whatever recent-generation ARM CPU they made, onto a PCIe card You can buy one of those, it's called a T2 Mac. Indeed some Apple Silicon technologies came from there (e.g. how SEP external storage is managed), but not in the way you describe; most of the T2<->Intel interface was thrown away for Apple Silicon, and now the OS sees things the same way as BridgeOS did natively on the T2 on those Macs.

derefr 5y ago

Re: this bespoke protocol, could it be something that emulates the same abstraction as Mach IPC (without being the same protocol), such that the kernel API of this protocol exposes functions similar-enough to Mach IPC functions to the driver that it would basically be a matter of an #ifdef to switch between the two? The tagged-64bit-messages thing sounds very reminiscent, is why I’m thinking along those lines. > They do have a non-DCP driver they're still shipping, that runs all the same code entirely within the macOS kext. I'm not sure exactly what it's for. Presumably, they didn’t always have the coprocessor as part of the design, especially at the prototype phase. Imagine what type of workstations were built to prototype the Apple Silicon release of macOS — they probably didn’t have any Apple-specific coprocessors at first (and then they were likely gradually added for testing by sticking them on a PCIe card or something.)

marcan_42 5y ago
It's much simpler than Mach IPC, and at the same time a strange beast in that they literally chopped the driver down the middle. The same objects exist on both sides, and certain methods call over to the other side. This is accomplished by having thunks on one side that marshal the arguments into a buffer and call the RPC function, and then the other side un-marshals and calls the real method on the object, then return data goes back in the opposite direction. I assume those thunks are autogenerated from some sort of IDL definition they have (they mostly use C++ templates on the argument types to perform the marshaling). As for the non-DCP driver, the odd thing is that it's specifically for the same chip generation (H13, i.e. A14 and M1), not an older one or any weird prototype thing. No weird "workstations" were built to prototype the Apple Silicon release of macOS; Apple Silicon has existed for many years now, coprocessors like these included, in iPhones and iPads. The M1 is an iPad chip, the A14X, that they beefed up just enough to be able to stick it in laptops, and then marketing rebranded it as M1. DCP specifically is relatively recent, though, I think it only showed up in a relatively recent Ax silicon generation.
- derefr 5y ago
  > DCP specifically is relatively recent, though, I think it only showed up in a relatively recent Ax silicon generation. This was more my point. I find it unlikely that Apple were doing most of the testing of macOS-on-ARM (which probably occurred for years prior to the M1 announcement, and prior to the A14X being created) directly using the iOS device architecture. Doing that wouldn’t have allowed them to develop AS support for regular PCIe devices attached through Thunderbolt, for example, since there’s nothing like a PCIe lane in that architecture. Instead, to test things like that, I suspect Apple would have needed some kind of testbench that allowed them to run macOS on an ARM CPU, while attaching arbitrary existing peripherals into said ARM CPU’s address space, without having to fab a new one-off board for it every time they tweaked the proposed architecture. I would guess that their approach, then, would have been very similar to the approach used in prototype bring-up in the game-console industry: - Use a regular Intel machine as a host / hypervisor (in Apple’s case, probably one of the internal mATX-testbench-layout Intel Mac Pros) - Put whatever recent-generation ARM CPU they made, onto a PCIe card - Build a bespoke hypervisor to drive that CPU, which presents to the CPU a virtualized chipset matching the current proposed architecture (e.g. a DCP or not, a Thunderbolt controller, etc.) - Have the hypervisor configure the host’s IOMMU to present both virtual peripherals (for bring-up), and arbitrary host peripherals, to the CPU It’s not like Apple are unfamiliar with “SoCs used to accelerate a host-run emulator”; decades ago, they put an Apple II on an accelerator card and drove it through a host hypervisor just like this :)
  - marcan_42 5y ago
    > I find it unlikely that Apple were doing most of the testing of macOS-on-ARM (which probably occurred for years prior to the M1 announcement, and prior to the A14X being created) directly using the iOS device architecture. Of course they would be doing it like that, since the macOS kernel was already ported to ARM for iOS. They even handed out developer kits that literally used an iPad chip. https://en.wikipedia.org/wiki/Developer_Transition_Kit_(2020) https://en.wikipedia.org/wiki/Developer_Transition_Kit_(2020... macOS on ARM is very clearly a descendant of the way iOS works. It's just macOS userspace on top of an ARM XNU kernel, which was already a thing. The way the boot process works, etc. is clearly iOS plus their new fancy Boot Policy stuff for supporting multiple OSes and custom kernels. > Doing that wouldn’t have allowed them to develop AS support for regular PCIe devices attached through Thunderbolt, for example, since there’s nothing like a PCIe lane in that architecture. iPhones and iPads use PCIe! How do you think WiFi/storage are attached? PCIe isn't only a desktop thing, it's in most phones these days, the Nintendo Switch, Raspberry Pi 4, etc. Most modern ARM embedded systems use PCIe for something. You'd be hard pressed to find a high-end ARM SoC without PCIe lanes. They didn't have Thunderbolt, but since Apple rolled their own bespoke controller too, there is absolutely nothing to be gained by bringing that up on Intel first. Instead they probably did the same thing everyone does: bring it up on FPGAs. Possibly as PCIe add-ons for existing iPad chips, possibly hosting a whole (reduced) SoC; both approaches would probably be mixed at different design stages. I'm sure they also had at least one unreleased silicon spin/design before the M1, during this project. You never quite get these things right the first time. No idea if that would've made it close to tape-out or not, but I'm sure there was something quite a bit different from the M1 in design at some point. > Use a regular Intel machine as a host > Put whatever recent-generation ARM CPU they made, onto a PCIe card You can buy one of those, it's called a T2 Mac. Indeed some Apple Silicon technologies came from there (e.g. how SEP external storage is managed), but not in the way you describe; most of the T2<->Intel interface was thrown away for Apple Silicon, and now the OS sees things the same way as BridgeOS did natively on the T2 on those Macs.