8 ms·

I’m guessing that what Apple did here was: 1. Probably years ago: refactor the relevant components (display driver, etc.) to run as Mach components with Mach I

by derefr 5y ago

I’m guessing that what Apple did here was: 1. Probably years ago: refactor the relevant components (display driver, etc.) to run as Mach components with Mach IPC, in the form of: - One XCode project, with - an “IPC client” build target - an “IPC server” build target - a “shared datatypes library” with well-defined IPC serialization semantics, static-linked into both the IPC client and server - a single test suite that tests everything 2. Possibly when designing the M1, or possibly years ago for iOS: split the IPC server build target in two — one build target that builds an IPC-server firmware blob (RTOS unikernel), and another that builds a regular on-CPU kernel-daemon IPC server; 3. Basically do nothing in the IPC client build target — it should now be oblivious to whether its Mach messages are going to an on-CPU kernel daemon or to an RTOS-coprocessor-hosted daemon, as the Mach abstraction layer is taking care of the message routing. (Kinda like the location-obliviousness you get in Erlang when sending messages to PIDs.) This seems likely to me because there was (and still is!) both an Intel and an Apple Silicon macOS release; and they would want to share as much driver code between those releases as possible. So I think it’s very likely that they’ve written drivers structured to be “split across” Apple Silicon, while running “locally” on their Intel devices, in such a way that the differences between these approaches is effectively transparent to the kernel. To achieve #3 — and especially to achieve it while having only one shared test suite — the firmware would have to be speaking the same wire protocol to the IPC client that the in-kernel IPC daemon speaks. And, given that the in-kernel IPC daemons were designed to presume a reliable C++ interface, access to shared-memory copies of low-level Frameworks like IOKit, etc.; the firmware would need to provide this same environment to get the pre-existing code to run.

rjzzleep 5y ago
That's the kind of theory I was looking for, thanks!
marcan_42 5y ago
It's not Mach IPC, it's a bespoke thing only for the main DCP endpoint (they have a whole separate IPC framework for sub-drivers that have more sanely fully migrated to the DCP). It also has nothing to do with Intel, since these display controllers are only in Apple SoCs. They do have a non-DCP driver they're still shipping, that runs all the same code entirely within the macOS kext. I'm not sure exactly what it's for.
- derefr 5y ago
  Re: this bespoke protocol, could it be something that emulates the same abstraction as Mach IPC (without being the same protocol), such that the kernel API of this protocol exposes functions similar-enough to Mach IPC functions to the driver that it would basically be a matter of an #ifdef to switch between the two? The tagged-64bit-messages thing sounds very reminiscent, is why I’m thinking along those lines. > They do have a non-DCP driver they're still shipping, that runs all the same code entirely within the macOS kext. I'm not sure exactly what it's for. Presumably, they didn’t always have the coprocessor as part of the design, especially at the prototype phase. Imagine what type of workstations were built to prototype the Apple Silicon release of macOS — they probably didn’t have any Apple-specific coprocessors at first (and then they were likely gradually added for testing by sticking them on a PCIe card or something.)
  - marcan_42 5y ago
    It's much simpler than Mach IPC, and at the same time a strange beast in that they literally chopped the driver down the middle. The same objects exist on both sides, and certain methods call over to the other side. This is accomplished by having thunks on one side that marshal the arguments into a buffer and call the RPC function, and then the other side un-marshals and calls the real method on the object, then return data goes back in the opposite direction. I assume those thunks are autogenerated from some sort of IDL definition they have (they mostly use C++ templates on the argument types to perform the marshaling). As for the non-DCP driver, the odd thing is that it's specifically for the same chip generation (H13, i.e. A14 and M1), not an older one or any weird prototype thing. No weird "workstations" were built to prototype the Apple Silicon release of macOS; Apple Silicon has existed for many years now, coprocessors like these included, in iPhones and iPads. The M1 is an iPad chip, the A14X, that they beefed up just enough to be able to stick it in laptops, and then marketing rebranded it as M1. DCP specifically is relatively recent, though, I think it only showed up in a relatively recent Ax silicon generation.
    - derefr 5y ago
      > DCP specifically is relatively recent, though, I think it only showed up in a relatively recent Ax silicon generation. This was more my point. I find it unlikely that Apple were doing most of the testing of macOS-on-ARM (which probably occurred for years prior to the M1 announcement, and prior to the A14X being created) directly using the iOS device architecture. Doing that wouldn’t have allowed them to develop AS support for regular PCIe devices attached through Thunderbolt, for example, since there’s nothing like a PCIe lane in that architecture. Instead, to test things like that, I suspect Apple would have needed some kind of testbench that allowed them to run macOS on an ARM CPU, while attaching arbitrary existing peripherals into said ARM CPU’s address space, without having to fab a new one-off board for it every time they tweaked the proposed architecture. I would guess that their approach, then, would have been very similar to the approach used in prototype bring-up in the game-console industry: - Use a regular Intel machine as a host / hypervisor (in Apple’s case, probably one of the internal mATX-testbench-layout Intel Mac Pros) - Put whatever recent-generation ARM CPU they made, onto a PCIe card - Build a bespoke hypervisor to drive that CPU, which presents to the CPU a virtualized chipset matching the current proposed architecture (e.g. a DCP or not, a Thunderbolt controller, etc.) - Have the hypervisor configure the host’s IOMMU to present both virtual peripherals (for bring-up), and arbitrary host peripherals, to the CPU It’s not like Apple are unfamiliar with “SoCs used to accelerate a host-run emulator”; decades ago, they put an Apple II on an accelerator card and drove it through a host hypervisor just like this :)

by derefr 5y ago

rjzzleep 5y ago

That's the kind of theory I was looking for, thanks!

marcan_42 5y ago

It's not Mach IPC, it's a bespoke thing only for the main DCP endpoint (they have a whole separate IPC framework for sub-drivers that have more sanely fully migrated to the DCP). It also has nothing to do with Intel, since these display controllers are only in Apple SoCs. They do have a non-DCP driver they're still shipping, that runs all the same code entirely within the macOS kext. I'm not sure exactly what it's for.

derefr 5y ago
Re: this bespoke protocol, could it be something that emulates the same abstraction as Mach IPC (without being the same protocol), such that the kernel API of this protocol exposes functions similar-enough to Mach IPC functions to the driver that it would basically be a matter of an #ifdef to switch between the two? The tagged-64bit-messages thing sounds very reminiscent, is why I’m thinking along those lines. > They do have a non-DCP driver they're still shipping, that runs all the same code entirely within the macOS kext. I'm not sure exactly what it's for. Presumably, they didn’t always have the coprocessor as part of the design, especially at the prototype phase. Imagine what type of workstations were built to prototype the Apple Silicon release of macOS — they probably didn’t have any Apple-specific coprocessors at first (and then they were likely gradually added for testing by sticking them on a PCIe card or something.)
- marcan_42 5y ago
  It's much simpler than Mach IPC, and at the same time a strange beast in that they literally chopped the driver down the middle. The same objects exist on both sides, and certain methods call over to the other side. This is accomplished by having thunks on one side that marshal the arguments into a buffer and call the RPC function, and then the other side un-marshals and calls the real method on the object, then return data goes back in the opposite direction. I assume those thunks are autogenerated from some sort of IDL definition they have (they mostly use C++ templates on the argument types to perform the marshaling). As for the non-DCP driver, the odd thing is that it's specifically for the same chip generation (H13, i.e. A14 and M1), not an older one or any weird prototype thing. No weird "workstations" were built to prototype the Apple Silicon release of macOS; Apple Silicon has existed for many years now, coprocessors like these included, in iPhones and iPads. The M1 is an iPad chip, the A14X, that they beefed up just enough to be able to stick it in laptops, and then marketing rebranded it as M1. DCP specifically is relatively recent, though, I think it only showed up in a relatively recent Ax silicon generation.
  - derefr 5y ago
    > DCP specifically is relatively recent, though, I think it only showed up in a relatively recent Ax silicon generation. This was more my point. I find it unlikely that Apple were doing most of the testing of macOS-on-ARM (which probably occurred for years prior to the M1 announcement, and prior to the A14X being created) directly using the iOS device architecture. Doing that wouldn’t have allowed them to develop AS support for regular PCIe devices attached through Thunderbolt, for example, since there’s nothing like a PCIe lane in that architecture. Instead, to test things like that, I suspect Apple would have needed some kind of testbench that allowed them to run macOS on an ARM CPU, while attaching arbitrary existing peripherals into said ARM CPU’s address space, without having to fab a new one-off board for it every time they tweaked the proposed architecture. I would guess that their approach, then, would have been very similar to the approach used in prototype bring-up in the game-console industry: - Use a regular Intel machine as a host / hypervisor (in Apple’s case, probably one of the internal mATX-testbench-layout Intel Mac Pros) - Put whatever recent-generation ARM CPU they made, onto a PCIe card - Build a bespoke hypervisor to drive that CPU, which presents to the CPU a virtualized chipset matching the current proposed architecture (e.g. a DCP or not, a Thunderbolt controller, etc.) - Have the hypervisor configure the host’s IOMMU to present both virtual peripherals (for bring-up), and arbitrary host peripherals, to the CPU It’s not like Apple are unfamiliar with “SoCs used to accelerate a host-run emulator”; decades ago, they put an Apple II on an accelerator card and drove it through a host hypervisor just like this :)