5 ms·

I'm not really a developer so maybe I'm just not understanding something, but why in the world isn't Apple making it easier for people to optimize for the M1? I

by glial 6y ago

I'm not really a developer so maybe I'm just not understanding something, but why in the world isn't Apple making it easier for people to optimize for the M1? I would think it's in their best interest to help developers make the best software possible, by sharing information about how to leverage the architecture. It's bizarre to me that the best sources of information are posts like this.

dylan604 6y ago
Why do you think they are not? They are helping developers developing for the macOS platform that they own,develop,support. They are not responsible for 3rd part OSes and the development on those platforms. Why would you expect anything else based on their history and track record for closed/walled ecosystems?
judge2020 6y ago
Believe it or not, you're just purchasing rare earth materials packed very tightly into a nice box when you buy a Mac or an iPhone. The operating system is paid for on the backend by developers giving up 30% of revenue that goes through the App Store. The M1 is turning the Mac into an iPhone in exchange for an extremely fast processor and insane battery life, so they're not interested in helping you bypass the technical challenges of running other operating systems on their hardware (much like how they don't help you jailbreak iOS in order to help you install Cydia or third-party app stores).
- marcan_42 6y ago
  You are drawing a false equivalency between the Mac and iPhones. iOS devices are deliberately locked so that running your own low-level software on them is not supposed to be possible, and requires breaking Apple's security. If they make no mistakes, doing it is completely impractical (the cost of an attack outside their attack model is greater than the price of the device). macOS devices are not, and Apple invested significant development effort into allowing third-party kernels on M1 Macs. The situation is very different. They are not actively supporting the development of third-party OSes, but they are actively supporting their existence. They built an entire boot policy system to allow not just this, but insecure/non-Apple-signed OSes to coexist and dual-boot on the same device next to a full-secure blessed macOS with all of their DRM stuff enabled, which is something not even open Android devices do. You can triple-boot a macOS capable of running iOS apps (only possible with secureboot enabled), a macOS running unsigned kernel modules or even your own XNU kernel build, and Linux on the same M1 Mac.
  - refulgentis 6y ago
    This pales in comparison to the rhetoric when OS X came out - you see that rhetoric survive today at opensource.apple.com, it's there, just, not in the spirit of the freedom that was promised.
    - marcan_42 6y ago
      Sure, but Apple didn't promise any openness with the M1, and over-delivered :)
    - coldtea 6y ago
      Since then, RedHat has been turned into a profit machine and is now owned by IBM and SUN has folded...
  - CyberRabbi 6y ago
    I acknowledge that it’s possible to run unsigned code at “ring 0” on M1 MacBooks but the existence of the DRM restrictions leads me to believe that certain DRM-relevant hardware is not accessible unless a signed OS is running. I’m not exactly sure how the attestation works from software to hardware but I have to guess that it exists, otherwise it would be relatively trivial to load a kext that enables 4K DRM with SIP disabled. One may consider that not important but I think it’s important to at least note the hardware of these machines are not fully under user software control. Then again, I don’t think the iOS support requires any hardware so I’m not sure why someone hasn’t released a mod (requiring a kext or not) that enables iOS app loading with SIP disabled.
    - marcan_42 6y ago
      The Secure Enclave Processor (SEP) is the part that you cannot run your own code on, and it knows what state the device was booted in. However, it doesn't get in the way of normal OS usage. It also serves as a secure element, e.g. we can use it to store SSH keys and authorize use with Touch ID, or to secure a password store, or as a U2F token, just like macOS does. This is much better than most unlocked Android devices, which do this kind of thing in EL3/TrustZone, which runs on the main CPU and means there is always a proprietary, locked "supervisor" over your OS. This is also the case on x86, where the firmware runs code in SMM (ring -2) that the OS cannot touch or control, and in addition there is a management engine running even more intrusive stuff. On the M1, the main CPU is owned entirely by the OS, and doesn't even implement EL3 (only EL2, the VM hypervisor level, which we boot in - we already have the Linux KVM hypervisor working on the M1), making it a much more user-controlled execution environment than almost all x86 machines and Android phones. In fact, the SEP is re-bootstrapped by the main OS itself after the boot stuff is done (we get the SEP firmware blob in memory passed in from the bootloader), so even though we cannot run our own code on it, we can choose not to run Apple's signed code either, and thus guarantee that it isn't doing nefarious things behind our back. For most intents and purposes, it's like an optional built-in YubiKey with a fingerprint reader. iOS apps should be FairPlay encrypted AIUI, and presumably that goes through the SEP to only authorize decryption when booted in secure mode. That's my understanding anyway, I haven't had anything to do with that ecosystem for ages. Of course, either way you could load decrypted iOS apps just like you can pirate iOS apps on a jailbroken phone. The M1 does have other co-processor CPUs that run signed firmware loaded before Linux boots (e.g. for power management, graphics management, sensors, etc), but all of those other firmware blobs are plaintext (only the SEP one is encrypted); we may not be able to change them (unclear yet how much control we have over those CPUs post-iBoot, there might be a way to stop them or reload the firmware) but we can at least reverse engineer them and audit that they don't do anything nasty. Besides, I think most of this stuff goes through OS-controlled IOMMUs to access memory anyway, so it can't do much harm to the main OS.
- Rhedox 6y ago
  TBF they explicitly implemented a way to boot other kernels. 'turning the Mac into an iPhone' suggest they are locking it down, which isn't entirely true. They could do more to help driver development though.
dev_tty01 6y ago
Apple is helping developers make the best software possible. They have a full suite of incredibly optimized and performant frameworks for compute, graphics, signal processing, machine learning, rendering, animation, networking, etc... That is all available via free download for writing MacOS and iOS apps. Remember, they are selling computers as a combination of hardware and software. They are not selling processors so they are of course not supporting driver and other low level software for other OSs. That's a bummer if you are into other OSs, but it is not part of their business model so it should not be surprising. Other OSs are supported via their virtualization framework. My limited tests show about a 10% performance penalty. Not too bad right out of the gate with a new system. That being said, Ms. Rosenzweig is doing some incredible and interesting work. Really enjoying the series.
- glial 6y ago
  That's helpful, thank you!
saagarjha 6y ago
In a few months Apple is going to release a successor to the M1 processor, and then maybe in a year or two another revision after that. Apple would like your code to be optimized for that processor as well, in addition to the M1. The way Apple does this is by wrapping their hardware in a high-level API that knows how to use the hardware it is running on, rather than exposing architectural details and having developers hardcode assumptions into their code until the end of time.
- snovv_crash 6y ago
  Abstractions are both leaky and expensive. There are a lot of things that could have much better performance if they had access to the lower level APIs.
  - astrange 6y ago
    Metal is a good fit for the M1 GPU (since the GPU was essentially designed to run it). There isn't a need for a lower level API than Metal. Most people will not end up writing code in the optimal way though, since they also want to support discrete GPUs with their own VRAM and those have totally different memory management.
nash 6y ago
Apple's official graphics API is Metal. There is plenty of documentation for that. Apple considers Metal a commercial advantage over OpenGL / Vulkan. That is; they want developers to develop against Metal.

by glial 6y ago

dylan604 6y ago

Why do you think they are not? They are helping developers developing for the macOS platform that they own,develop,support. They are not responsible for 3rd part OSes and the development on those platforms. Why would you expect anything else based on their history and track record for closed/walled ecosystems?

judge2020 6y ago

Believe it or not, you're just purchasing rare earth materials packed very tightly into a nice box when you buy a Mac or an iPhone. The operating system is paid for on the backend by developers giving up 30% of revenue that goes through the App Store. The M1 is turning the Mac into an iPhone in exchange for an extremely fast processor and insane battery life, so they're not interested in helping you bypass the technical challenges of running other operating systems on their hardware (much like how they don't help you jailbreak iOS in order to help you install Cydia or third-party app stores).

marcan_42 6y ago
You are drawing a false equivalency between the Mac and iPhones. iOS devices are deliberately locked so that running your own low-level software on them is not supposed to be possible, and requires breaking Apple's security. If they make no mistakes, doing it is completely impractical (the cost of an attack outside their attack model is greater than the price of the device). macOS devices are not, and Apple invested significant development effort into allowing third-party kernels on M1 Macs. The situation is very different. They are not actively supporting the development of third-party OSes, but they are actively supporting their existence. They built an entire boot policy system to allow not just this, but insecure/non-Apple-signed OSes to coexist and dual-boot on the same device next to a full-secure blessed macOS with all of their DRM stuff enabled, which is something not even open Android devices do. You can triple-boot a macOS capable of running iOS apps (only possible with secureboot enabled), a macOS running unsigned kernel modules or even your own XNU kernel build, and Linux on the same M1 Mac.
- refulgentis 6y ago
  This pales in comparison to the rhetoric when OS X came out - you see that rhetoric survive today at opensource.apple.com, it's there, just, not in the spirit of the freedom that was promised.
  - marcan_42 6y ago
    Sure, but Apple didn't promise any openness with the M1, and over-delivered :)
  - coldtea 6y ago
    Since then, RedHat has been turned into a profit machine and is now owned by IBM and SUN has folded...
- CyberRabbi 6y ago
  I acknowledge that it’s possible to run unsigned code at “ring 0” on M1 MacBooks but the existence of the DRM restrictions leads me to believe that certain DRM-relevant hardware is not accessible unless a signed OS is running. I’m not exactly sure how the attestation works from software to hardware but I have to guess that it exists, otherwise it would be relatively trivial to load a kext that enables 4K DRM with SIP disabled. One may consider that not important but I think it’s important to at least note the hardware of these machines are not fully under user software control. Then again, I don’t think the iOS support requires any hardware so I’m not sure why someone hasn’t released a mod (requiring a kext or not) that enables iOS app loading with SIP disabled.
  - marcan_42 6y ago
    The Secure Enclave Processor (SEP) is the part that you cannot run your own code on, and it knows what state the device was booted in. However, it doesn't get in the way of normal OS usage. It also serves as a secure element, e.g. we can use it to store SSH keys and authorize use with Touch ID, or to secure a password store, or as a U2F token, just like macOS does. This is much better than most unlocked Android devices, which do this kind of thing in EL3/TrustZone, which runs on the main CPU and means there is always a proprietary, locked "supervisor" over your OS. This is also the case on x86, where the firmware runs code in SMM (ring -2) that the OS cannot touch or control, and in addition there is a management engine running even more intrusive stuff. On the M1, the main CPU is owned entirely by the OS, and doesn't even implement EL3 (only EL2, the VM hypervisor level, which we boot in - we already have the Linux KVM hypervisor working on the M1), making it a much more user-controlled execution environment than almost all x86 machines and Android phones. In fact, the SEP is re-bootstrapped by the main OS itself after the boot stuff is done (we get the SEP firmware blob in memory passed in from the bootloader), so even though we cannot run our own code on it, we can choose not to run Apple's signed code either, and thus guarantee that it isn't doing nefarious things behind our back. For most intents and purposes, it's like an optional built-in YubiKey with a fingerprint reader. iOS apps should be FairPlay encrypted AIUI, and presumably that goes through the SEP to only authorize decryption when booted in secure mode. That's my understanding anyway, I haven't had anything to do with that ecosystem for ages. Of course, either way you could load decrypted iOS apps just like you can pirate iOS apps on a jailbroken phone. The M1 does have other co-processor CPUs that run signed firmware loaded before Linux boots (e.g. for power management, graphics management, sensors, etc), but all of those other firmware blobs are plaintext (only the SEP one is encrypted); we may not be able to change them (unclear yet how much control we have over those CPUs post-iBoot, there might be a way to stop them or reload the firmware) but we can at least reverse engineer them and audit that they don't do anything nasty. Besides, I think most of this stuff goes through OS-controlled IOMMUs to access memory anyway, so it can't do much harm to the main OS.
Rhedox 6y ago
TBF they explicitly implemented a way to boot other kernels. 'turning the Mac into an iPhone' suggest they are locking it down, which isn't entirely true. They could do more to help driver development though.

dev_tty01 6y ago

Apple is helping developers make the best software possible. They have a full suite of incredibly optimized and performant frameworks for compute, graphics, signal processing, machine learning, rendering, animation, networking, etc... That is all available via free download for writing MacOS and iOS apps. Remember, they are selling computers as a combination of hardware and software. They are not selling processors so they are of course not supporting driver and other low level software for other OSs. That's a bummer if you are into other OSs, but it is not part of their business model so it should not be surprising. Other OSs are supported via their virtualization framework. My limited tests show about a 10% performance penalty. Not too bad right out of the gate with a new system. That being said, Ms. Rosenzweig is doing some incredible and interesting work. Really enjoying the series.

glial 6y ago
That's helpful, thank you!

saagarjha 6y ago

In a few months Apple is going to release a successor to the M1 processor, and then maybe in a year or two another revision after that. Apple would like your code to be optimized for that processor as well, in addition to the M1. The way Apple does this is by wrapping their hardware in a high-level API that knows how to use the hardware it is running on, rather than exposing architectural details and having developers hardcode assumptions into their code until the end of time.

snovv_crash 6y ago
Abstractions are both leaky and expensive. There are a lot of things that could have much better performance if they had access to the lower level APIs.
- astrange 6y ago
  Metal is a good fit for the M1 GPU (since the GPU was essentially designed to run it). There isn't a need for a lower level API than Metal. Most people will not end up writing code in the optimal way though, since they also want to support discrete GPUs with their own VRAM and those have totally different memory management.

nash 6y ago

Apple's official graphics API is Metal. There is plenty of documentation for that. Apple considers Metal a commercial advantage over OpenGL / Vulkan. That is; they want developers to develop against Metal.