5 ms·

Towards the next generation of XNU memory safety: kalloc_type

182 points by olliej 4y ago

sneak 4y ago
- olliej 4y ago
  I actually expected this to have already been posted and couldn’t find it - so the submission went through. Anyway I haven’t ever gone out of my way to hide my employers, and I’m a tech worker in the Bay Area so I’ve worked a multiple companies including Google and Apple. That said if you’d rather I delete this and wait for someone else to post it I can do that? Obviously anything I say is my personal view and not reflective of my employers, past or present, and I’m only ever going to submit things that /I/ think would be of interest to HN. [edited to make sentence that conform to silly societal rules like “must follow basic rules of English grammar”, “must not have absurd amounts of ambiguity”]
  - pvg 4y ago
    rather I delete this and wait for someone else to post it You don't have to change the way you post just because someone is on some innuendo-laden crapcomment bender.
    - olliej 4y ago
      I get the innuendo, but it’s a reasonable thing to think about. The problem of course is that plenty of small companies, startups, and I guess blogs exist where someone might be proud of their work and want to share it on HN which I don’t think should be outright banned. But then you also have the periodic content less and clearly marketing content that ends up on HN front page, which I always find deeply suspicious, so..?
      - pvg 4y ago
        It's reasonable and if you think something is wrong you mail hn@ycombinator.com. But there's no hunting woozles on the forum itself otherwise it would be an infinite woozle hunt, as you know.
        olliej 4y ago
        Ah, I didn't realise that was considered a reasonable option - I mostly rely on downvote or (rarely)flagging
- glhaynes 4y ago
  Genuine question: why do you ask?
  - olliej 4y ago
    I pondered that myself and then realized we’d intersected on some security of iMessage stuff the other day, so it’s reasonable for them to ask “is this a marketing shill?”. Which to be clear I’m not, I think this stuff is interesting, just as I think the Google security posts are. (I just googled and it does look like it’s not as obvious anymore, apparently DJ Olliej is much more popular :D)
    - glhaynes 4y ago
      It just would seem like a more interesting question if this post was a link promoting your private thoughts rather than a generic link to the public blog of one of the largest companies on earth. But maybe the question wasn’t really related to the post in particular.
      - olliej 4y ago
        Like I said we were talking about iMessage security in comments yesterday (or maybe this morning?) so presumably if the next time they saw my nick was in an Apple blog submission they became suspicious that I was a shill. Given that specific context I don’t think it’s wholly unreasonable to question things. The phrasin of the question is obviously unpleasant to me as it does come off as accusatory (due to the conspiratorial implication you get from the “no weaseling” text). But again if someone was a shill you probably would want a question like that. But I’d expect a shill to just not acknowledge the question - it is afaict being fairly heavily downvoted which I don’t think is reasonable either (because everyone loves fake internet points :) ) as that would benefit a shill/marketing person. Anyway to be super clear again: anything I say is my personal thoughts and opinions and in no way reflects what any of my employers, past or present may be thinking or doing. I guess I could put that in my HN bio? I hadn’t previously because I do try to separate my identity from my job, as when I first started out in tech I did not do that, and it was unhealthy.
  - 4y ago
- crecker 4y ago
  ? Anyone can submit a story to HN.
- dang 4y ago
  Surely you've been around HN long enough to know that you can't post like this here, or post personal attacks in general. No more of this, please. And while I'm at it: please stop posting flamebait comments generally. We've asked you this many times, and you're still doing it. That's not cool. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
  - sneak 4y ago
    Apologies; I didn’t realize that this was disallowed. I was sincerely curious and didn’t mean it as a flame or flamebait. I don’t wish to cause any problems for you or HN. I will endeavor to post less.
vlovich123 4y ago
I’m a bit curious how this confers a security advantage. Isn’t the kernel clearing free pages before handing them out? Or does it not bother when it’s a kernel allocation? If the latter, wouldn’t that be an obvious risk mitigation without even needing to segment by type (ie only hand out zeroed pages for allocations)? If they’re being zeroed out, then I’m not sure I understand how grouping by type improves UAF security since the attacker couldn’t control the contents. I’m sure I’m just ignorant here since there’s so much research into this type of hardening. Genuinely curious.
- helloooooooo 4y ago
  It’s not pages. It’s individual allocations. When free-ing and allocation, it returns it back to the free list to be popped off the next time an allocation of appropriate size comes along. Some implementations have a stochastic element to randomize the freelist entry returned to alloc. A type segregated heap mitigates many classes of type confusion exploitation by preventing confusing objects in use-after-free scenarios. It’s also incredibly expensive to zero out free-ed allocations each time.
  - saagarjha 4y ago
    It’s not too bad, the kernel has been doing this for a couple years already.
- saagarjha 4y ago
  I haven’t read the whole thing yet but just zeroing allocations (see comment below on allocations versus pages) is not a full fix, because a UAF can come through a dangling pointer. What you need to mitigate against this is preventing allocations from being reused. This is infeasible to do perfectly because it just means you leak everything but in isolated cases you can do things like prevent different types from being given the same allocation (and thus allowing for shenanigans when code does a type confusion) or do other kinds of segregation and randomization to make it difficult to predict when it will be coming back.
- malf 4y ago
  Search the article for ‘iovec’, they explain exactly that.
MBCook 4y ago
Very interesting. Unfortunately I don’t see a way to follow new entries. There’s no RSS feed.
- musicale 4y ago
  Safari even used to show a little RSS badge on sites that supported it, along with a button that subscribed you to the feed. I miss that.
  - ttepasse 4y ago
    Reeder and NetNewsWire, two great feedreader apps for the Mac, both have a builtin extension as a replacement for that.
davidbalbert 4y ago
The article references a few compiler extensions: __builtin_xnu_type_signature, __builtin_xnu_type_summary, and the xnu_usage_semantics attribute. I can see references to these builtins in the latest xnu source[1], but not in upstream llvm[2], or Apple's forks of llvm[3] and clang[4]. It's been possible to build your own XNU[5] for a long time. Is that still possible now? [1]: https://github.com/apple-oss-distributions/xnu [2]: https://github.com/llvm/llvm-project [3]: https://github.com/apple-oss-distributions/llvmCore [4]: https://github.com/apple-oss-distributions/clang [5]: https://kernelshaman.blogspot.com/2021/02/building-xnu-for-macos-112-intel-apple.html
- saagarjha 4y ago
  I believe Apple’s toolchain ship with this.
  - davidbalbert 4y ago
    You're right. Just compiled the following program with clang from Xcode 14 (Apple clang version 14.0.0 (clang-1400.0.29.102)) without issue. #include <stdio.h> struct foo { uintptr_t p __attribute__((xnu_usage_semantics("pointer"))); }; int main() { struct foo f = { 100 }; printf("%lu\n", f.p); return 0; } Doesn't seem -Wxnu-typed-allocators (which I forgot to mention above) is present in my version. Didn't test the others. I also just realized that the latest tagged release of https://github.com/apple-oss-distributions/clang https://github.com/apple-oss-distributions/clang is 800.0.38, my clang is reporting 1400.0.29.102. Is Apple no longer releasing the source for their compilers?
    - MBCook 4y ago
      The article sounded to me like Apple had a private fork with extra tooling to help with their kernel work.
      - cbsmith 4y ago
        Heh... and it was just a few weeks ago I was assured that all of the platform internals were fully open sourced. ;-)
panny 4y ago
Memory safe C, huh? I wonder why Linux needs to add rust if it's this straightforward?
- olliej 4y ago
  That’s not what this. This is largely about mitigating the damage that can be done once someone finds a memory safety error. Things like pointer authentication, memory tagging, CHERI, aslr, stack cookies, etc, etc are all mitigations to limit what can be done once someone finds a memory error, and all of these things are relatively expensive costs incurred as a result of the lack of safety of the code being written. I am somewhat curious as to whether anyone has done a like vs like benchmark of “real” code with and without these mitigations (and i mean the entire system, libraries, and kernel because otherwise the benchmark picks up those perf hits). We can then compare that cost to the performance gain you get from pointer nonsense, etc people do for performance vs comparable rust/go/swift code that is memory safe. Obviously the mitigations can’t be removed (yet?), but I would like to be able to point to some comparison when people start talking about how much slower safe languages are.
  - musicale 4y ago
    > Things like pointer authentication, memory tagging, CHERI, aslr, stack cookies, etc, etc are all mitigations to limit what can be done once someone finds a memory error, and all of these things are relatively expensive costs incurred as a result of the lack of safety of the code being written Presumably CHERI/MTE/etc. hardware support would help (at the expense of die area and 128-bit pointers); it would be nice to see something like this in Apple Silicon.
    - Veliladon 4y ago
      Apple already does PAC on Apple Silicon. It uses the unused 16 bits of the 64-bit address space on pointers to authenticate them.
      - saagarjha 4y ago
deleted 4y ago
[deleted]
1letterunixname 4y ago
The engineering world has long since moved on from CMU/OSFMK. Rewrite the internals outward using formally-verified, capabilities-secure (seL4 with MAC tagging added) and zero-copy message passing RPC in thread-safe Rust. Take IOKit and make it entirely user-space. And make almost everything run in user-space. The need for worrying about placing and cleaning individual allocations goes by the wayside when there's fewer "cooks" throwing spinning fragile plates around and trusting that their aim is perfect. Apple could rewrite it with the same syscall ABI but make it immensely more secure, compartmentalized, and efficient. They won't because of the performance pressures and rewards their employees are under. This is how big companies fail repeatedly and are bested by startups.
- conradev 4y ago
  The kernel is massive, and rewriting the entire thing takes a serious investment of time (and therefore capital) Zircon is the closest attempt I’ve seen and it has taken years to be shipped on a single, battery-less device. It might be close to ready, but it was likely expensive for Google, which is now scrambling to rein in R&D spending. Meanwhile, the incremental security improvements that Apple and Google have made have driven up the cost of zero days sharply on their devices
  - kergonath 4y ago
    Right. Don’t let the perfect be the enemy of the good. If anything, Apple has demonstrated on several occasions that they are able to commit to long-term changes and the sustained effort they may require. It depends on their analysis, but if they think they should do it, they probably already have a team working on it somewhere and a plan to bring it progressively over ~5 years, with a series of incremental steps.
    - pjmlp 4y ago
      The strategy that was communicated at WWDC was that every time a kext model has an userspace alternative, there will be one year transition, and then the following OS version won't support the former kext variant. The long term roadmap is to keep moving the infrastructure to userspace this way, until no one else besides Apple themselves can touch kernel memory space.

182 points by olliej 4y ago

olliej 4y ago
I actually expected this to have already been posted and couldn’t find it - so the submission went through. Anyway I haven’t ever gone out of my way to hide my employers, and I’m a tech worker in the Bay Area so I’ve worked a multiple companies including Google and Apple. That said if you’d rather I delete this and wait for someone else to post it I can do that? Obviously anything I say is my personal view and not reflective of my employers, past or present, and I’m only ever going to submit things that /I/ think would be of interest to HN. [edited to make sentence that conform to silly societal rules like “must follow basic rules of English grammar”, “must not have absurd amounts of ambiguity”]
- pvg 4y ago
  rather I delete this and wait for someone else to post it You don't have to change the way you post just because someone is on some innuendo-laden crapcomment bender.
  - olliej 4y ago
    I get the innuendo, but it’s a reasonable thing to think about. The problem of course is that plenty of small companies, startups, and I guess blogs exist where someone might be proud of their work and want to share it on HN which I don’t think should be outright banned. But then you also have the periodic content less and clearly marketing content that ends up on HN front page, which I always find deeply suspicious, so..?
    - pvg 4y ago
      It's reasonable and if you think something is wrong you mail hn@ycombinator.com. But there's no hunting woozles on the forum itself otherwise it would be an infinite woozle hunt, as you know.
      - olliej 4y ago
        Ah, I didn't realise that was considered a reasonable option - I mostly rely on downvote or (rarely)flagging
glhaynes 4y ago
Genuine question: why do you ask?
- olliej 4y ago
  I pondered that myself and then realized we’d intersected on some security of iMessage stuff the other day, so it’s reasonable for them to ask “is this a marketing shill?”. Which to be clear I’m not, I think this stuff is interesting, just as I think the Google security posts are. (I just googled and it does look like it’s not as obvious anymore, apparently DJ Olliej is much more popular :D)
  - glhaynes 4y ago
    It just would seem like a more interesting question if this post was a link promoting your private thoughts rather than a generic link to the public blog of one of the largest companies on earth. But maybe the question wasn’t really related to the post in particular.
    - olliej 4y ago
      Like I said we were talking about iMessage security in comments yesterday (or maybe this morning?) so presumably if the next time they saw my nick was in an Apple blog submission they became suspicious that I was a shill. Given that specific context I don’t think it’s wholly unreasonable to question things. The phrasin of the question is obviously unpleasant to me as it does come off as accusatory (due to the conspiratorial implication you get from the “no weaseling” text). But again if someone was a shill you probably would want a question like that. But I’d expect a shill to just not acknowledge the question - it is afaict being fairly heavily downvoted which I don’t think is reasonable either (because everyone loves fake internet points :) ) as that would benefit a shill/marketing person. Anyway to be super clear again: anything I say is my personal thoughts and opinions and in no way reflects what any of my employers, past or present may be thinking or doing. I guess I could put that in my HN bio? I hadn’t previously because I do try to separate my identity from my job, as when I first started out in tech I did not do that, and it was unhealthy.
- 4y ago
crecker 4y ago
? Anyone can submit a story to HN.
dang 4y ago
Surely you've been around HN long enough to know that you can't post like this here, or post personal attacks in general. No more of this, please. And while I'm at it: please stop posting flamebait comments generally. We've asked you this many times, and you're still doing it. That's not cool. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- sneak 4y ago
  Apologies; I didn’t realize that this was disallowed. I was sincerely curious and didn’t mean it as a flame or flamebait. I don’t wish to cause any problems for you or HN. I will endeavor to post less.

I’m a bit curious how this confers a security advantage. Isn’t the kernel clearing free pages before handing them out? Or does it not bother when it’s a kernel allocation? If the latter, wouldn’t that be an obvious risk mitigation without even needing to segment by type (ie only hand out zeroed pages for allocations)? If they’re being zeroed out, then I’m not sure I understand how grouping by type improves UAF security since the attacker couldn’t control the contents. I’m sure I’m just ignorant here since there’s so much research into this type of hardening. Genuinely curious.

helloooooooo 4y ago
It’s not pages. It’s individual allocations. When free-ing and allocation, it returns it back to the free list to be popped off the next time an allocation of appropriate size comes along. Some implementations have a stochastic element to randomize the freelist entry returned to alloc. A type segregated heap mitigates many classes of type confusion exploitation by preventing confusing objects in use-after-free scenarios. It’s also incredibly expensive to zero out free-ed allocations each time.
- saagarjha 4y ago
  It’s not too bad, the kernel has been doing this for a couple years already.
saagarjha 4y ago
I haven’t read the whole thing yet but just zeroing allocations (see comment below on allocations versus pages) is not a full fix, because a UAF can come through a dangling pointer. What you need to mitigate against this is preventing allocations from being reused. This is infeasible to do perfectly because it just means you leak everything but in isolated cases you can do things like prevent different types from being given the same allocation (and thus allowing for shenanigans when code does a type confusion) or do other kinds of segregation and randomization to make it difficult to predict when it will be coming back.
malf 4y ago
Search the article for ‘iovec’, they explain exactly that.

MBCook 4y ago

Very interesting. Unfortunately I don’t see a way to follow new entries. There’s no RSS feed.

musicale 4y ago
Safari even used to show a little RSS badge on sites that supported it, along with a button that subscribed you to the feed. I miss that.
- ttepasse 4y ago
  Reeder and NetNewsWire, two great feedreader apps for the Mac, both have a builtin extension as a replacement for that.

davidbalbert 4y ago

The article references a few compiler extensions: __builtin_xnu_type_signature, __builtin_xnu_type_summary, and the xnu_usage_semantics attribute. I can see references to these builtins in the latest xnu source[1], but not in upstream llvm[2], or Apple's forks of llvm[3] and clang[4]. It's been possible to build your own XNU[5] for a long time. Is that still possible now? [1]: https://github.com/apple-oss-distributions/xnu [2]: https://github.com/llvm/llvm-project [3]: https://github.com/apple-oss-distributions/llvmCore [4]: https://github.com/apple-oss-distributions/clang [5]: https://kernelshaman.blogspot.com/2021/02/building-xnu-for-macos-112-intel-apple.html

saagarjha 4y ago
I believe Apple’s toolchain ship with this.
- davidbalbert 4y ago
  You're right. Just compiled the following program with clang from Xcode 14 (Apple clang version 14.0.0 (clang-1400.0.29.102)) without issue. #include <stdio.h> struct foo { uintptr_t p __attribute__((xnu_usage_semantics("pointer"))); }; int main() { struct foo f = { 100 }; printf("%lu\n", f.p); return 0; } Doesn't seem -Wxnu-typed-allocators (which I forgot to mention above) is present in my version. Didn't test the others. I also just realized that the latest tagged release of https://github.com/apple-oss-distributions/clang https://github.com/apple-oss-distributions/clang is 800.0.38, my clang is reporting 1400.0.29.102. Is Apple no longer releasing the source for their compilers?
  - MBCook 4y ago
    The article sounded to me like Apple had a private fork with extra tooling to help with their kernel work.
    - cbsmith 4y ago
      Heh... and it was just a few weeks ago I was assured that all of the platform internals were fully open sourced. ;-)

panny 4y ago

Memory safe C, huh? I wonder why Linux needs to add rust if it's this straightforward?

olliej 4y ago
That’s not what this. This is largely about mitigating the damage that can be done once someone finds a memory safety error. Things like pointer authentication, memory tagging, CHERI, aslr, stack cookies, etc, etc are all mitigations to limit what can be done once someone finds a memory error, and all of these things are relatively expensive costs incurred as a result of the lack of safety of the code being written. I am somewhat curious as to whether anyone has done a like vs like benchmark of “real” code with and without these mitigations (and i mean the entire system, libraries, and kernel because otherwise the benchmark picks up those perf hits). We can then compare that cost to the performance gain you get from pointer nonsense, etc people do for performance vs comparable rust/go/swift code that is memory safe. Obviously the mitigations can’t be removed (yet?), but I would like to be able to point to some comparison when people start talking about how much slower safe languages are.
- musicale 4y ago
  > Things like pointer authentication, memory tagging, CHERI, aslr, stack cookies, etc, etc are all mitigations to limit what can be done once someone finds a memory error, and all of these things are relatively expensive costs incurred as a result of the lack of safety of the code being written Presumably CHERI/MTE/etc. hardware support would help (at the expense of die area and 128-bit pointers); it would be nice to see something like this in Apple Silicon.
  - Veliladon 4y ago
    Apple already does PAC on Apple Silicon. It uses the unused 16 bits of the 64-bit address space on pointers to authenticate them.
    - saagarjha 4y ago

deleted 4y ago

[deleted]

1letterunixname 4y ago

The engineering world has long since moved on from CMU/OSFMK. Rewrite the internals outward using formally-verified, capabilities-secure (seL4 with MAC tagging added) and zero-copy message passing RPC in thread-safe Rust. Take IOKit and make it entirely user-space. And make almost everything run in user-space. The need for worrying about placing and cleaning individual allocations goes by the wayside when there's fewer "cooks" throwing spinning fragile plates around and trusting that their aim is perfect. Apple could rewrite it with the same syscall ABI but make it immensely more secure, compartmentalized, and efficient. They won't because of the performance pressures and rewards their employees are under. This is how big companies fail repeatedly and are bested by startups.

conradev 4y ago
The kernel is massive, and rewriting the entire thing takes a serious investment of time (and therefore capital) Zircon is the closest attempt I’ve seen and it has taken years to be shipped on a single, battery-less device. It might be close to ready, but it was likely expensive for Google, which is now scrambling to rein in R&D spending. Meanwhile, the incremental security improvements that Apple and Google have made have driven up the cost of zero days sharply on their devices
- kergonath 4y ago
  Right. Don’t let the perfect be the enemy of the good. If anything, Apple has demonstrated on several occasions that they are able to commit to long-term changes and the sustained effort they may require. It depends on their analysis, but if they think they should do it, they probably already have a team working on it somewhere and a plan to bring it progressively over ~5 years, with a series of incremental steps.
  - pjmlp 4y ago
    The strategy that was communicated at WWDC was that every time a kext model has an userspace alternative, there will be one year transition, and then the following OS version won't support the former kext variant. The long term roadmap is to keep moving the infrastructure to userspace this way, until no one else besides Apple themselves can touch kernel memory space.