6 ms·

Yeah - the failed mitigations followed by a "long term" fix was interesting as well. Apple literally had to change execve() this late in the OS's development cy

Yeah - the failed mitigations followed by a "long term" fix was interesting as well. Apple literally had to change execve() this late in the OS's development cycle to allocate new task and thread structs(that's two extra allocations and copies in hot path!) to fix it for good. That this design problem lingered around for so long doesn't look good for Apple - it's one thing for a use after free bug in obscure piece of code to linger but bad design affecting a ton of their own frequently used code to stand so long is somewhat brown bag category! I wonder what other fallout we might experience from this.

deleted 10y ago
[deleted]
deleted 10y ago
[deleted]
blumentopf 10y ago
Business as usual with macOS. The other day I was browsing the ocspd source code. Turns out it calls openssl using system(). So openssl is officially deprecated on macOS and yet they're using it internally to handle certificates?! And there's an enlightening comment: /* Given a path to a DER-encoded CRL file and a path to a PEM-encoded * CA issuers file, use OpenSSL to validate the CRL. This is a hack, * necessitated by performance issues with inserting extremely large * numbers of CRL entries into a CSSM DB (see <rdar://8934440>). http://opensource.apple.com/source/security_ocspd/security_ocspd-55128.40.1/server/ocspdServer.cpp http://opensource.apple.com/source/security_ocspd/security_o... ocspd was introduced with 10.4. A decade ago. And that's really the problem with macOS: There's no refactoring of old hacks, but rather just bolting on of ever more new stuff.
- 0x0 10y ago
  o.O
- zimpenfish 10y ago
  Linking to openssl was deprecated (lack of binary stability) - not the command line tools.
- mikeash 10y ago
  I don't see a major problem with using the openssl command for this, but using system() to do it is completely insane.
- wyager 10y ago
  Apple needs to take a bit of those tens of billions of dollars they have sitting around and spend it on starting from scratch with something that's not horrifically crufty. The quality of their software is lagging so far behind the quality of their hardware right now. Realistically, I think we may just be at the point where operating systems and all the stuff the companies put on top of them are too complicated to keep developing in the traditional way with traditional tools. Formal verification might be the cheapest way forward at this point.
  - JoeAltmaier 10y ago
    Hundreds of billions?
    - protomyth 10y ago
      Yep, $237.6B in cash and equivalents http://blogs.marketwatch.com/thetell/2016/10/25/apple-earnings-expected-to-show-another-revenue-decline-live-blog/ http://blogs.marketwatch.com/thetell/2016/10/25/apple-earnin...
  - tptacek 10y ago
    So far as the current state of the art in computer engineering goes, we don't know how to completely rewrite a system as complicated as XNU without creating fresh batches of implementation errors. So this is a little like suggesting Apple use its hundreds of billions of dollars to build an iPhone battery that only needs to be recharged once a month. We may someday get an XNU rewrite, but probably not until software engineering produces a new approach to building complex systems reliably that works at the scale (here: number of developers and shipping schedule) Apple needs.
    - pcwalton 10y ago
      > We may someday get an XNU rewrite, but probably not until software engineering produces a new approach to building complex systems reliably that works at the scale (here: number of developers and shipping schedule) Apple needs. It's conceivable to perform a gradual transition away, though. They could demote Mach to a fast IPC system that just augments BSD, similar to the way the kdbus/bus1 proposal for Linux does. That would be difficult and a long-term project, but it would fix the underlying issue in a way that mostly retains userspace compatibility. Driver compatibility would be more difficult, of course…
kevingadd 10y ago
Incidentally the huge set of problems with execve were very old news - I know a developer who tried to get them fixed (because they affected a piece of Apple software) and failed. Presumably this was because execve bugs were considered unimportant. Oops!
tptacek 10y ago
This is kind of a "perfect storm" situation for Apple. At least three vectors are converging: 1. Apple inherited OSX from NeXT, and with it the Mach subsystem. Mach overcomplicates XNU. 2. XNU has become incredibly popular by dint of being shipped in the iPhone. Avie Tevanian probably did not see that coming when they designed the original BSD/Cocoa/XNU/whatever architecture. Regardless: it is now difficult to make sweeping architectural changes in XNU, because of the enormous installed base. 3. Ian Beer is simultaneously very clever and also willing to wade into the XNU Mach fire swamp. I think it's fair to criticize Apple for designing the XNU frankenkernel. I think it's less legit to say that the presence of this bug class "looks bad" --- it's 2016 and this is just getting published. This is one of those bug classes that is sort of obvious in retrospect, and you wonder why people didn't catch it earlier.
- blinkingled 10y ago
  Keep in mind that this is a dangling pointer problem that is a stupid design mistake and Apple's own kexts used the same vulnerable pattern for years before Project Zero had to disclose it to Apple after which there were two failed mitigations and a fix that changes core OS code! That's not very justifiable no matter how many storms and twisters you throw in the mix ;) I am not going to argue but for OS development this looks bad that it wasn't even looked at much rather fixed for so long given many of Apple's kexts had the same issue. [Also this isn't an isolated problem by the way - look up the thread and you'll find two other egregious errors including one involving execve that was pointed to Apple in vain. Not long ago I could just remove and insert a kmod and that would lock up the system - go ahead and argue that's not a hot path or nobody does that but it does speak to general lack of code and testing quality. To their credit Apple fixed that particular one though when I reported them.]
  - tptacek 10y ago
    The dangling pointer isn't the interesting part of the bug --- dangling pointers are, to a first approximation, the key component of all UAFs, which are a 15 year old bug class. The interesting bit about this bug is how they arrived at the dangling pointer: they had to manipulate at least three different object lifecycles in the kernel, one of of which involved a credential passing trick that is not super common in normal code. I disagree that this is a simple bug that anyone should have been able to find. If you'd like to put money on whether or not this is going to win a 2017 Pwnie Award, I would be happy to take your money.
hueving 10y ago
>(that's two extra allocations and copies in hot path!) I've never really thought of process spawning as a hot-path (in the hundreds+ of calls per second sense). What software so heavily relies on spawning so many processes so quickly that the overhead of malloc would be noticeable?
- tptacek 10y ago
  Yeah, I don't see the hot path problem either. I think the bigger issue is doing deep brain surgery on XNU as a hotfix; it's the kind of thing you want to put off for a next major release.
- blinkingled 10y ago
  On UNIX systems fork+execve is a very commonly used code path - build a piece of software using make for example and the compiler process is forked/exec'ed lot of times, web servers like Apache used multiprocess model for long time etc. Yeah you could decide to not care about fork+exec* performance but what I am familiar with (Linux land) a lot of optimizations go into making fork/exec faster. Also my bigger point though was not that it was in hot code path (although I would prefer to not be in a position where I need to change execve to add two allocs and copies if I can avoid it) - it was that they had to change execve() this late in the OS's development cycle to fix this long standing bug - typically you get to a point where you don't really need to touch core OS code and when you do you risk adding new issues to a central piece of code.
abraham6464 10y ago
Pardon my ignorance, could you explain what "hot path" means in this context?

deleted 10y ago

[deleted]

deleted 10y ago

[deleted]

blumentopf 10y ago

Business as usual with macOS. The other day I was browsing the ocspd source code. Turns out it calls openssl using system(). So openssl is officially deprecated on macOS and yet they're using it internally to handle certificates?! And there's an enlightening comment: /* Given a path to a DER-encoded CRL file and a path to a PEM-encoded * CA issuers file, use OpenSSL to validate the CRL. This is a hack, * necessitated by performance issues with inserting extremely large * numbers of CRL entries into a CSSM DB (see <rdar://8934440>). http://opensource.apple.com/source/security_ocspd/security_ocspd-55128.40.1/server/ocspdServer.cpp http://opensource.apple.com/source/security_ocspd/security_o... ocspd was introduced with 10.4. A decade ago. And that's really the problem with macOS: There's no refactoring of old hacks, but rather just bolting on of ever more new stuff.

0x0 10y ago
o.O
zimpenfish 10y ago
Linking to openssl was deprecated (lack of binary stability) - not the command line tools.
mikeash 10y ago
I don't see a major problem with using the openssl command for this, but using system() to do it is completely insane.
wyager 10y ago
Apple needs to take a bit of those tens of billions of dollars they have sitting around and spend it on starting from scratch with something that's not horrifically crufty. The quality of their software is lagging so far behind the quality of their hardware right now. Realistically, I think we may just be at the point where operating systems and all the stuff the companies put on top of them are too complicated to keep developing in the traditional way with traditional tools. Formal verification might be the cheapest way forward at this point.
- JoeAltmaier 10y ago
  Hundreds of billions?
  - protomyth 10y ago
    Yep, $237.6B in cash and equivalents http://blogs.marketwatch.com/thetell/2016/10/25/apple-earnings-expected-to-show-another-revenue-decline-live-blog/ http://blogs.marketwatch.com/thetell/2016/10/25/apple-earnin...
- tptacek 10y ago
  So far as the current state of the art in computer engineering goes, we don't know how to completely rewrite a system as complicated as XNU without creating fresh batches of implementation errors. So this is a little like suggesting Apple use its hundreds of billions of dollars to build an iPhone battery that only needs to be recharged once a month. We may someday get an XNU rewrite, but probably not until software engineering produces a new approach to building complex systems reliably that works at the scale (here: number of developers and shipping schedule) Apple needs.
  - pcwalton 10y ago
    > We may someday get an XNU rewrite, but probably not until software engineering produces a new approach to building complex systems reliably that works at the scale (here: number of developers and shipping schedule) Apple needs. It's conceivable to perform a gradual transition away, though. They could demote Mach to a fast IPC system that just augments BSD, similar to the way the kdbus/bus1 proposal for Linux does. That would be difficult and a long-term project, but it would fix the underlying issue in a way that mostly retains userspace compatibility. Driver compatibility would be more difficult, of course…

kevingadd 10y ago

Incidentally the huge set of problems with execve were very old news - I know a developer who tried to get them fixed (because they affected a piece of Apple software) and failed. Presumably this was because execve bugs were considered unimportant. Oops!

tptacek 10y ago

This is kind of a "perfect storm" situation for Apple. At least three vectors are converging: 1. Apple inherited OSX from NeXT, and with it the Mach subsystem. Mach overcomplicates XNU. 2. XNU has become incredibly popular by dint of being shipped in the iPhone. Avie Tevanian probably did not see that coming when they designed the original BSD/Cocoa/XNU/whatever architecture. Regardless: it is now difficult to make sweeping architectural changes in XNU, because of the enormous installed base. 3. Ian Beer is simultaneously very clever and also willing to wade into the XNU Mach fire swamp. I think it's fair to criticize Apple for designing the XNU frankenkernel. I think it's less legit to say that the presence of this bug class "looks bad" --- it's 2016 and this is just getting published. This is one of those bug classes that is sort of obvious in retrospect, and you wonder why people didn't catch it earlier.

blinkingled 10y ago
Keep in mind that this is a dangling pointer problem that is a stupid design mistake and Apple's own kexts used the same vulnerable pattern for years before Project Zero had to disclose it to Apple after which there were two failed mitigations and a fix that changes core OS code! That's not very justifiable no matter how many storms and twisters you throw in the mix ;) I am not going to argue but for OS development this looks bad that it wasn't even looked at much rather fixed for so long given many of Apple's kexts had the same issue. [Also this isn't an isolated problem by the way - look up the thread and you'll find two other egregious errors including one involving execve that was pointed to Apple in vain. Not long ago I could just remove and insert a kmod and that would lock up the system - go ahead and argue that's not a hot path or nobody does that but it does speak to general lack of code and testing quality. To their credit Apple fixed that particular one though when I reported them.]
- tptacek 10y ago
  The dangling pointer isn't the interesting part of the bug --- dangling pointers are, to a first approximation, the key component of all UAFs, which are a 15 year old bug class. The interesting bit about this bug is how they arrived at the dangling pointer: they had to manipulate at least three different object lifecycles in the kernel, one of of which involved a credential passing trick that is not super common in normal code. I disagree that this is a simple bug that anyone should have been able to find. If you'd like to put money on whether or not this is going to win a 2017 Pwnie Award, I would be happy to take your money.

hueving 10y ago

>(that's two extra allocations and copies in hot path!) I've never really thought of process spawning as a hot-path (in the hundreds+ of calls per second sense). What software so heavily relies on spawning so many processes so quickly that the overhead of malloc would be noticeable?

tptacek 10y ago
Yeah, I don't see the hot path problem either. I think the bigger issue is doing deep brain surgery on XNU as a hotfix; it's the kind of thing you want to put off for a next major release.
blinkingled 10y ago
On UNIX systems fork+execve is a very commonly used code path - build a piece of software using make for example and the compiler process is forked/exec'ed lot of times, web servers like Apache used multiprocess model for long time etc. Yeah you could decide to not care about fork+exec* performance but what I am familiar with (Linux land) a lot of optimizations go into making fork/exec faster. Also my bigger point though was not that it was in hot code path (although I would prefer to not be in a position where I need to change execve to add two allocs and copies if I can avoid it) - it was that they had to change execve() this late in the OS's development cycle to fix this long standing bug - typically you get to a point where you don't really need to touch core OS code and when you do you risk adding new issues to a central piece of code.

abraham6464 10y ago

Pardon my ignorance, could you explain what "hot path" means in this context?