11 ms·
Reverse Engineering MacOS High Sierra Supplemental Update
- atas 9y agoSimple technique for preventing bugs like this: don't copy-paste code. If you find yourself copy-pasting code think twice why you even have to do it (DRY principle) and be aware of the potential consequences. Even if some code has to be duplicate, I am forcing myself to just write it from scratch, exactly for this reason: do you really know that you have updated all your data? And yeah, unit tests would help in spotting this and other things. But aren't unit tests a duplication of your code, already? UPDATE: Maybe I should have stated my last question differently. I meant that in the context of checking that the data is correct, it would be the same as writing the duplicate code from scratch.
- blowski 9y agoJust to play devil's advocate: sometimes DRY code is harder to understand, which makes it harder to see bugs. Whenever you introduce an abstraction to make your code DRYer, you have to remember the law of leaky abstractions. DRY code is a good principle to follow, but not an absolute law.
- atas 9y agoYes, and that's why you have to be extra-careful with copy-pasting :-)
- eru 9y agoSome languages are leakier than others.. As for terse code: a line that's five times as hard to understand might be worth it, if it saves ten. (But I usually code in languages that are famously terse and have watertight abstractions---at least in the correctness sense, even if not in the performance sense.)
- icebraining 9y agoBut aren't unit tests a duplication of your code, already? No?
- slhck 9y ago> But aren't unit tests a duplication of your code, already? No, as you'd only be specifying the input and output state, e.g. the change of a model's attributes or the correctness of a mathematical calculation. The code that implements the (business) logic of going from input to output state should be part of your application, not the test.
- deleted 9y ago[deleted]
- eru 9y ago> But aren't unit tests a duplication of your code, already? > UPDATE: > Maybe I should have stated my last question differently. I meant that in the context of checking that the data is correct, it would be the same as writing the duplicate code from scratch. Good units tests, especially for UI entry are difficult to envision. Have a look at https://news.ycombinator.com/item?id=15432567 https://news.ycombinator.com/item?id=15432567 where I tried to sketch a general way to address this class of problems.
- sparkpeasy 9y agoScary to think that the reveal of all passwords on MacOS shoulders on one person's accidental copy + paste of one line.
- saagarjha 9y agoIt’s not a reveal of all the passwords, certainly. It’s only disclosing the disk’s password, and only in the situation provided.
- IncRnd 9y agoif ((err = SSLHashSHA1.update(&hashCtx, &signedParams)) != 0) goto fail; goto fail; /* MISTAKE! THIS LINE SHOULD NOT BE HERE */ if ((err = SSLHashSHA1.final(&hashCtx, &hashOut)) != 0) goto fail; History repeats itself in ever stranger ways. While the effects are different, the original mistake can be quite similar.
- saagarjha 9y agoThis is the SSL bug from iOS 9 for those wondering.
- caseymarquis 9y agoNeat that the textbook justification for Python whitespace syntax over curly brace syntax has caused serious real world bugs.
- emsy 9y agoYou don't even need static analysis (which they should use too) but just a strict coding style check that enforces curly braces. Should've happened after the first incident!
- tempodox 9y agoI find it jaw-dropping how fashionable cowboy coding still is, especially in C family languages.
- eru 9y agoC-family languages are cowboy languages. The whole point of eg C (and C++)'s undefined behaviour is to allow the compiler to make cowboy assumptions like "this array access will never be out of bounds" or "this signed int will never overflow" without having to prove or even justify them. All in the name of 'efficiency'.
- eru 9y agoA coding style check _is_ static analysis. (But that's beating a straw man, of course: with that expanded definition your comment just because "you don't even need anything more than the most primitive forms of static analysis".)
- bsaul 9y agoSeems like Unit testing really is NOT popular at all at Apple. I mean, they release a framework for storage, and they don’t even unit test security related functions ? Tdd introduced the wrong idea that unit tests should be all or nothing. I think it’s not. I unit test only the most critical parts of my programs (and only if there are), and i see value in it.
- nicolasd 9y agoSerious question: How would you suggest to write a test that prevents this? You wouldn't make an assertion on clearTextPassword !== passwordHint. While developing I would think "Who will ever do that? That would be insane". But yes, even if there is no test, this should have been caught in code review or latest when testing the OS.
- maccard 9y agoI noticed when I upgraded that they block you from setting your account password to the same as your iCloud password, so they've obviously considered that people will do stupid things.
- bsaul 9y agoIt seems like the error was in the creation of an intermediate data structure ( minimal dictionary something) in a storageKit framework. I’m not talking about the disk util app ( which doesn’t contain any bug in itself apparently) To give a better answer : I’m not knowledgeable of the whole source for the framework, but there are only two cases : - either the outputs of your critical function are testable and then just test it. - or they’re not ( most often because of side effects), in which case you should extract the critical part in purer function, and test that. It’s a great benefit of unit test : they force you to isolate side effects from business logic, to be able to test the latter. In that case, if the function isn’t testable, then maybe the creation of this intermediate minimal dictionary should belong in its own function that just does the data mapping.
- alextooter 9y agoI have developed a UI test automation software years ago, it should be getHintFromUI() == passwordHint.Here getHintFromUI is a function which get string from User Interface directly.
- danielsamuels 9y ago> did the QA engineer test setting a password and a password hint?, easily forgettable on a tight deadline Or they used 'asd' as both the password and the password hint and therefore it looked ok.
- sschueller 9y agoWhat kind of amateur uses the same entry for two different fields? In the early days of coding your entries going into the wrong place is something that will happen to you.
- justinjlynn 9y agowhat kind of amateur does manual testing of basic functionality?
- colinzoy 9y ago1. to me the most important question is: why on earth can the disk password be retrieved in clear text in the first place ?! 2. also: the buggy version will be able to show disk passwords forever, until the encryption scheme is changed. macOS native encryption is useless until then (but given 1., it might already have been for some time).
- a_t48 9y ago(1) is the bug - you aren't supposed to be able to. the password was copied into a cleartext hint field. (2) you should be fine if you reset your password and hint.
- chaosite 9y agoIf I understand the blog post correctly, the bug was in the software that creates (or maybe mounts? Not too clear about that) the encrypted volume. It needs to have the clear-text password in order to function. Then it proceeded to set the clear-text password as a password hint due to the bug.
- deleted 9y ago[deleted]
- sitharus 9y ago1. The disk password cannot be retrieved. The bug occurred when creating a new encrypted volume - the password was copied in to the hint field as well as the password field. 2. Changing your password with the updated disk utility will fix the issue.
- deleted 9y ago[deleted]
- jsjohnst 9y agoThe answer to your question was shown in code and also explicitly stated in English in the article. Edit: removed the “did you read the article?” part
- danra 9y agoIt's pretty insane this sort of bug either passed or didn't even go through code review and testing at Apple, a company which has approximately infinite resources and whose marketing pitch is making high-quality products. I loved Apple products and still do, but as both a user of and a developer for their systems, I feel the quality has been steadily going downhill the last few years.
- rusk 9y ago> whose marketing pitch is making high-quality products "It just works" was retired some time ago. "Think different" is the modern day mantra. It's less of an open-ended commitment.
- rangibaby 9y agoI remember the Think different posters from when I was a kid. As far as I can tell "It just works" is from the PC vs Mac tv ads from about ten years ago.
- rusk 9y agoPerhaps you're right, but the comparative prevalence of these phrases does seem to correspond to largely different customer experiences.
- tonyedgecombe 9y agoIt's less of an open-ended commitment. Biting.
- toyg 9y ago> a company which has approximately infinite resources I believe the leadership insists in keeping up a pretension that this is not actually the case. To be fair, most of that "infinity" is sitting in offshore bank accounts and can only be used as leverage for borrowing rather than being spent directly.
- gm-conspiracy 9y ago
- tempodox 9y agoInstalling a new Mac OS on a machine you care for has become an absolute no-go area. I wait for at least half a year and then only install in a place that I can trash in case I still see problems.
- javabean22 9y ago> at least half a year oh yeah half a year, really? oh shut the fuck up. fuck off.
- sctb 9y agoWe've asked you to please stop this already, so we've banned the account. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- rjzzleep 9y agoI'm not sure why you say "has become". In the circles I've been it has been common practice to wait for the .2(but minimum .1) release before it goes onto real work machines. The early Mac OS X releases were even worse. Even if for some reason the OS itself was fine by itself, they almost always broke compat with a bunch of 3rd party applications. EDIT: typo
- devdoomari 9y agodoes this mean that all passwords for osx-encrypted-drives have been 'recoverable'? e.g. if I created an encrypted drive using El Capitan, someone else can crack my drive's password without even cracking a password-hash? Or is there a bug also in high-sierra's 'create-encrypted-disk' functionality? (but not in lower-versions)
- futurix 9y agoOnly in the passwords created using GUI version of the new Disk Utility (command line is safe).
- devdoomari 9y agothank you! thought I was screwed...
- arghwhat 9y agoThe bug would be that for encrypted APFS volumes made using Disk Utility and similar applications, the password hint was accidentally set as the password itself. High Sierra is the first OS X release with APFS.
- blinkingled 9y ago> Apparently, Disk Utility and command line diskutil use different code paths. StorageKit does not appear as a direct dependency of diskutil. [snip] This duplication in what’s more or less the same functionality, while sometimes justified, certainly increases the opportunity for bugs. To me this is the more problematic part - good design would use same code paths as much as possible for the GUI app and the command line one - the UI code in this case will differ but there should really be no need for diskutil and Disk Utility to use duplicate code for storage functions.
- rtpg 9y agoYeah it seems bonkers to me that an OS would ship with two sets of disk management libraries. At worst, at least try to make one depend on the other.
- pducks32 9y agoI can’t believe I’m actually arguing this but devils advocate I guess: in this instance if they both used the same library then the command line would also have this problem. In this case at least there was a work around in that you could use the cli.
- throwanem 9y agoLately, every new macOS release displays less in the way of good design.
- burntcaramel 9y agoI imagine StorageKit is a relatively new addition, or is tailored for UI applications. So I can see why it is not used by diskutil (which has been there from the early days of OS X), at least not yet.
- blinkingled 9y agoUltimately though it comes down to how much effort Apple is willing to put to get things right - doesn't look like that's much in case of macOS.
- intellix 9y agothe good thing is, you know that when they break something, it'll probably stay broken for a couple more releases and a few years. Like how my mouse Y axis is reversed when I restart... for about 2 years now
- wtallis 9y agoThat sounds like a really interesting bug, but also like your mouse is going far beyond standard USB HID in some very unnecessary way.
- andai 9y agoI had a mouse that did just that. Nothing I tried could fix it. I gave up and took the batteries out, and forgot about it for a few weeks. When I tried it again it worked, and works fine ever since.
- dep_b 9y agoI don't like the old style NSDictionary way of packing information that so many API's use. First of all they're lacking an enforcement for the type of the value they're expecting (sometimes leading to crashes if you get them wrong) and the discoverability of possibilities is less easy than with a struct or object. Still, passing in password twice would still be a possible mistake. The only thing that would help in my opinion would be types that can be applied to primitives, a bit like F# does. So you can declare certain Doubles to be Miles, Kilometers, Meters and so on. Or a certain type of string can be of type DatabaseID or a Password.
- pducks32 9y agoYea I was thinking about this too! We can talk all day about static typing vs productivity but if they had a basic string type called PasswordHint and Password then maybe (maybe!) this wouldn’t happen.
- UnoriginalGuy 9y agoI'd leave passwordHint as a string; but password... That might be an interesting type. Particularly if it zeroed out memory as a freebie in the destructor. Actually that was my take-away from this article, the "maybe in MY projects I should wrap passwords in a special type to stop me typo-ing the raw password to output."
- ForFreedom 9y agoSo, is it safe to install High Seirra since it was a low sierra couple of days ago?
- vbezhenar 9y agoWhy they are duplicating StorageKit code? If diskutil was using StorageKit, it's more likely that someone would find this bug (because apparently everyone in Apple uses diskutil instead of GUI).
- abalone 9y agoSo the lesson here is, it’s a good idea to try to have your GUI and command line use the same code paths. Looks like they created a separate API in StorageKit just for Disk Utility and got sloppy with unit test.
- jagger27 9y agoWhat's interesting to me is that the version of Disk Utility in the High Sierra installer doesn't have this issue.
- feelin_googley 9y ago"This research has shone a light on some of the ways in which security patch support provided by Apple for 'firmware' is quite different than the security patch support they provide for 'software'." "You can be software 'secure' but firmware vulnerable." "The release QA on the FirmwareUpdate bundles is concerning." https://duo.com/assets/ebooks/Duo-Labs-The-Apple-of-Your-EFI.pdf https://duo.com/assets/ebooks/Duo-Labs-The-Apple-of-Your-EFI... "The advent of UEFI brought with it a far more 'modern' pre-boot environment and 'finally' put an end to the many years of legacy workarounds that had to be applied to the aging IBM BIOS 'standard', providing a common, uniform and higher-level platform to 'innovate' on." "However, that uniformity and accessibility also opened the door to far more generic and useful pre-boot environment attack opportunities."
- SkyMarshal 9y agoOff topic but related, anyone know of any active OS X hacking/tweaking forums or communities? I have an I/O issue with a peripheral I'm trying to isolate, could use some expert help.
- sethhochberg 9y agowww.tonymacx86.com is where a lot of the hackintosh community gathers. Plenty of folks in the forums there who understand the kernel / IO / driver stack.
- rconti 9y agoThe Supplemental Update took ages to install on my SSD-equipped 5k iMac (it's only about a year old). In fact, I'm quite sure it took longer to install than High Sierra itself, which is quite an achievement because the High Sierra install involved upgrading my filesystem to APFS! Maybe I just found myself more annoyed than usual because I clicked "try later tonight" for the supplemental update prompt when using the computer the day before, and then when I tried to use the computer in the morning, it went about its installing business which featured 2 backwards-moving progress bars and took at least 20 minutes, while I sat around twiddling my thumbs waiting to use my computer for what should have been a simple task. I'm not sure if this is normal behavior when you postpone an update and then wake from sleep later, but I've never seen what effectively turned into an un-prompted forced install before.
- mark_l_watson 9y agoSorry if this is off topic, but I ran into another serious bug in High Sierra that I think is a result of their de-duplicating of files feature: I had about 30 GB of family videos and pictures on OneDrive and wanted a second copy on Dropbox. As I copied them I noticed that the disk space did not decrease, a good thing to have file de-duplication. All was well for a long while until I decided to have OneDrive and DropBox not keep my family videos and pictures on my laptop. After DropBox and OneDrive deleted these files, I never got the 30 GB of free space back. Apple should provide an option in the Disk Utility to check for files that don’t have references and free the space.