31 ms·
Unbelievable. When I read the tweet (tried to post here as well), I suddenly realized why my Mac was unresponsive an hour ago. Here is another tweet that descr
by submeta 6y ago
Unbelievable. When I read the tweet (tried to post here as well), I suddenly realized why my Mac was unresponsive an hour ago.
Here is another tweet that describes the problem in more detail:
https://mobile.twitter.com/llanga/status/1326989724704268289 https://mobile.twitter.com/llanga/status/1326989724704268289
> I am currently unable to work because macOS sends hashes of every opened executable to some server of theirs and when `trustd` and `syspolicyd` are unable to do so, the entire operating system grinds to a halt.
EDIT:
As others pointed out, I put this to my `/etc/hosts` file and refreshed it like so:
sudo emacs /etc/hosts # add `0.0.0.0 ocsp.apple.com`
sudo dscacheutil -flushcache; sudo killall -HUP mDNSResponder # refresh hosts
- vsskanth 6y agoCan apple not use security certificates to verify publishers ? why does it need to go to their servers ?
- fpoling 6y agoThey are checking for revoked certificates.
- loeg 6y agoThe URL mentioned in sibling comments suggests this has to do with certificate revocation (OCSP): https://en.wikipedia.org/wiki/Online_Certificate_Status_Protocol https://en.wikipedia.org/wiki/Online_Certificate_Status_Prot... I agree that breaking system availability when an OCSP server isn't available is user-hostile and unnecessary.
- freeone3000 6y agoThe alternative is OCSP being allowed if internet isn't available, which is a security risk for reasonable defense-in-depth strategies.
- gruez 6y agoMost OSCP implementations fail-open, not fail-closed. I get the benefits of having it fail-closed, but it should be opt in, because having an always-online requirement for using a mac is ridiculous.
- jrochkind1 6y agoThe OP literally says if you disallow connection or unplug the intenret it does fail open. I think it's probably an unintended bug that this failure mode was fail-closed. The costs of this unintended bug are going to be huge to Apple's reputation, as demonstrated in this whole HN thread, where many assuming what's going on is even WORSE than it really is. (Personally I think having signed certs (with opt-in ability to run unsigned apps, as MacOS has) is fine. And fail-open OSCP revocation check is also fine-ish, although it would annoy me if it's making it slower to launch apps on the regular. The problem here is a bug, not one of design. But most of this thread is assuming Apple was doing something different than this. Of course, how often a company produces fairly catastrophic bugs is also on them).
- closeparen 6y agoIf your Mac is unambiguously offline it fails open. What it's handling poorly is the fail-slow case.
- bentcorner 6y agoUgh. IMO the network should not be on the critical path to running an executable.
- Spivak 6y agoMost browser vendors agree because they all stopped checking CRLs (like they technically should) when verifying certs. I don’t think the design is wrong, I just think it’s tuned a little too cautious. If you’re going to verify certs then checking the CRL is something you really should do before approval. And you can’t sync the database entirely because it’s too big. There really aren’t any good solutions to this unless you can solve the cache invalidation problem.
- valuearb 6y agoWhat’s the alternative tho?
- LgWoodenBadger 6y agoPublish revocations as security updates to the OS?
- valuearb 6y agoSecurity updates take too long. How bout each copy of MacOS keeps local copy of revocation database, and updates in background? Much faster, updates relatively quickly, and not subject to network outages.
- cromka 6y agoI'd imagine that revocations don't happen often. And when they do, Apple has a perfectly capable infrastructure to push those small incremental changes on demand. It's almost as if they intentionally ignored such superior solution and chose calling home for other reasons...
- throwaway888abc 6y agoThat way (current) Apple also has the app usage statistic ?
- jimmaswell 6y agoYou don't need an alternative. The entire concept is totally unnecessary.
- loeg 6y agoA limited change would be to fail-open more of the time, e.g., if the OCSP server does not respond within a few milliseconds. (MacOS already fails-open in some internet scenarios.) A better option is to asynchronously update a Certificate Revocation List ("CRL") and perform any check local to the machine. This avoids disclosing to Apple every single time you run a program, which program it is, and what network you're on. It could also emergency-revoke certificates just as quickly as the OCSP design by polling at the same frequency (every app startup).
- deleted 6y ago[deleted]
- LinusS1 6y agoNormally if there's no internet Gatekeeper instead checks the "stapled" notarization ticket from the notarization process. But since there is internet, and the ocsp server is technically "up" gatekeeper isn't checking the tickets.
- merb 6y agoactually I think the problem is not that it is not available, heck /etc/hosts fixes wouldn't work than. it's that it is unresponsive as hell, and they have no system wide circuit breaker, if it is slow.
- jrochkind1 6y ago> I agree that breaking system availability when an OCSP server isn't available is user-hostile and unnecessary. Based on the OP tweet... depending on the way it is unavailable, the failure is indeed ignored in some cases. "Denying that connection fixes it, because OCSP is a soft failure (Disconnect internet also fixes.)" So it may be an actual unintended bug that a particular failure path results in a DoS instead?
- deleted 6y ago[deleted]
- burlesona 6y agoIt does go locally if you are not on wifi. I thought the issue was my slow internet so I turned off wifi and suddenly everything launched just fine.
- draebek 6y agoNote that it's ocsp.apple.com, not oSCp.apple.com.
- merb 6y agodns is case insensitive
- ajford 6y agoOP was commenting on the order of the S and C
- fipar 6y agoparent is using case to highlight a typo in the domain name, not to imply that the problem is with the case.
- jolux 6y ago"oSCp".ToLower() != "oCSp".ToLower()
- deleted 6y ago[deleted]
- deleted 6y ago[deleted]
- mediaman 6y agoit's transposed, not case difference
- labster 6y agoI'm sure if the SCP Foundation gets involved in filtering our applications, they have a very good reason, like keeping Zalgo out of our reality.
- Commodore_64 6y agoI would want to see what applications foundation is hiding from us. Like a FOSS version of Windows 10
- read_if_gay_ 6y agoI started panicking mildly thinking my drive was failing or something. And just before this, I finally managed to fix Spotlight pegging one core at 100% constantly. Next thing, I reboot into a laggy system. macOS is my favorite OS, but the shit I put up with... it's basically an abusive relationship at this point.
- jrib 6y agoyeah, I had spotlight thrashing my disk too. Odd.
- rootsudo 6y agoHow interesting...Apple, couldn't, be doing a pied piper, right? /s obviously.
- SoSoRoCoCo 6y ago> macOS is my favorite OS, but the shit I put up with... Right there with ya.
- 27182818284 6y agoSame. Panic attack. Thought the SSD was dying. I ran Disk Utility diagnostics and started coming up with plans to reformat and restore as a last resort. Apple folks in this thread, this was terrible
- deleted 6y ago[deleted]
- webvictim 6y agoI genuinely thought the same thing. I opened my MBP and it was sluggish, felt like it was dead. Browser wouldn't load, Zoom wouldn't load, I rebooted and the same problems persisted. I honestly thought the hardware was giving out. I almost cannot believe the actual cause. Absolutely awful experience.
- tudelo 6y ago
- deleted 6y ago[deleted]
- charwalker 6y agoI thought this was an old issue that was known or resolved months ago. Is this still an ongoing security practice that kills devs on MacOS? This is about when I remember seeing it: https://medium.com/@acecilia/apple-is-sending-a-request-to-their-servers-for-every-piece-of-software-you-run-on-your-mac-b0bb509eee65 https://medium.com/@acecilia/apple-is-sending-a-request-to-t...
- vishesh92 6y agoFound another reason for me to not get a Mac
- miguelmota 6y agoYou can't go wrong with a ThinkPad. I switched from Mac to a T480 with Arch for dev work and it's been great.
- jagger27 6y agoIf they brought back taller displays I’d be right there with you.
- scns 6y agoCheck the article on anandtech about the new Razer laptop. Disclaimer: not affiliated.
- michalf6 6y agoThey are, next crop will be 16:10
- dotancohen 6y agoAnother poster mentioned the Huawei Matebook Pro has a 3:2 screen. I'm now looking into getting one for that reason alone.
- m4rtink 6y agoI'm running a bunch of ThinkPads with Fedora & all works fine (and worked fine for years).
- VyperCard 6y agoWe’re running Thinkpads at work with fedora and they really don’t.
- 6y ago
- jrochkind1 6y agoHuh apparently I win by still being on an old OS version?
- pgt 6y agoMy policy is to never upgrade anything until everyone I know has upgraded to the next version and not downgraded after N weeks.
- sjwright 6y agoMy policy is to upgrade my secondary/personal/low importance computer on day one and my primary computer a few weeks later.
- JKCalhoun 6y agoLOL, my policy is to never major-upgrade the OS the machine came with. I have machines around the house with OS'es going back a ways...
- trophycase 6y agoThis is the correct policy. I upgraded my mac because I couldn't install a certain application on the version I was running and now it runs crazy hot and the fans run on full blast whenever I watch a video on the internet.
- wtetzner 6y agoDepends on how old, I guess. I'm running Mojave, and ran into the problem.
- SnowflakeOnIce 6y agoRight around this same time, I had 1 macBook hard reboot (watchdogd timeout) and shortly thereafter, a second macBook froze, fan maxed out, with the display not coming up. Then it rebooted into recovery mode. Yeah, these _could_ be unrelated issues to what has been going on in Apple land today, but it's uncanny...
- dylan604 6y agoI keep reading in the tweets how all Macs are unusable. Is this an OS bug that doesn't effect older OSes? I'm on Mojave on my 2017 MBP, and have had zero issues at all. When was `trustd` introduced?
- TMWNN 6y ago`/usr/libexec/trustd` exists on Mojave, too. There's a (very unhelpful) manpage. I think you were just lucky to not open non-Apple applications during the outage.
- plorkyeran 6y agoChecking for notarization on each launch was introduced in catalina. Older versions have trustd, but it was only used for the gatekeeper checks added in 10.8.
- ericd 6y agoI ran into this on trying to load a new video file on VLC, with Mojave, so I guess it's not just apps, but maybe any new file load.
- jwineinger 6y agoMy 2018 MPB on Mojave had some serious issues launching apps for a little while yesterday (3PM central) afternoon. It seemed to resolve within an hour though. Not sure how that lines up with the outage described here.
- jbergstroem 6y agoI discovered this by running unbound – a DNS server – locally (block some unwanted hosts and do dns over TLS). I guess the rest of the story is pretty obvious; having your default dns server not being able to resolve because you're trying to verify it – since you cannot resolve your verify hostname – is obviously Not Great. As you can imagine, there is no waiting in the world that fixes this. I couldn't kill (-9) the process either; had to reboot into safe mode, rename the binary and switch the default dns on the network.
- areoform 6y agoSo yesterday I wrote about the blurring lines of ownership, and people came back with some fairly disparate responses. It's fair to say that I was mostly dismissed. https://news.ycombinator.com/item?id=25058952 https://news.ycombinator.com/item?id=25058952 And this is why I won't be moving to Apple silicon. Apple already has the ability to restrict whats apps I can run (they can simply toggle a switch for all users to "no unsigned binaries"), and congrats! Apple is the sole decider of what we get to use on our computers. Of course Apple's Craig Federighi assures us that the people making such assertions are "tools" (https://youtu.be/Hg9F1Qjv3iU?t=3177 https://youtu.be/Hg9F1Qjv3iU?t=3177 , timestamp 53:33) and they have no intention whatsoever of taking away our ability to do general compute on the machines we buy and own. Except... Apple can already decide what binaries you can execute. Should they choose to. Apple is now restricting what other OSes you can boot into. As they've chosen to. Apple can now make their machine reject a new, third-party repair part like a bad transplant. Should they choose to. It's clear where they're going. And I'm jumping ship. It's painful to do so, given how invested I am in the ecosystem, but we're already beyond the threshold that many of us would have left earlier in the decade. --- edit - It's also really hard as a designer + developer + would-be researcher in the making to find a good computer. Most non-Apple laptops don't have very good color accuracy. They also don't have good trackpads, and their keyboard + trackpad alignment is wonky (it's off-center in a lot of cases! How weird is that???) I'm trying to find a laptop with good build quality, long battery life, a good display that I can design on, a good trackpad so that I don't have to carry around a mouse, good speakers would be a plus, and light enough that I don't feel like I'm lifting weights while working on my laptop. And this package should ideally come with 512GB of SSD storage and, at least, 16GB to 32GB of RAM. Oh and it shouldn't be more expensive than a Mac as many of these laptops are! Any suggestions?
- acomjean 6y agoMy partner bought a razer 13 inch to replace a MacBook Air. It wasn’t cheap, the build quality is excellent and it handles everything (she’s in an orchestra and records her parts on it, does graphic design and sometimes plays fortnite.). The screen is quite nice and the build quality is better than my system 76 (onyx pro) which I really like too. Dave2d on YouTube gives pretty short and decent laptop reviews. I think he has a discord channel discussing the machines too
- phkahler 6y ago> I am currently unable to work because macOS sends hashes of every opened executable to some server of theirs and when `trustd` and `syspolicyd` are unable to do so, the entire operating system grinds to a halt. That's another case of a product not doing its primary function - OS running apps - because company placed their own (data gathering) objective above it. See thermostats not turning on heat when the internet connection is down and other equally stupid examples...
- octoberfranklin 6y agoSee also: all electric vehicles (except a few very old designs).
- Faaak 6y agoTesla is not all electric vehicles. My Twizy and Ioniq haven't got a single touch of data gathering neither a SIM card/wifi connectivity.
- octoberfranklin 6y agoYeah those are golf carts.
- fastball 6y agoPretty sure Apple is doing this for security reasons, not data gathering reasons.
- mister_hn 6y agoand people was shocked at Windows 10 doing telemetry. MacOS isn't doing it better as I see
- duncanc4 6y agoThere is a mistake here. It should be “ocsp.apple.com”
- playcache 6y agoHa! So that's what it was. Last night (I just woke up in the UK) my macbook pro started to crawl, I started to threat that it might be the SSD starting to fail.
- chimen 6y agoUsing a premium DNS with filtering features make sense: https://dnsadblock.com https://dnsadblock.com
- lucasverra 6y agoor https://nextdns.io/ https://nextdns.io/
- sildur 6y agoThat oscp server must be compiling a huge set of stats on application usage. That doesn't sound right, privacy-wise.
- masklinn 6y agoIt probably just gets a fingerprint, or the cert’ information. But when the endpoint is dying and it gets called every time you try to run any binary…
- Cthulhu_ 6y agoWelp, I won't be updating today then, not unless they fix that.
- antihero 6y agoThe server is called OSCP which suggests to me that if we look at Apple in the most positive light - they sign and certify binaries as safe. If an app gets later reported as malicious, they need to revoke the certificate that has been used to sign said binary. So when you open an app, how else are they going to check whether the certificate is still valid or whether it has been revoked? Can anyone confirm whether this lookup applies to unsigned as well as signed binaries? As far as I know if I build a brand new binary with cargo, and run it, it doesn't do any checks.
- habosa 6y agoHere's an idea: log all opened binaries somewhere and then every hour or so check them against the list. Never block me from opening something, but warn me about bad stuff on a regular basis.
- WhiteWestie 6y agoThey could also keep the current solution and just use a CRL as a backup to OCSP to check the revoked certificates and update it every other hour...
- antihero 6y agoYes but with your solution if an app is malicious, and did malicious things, it now has a whole hour to fuck your shit up before being disabled.
- jojobas 6y agoHere's a wild idea: don't block executables from running. Or if you do, only do it for a set of known bad ones, as antivirus products do. Do not put a cloud service (or anything for that matter) between the users and their ability to run what they want.
- antihero 6y agoSure but how does that work? If a cert-revoked app is allowed to run, the damage is already done. I think perhaps a better tradeoff would be if a revocation list could be synced hourly or so and the app could be checked sync locally and then asyncronously on open. And of course, always give the power user an option to ignore things.
- LinuxBender 6y agoWhy isn't apple doing OCSP stapling & caching? Reverse proxies have long since solved OCSP availability with stapling and caching.
- neop1x 6y agoCurrently the workaround seems to be /etc/hosts override or firewall-level blocking. Just a small reminder that this can soon stop working: Apple's apps bypass firewalls like LittleSnitch and LuLu on macOS Big Sur - https://news.ycombinator.com/item?id=24838816 https://news.ycombinator.com/item?id=24838816
- rpastuszak 6y agoThis might be a stupid question, but is there a downside to blocking this "feature"? I can't think of any. I've been using Big Sur beta for some time and one of the things that annoyed me a bit was the sudden lack of responsiveness, which is a tad annoying given that I upgraded to a 16inch MBP earlier this year and everything felt so snappy.
- Bondi_Blue 6y agoOCSP not OSCP You can also run these commands to disable ocsp (and crl) since it can no longer be accomplished in Keychain Access → Preferences: defaults write /Library/Preferences/com.apple.security.revocation.plist CRLStyle None defaults write /Library/Preferences/com.apple.security.revocation.plist OCSPStyle None defaults write com.apple.security.revocation.plist CRLStyle None defaults write com.apple.security.revocation.plist OCSPStyle None
- ummonk 6y agoI had both my personal and work laptop become unresponsive at the same time. I was wondering what kind of problem could cause that - was thinking EM interference or possibly something on my network. This explains it.
- deleted 6y ago[deleted]
- passwordreset 6y agoocsp.apple.com also has an IPv6 address. Firefox connects to it even with 0.0.0.0 in the hosts file and a flushed cache (you need to also clear firefox's internal cache if you're testing with it), so I'd assume that trustd could connect to the ocsp site as well. I don't think this will work without ensuring there is no IPv6 traffic on your network, or otherwise dumping both IPv4 and v6 packets to ocsp.apple.com. Disable IPv6: sudo networksetup -setv6off Wi-Fi (where Wi-Fi is the name of the network service)
- hxtk 6y agoCan you not just add an IPv6 entry for it in your hosts file, e.g., ::1? That would work in Linux and seems like a much less nuclear option than disabling ipv6 all together, but admittedly I've never worked with ipv6 networking on Macs. Last time I played with a Mac they also had the BSD `ipfw` command for kernel packet filtering [1]. Could try something there if it still exists. [1]: https://www.unix.com/man-page/FreeBSD/8/ipfw/ https://www.unix.com/man-page/FreeBSD/8/ipfw/
- passwordreset 6y agoJust to confirm: Yes, that works fine. It's probably the better solution here.