9 ms·

Great writeup, very succinct and informative, they even have a simple reproduction of the bug. I love Mullvad! Tangentially, MacOS has had a lot of weird fire

Great writeup, very succinct and informative, they even have a simple reproduction of the bug. I love Mullvad! Tangentially, MacOS has had a lot of weird firewall bugs in the last few releases in general, I wonder what drive them to rip up and redo (I assume? so much of it recently.

nhubbard 3y ago
The rewrite was definitely influenced by the mandatory migration from kernel extensions to userspace System Extensions, specifically NetworkExtension, between Catalina and Big Sur: https://developer.apple.com/documentation/networkextension https://developer.apple.com/documentation/networkextension
- hulitu 3y ago
  One would expect they have a test suite.
  - dclowd9901 3y ago
    Oh you poor sweet summer child
  - londons_explore 3y ago
    One would suspect their test suite is lacking...
    - MichaelZuo 3y ago
      There are a lot of errors and faults that show up in Console on a brand new MacBook just sitting on the desktop. And with every version the number seems to increase. So it's not even their test suite that's the issue.
      - dmix 3y ago
        I'm curious, why do you look at random OS errors in Console to the point you noticed such a thing?
        MichaelZuo 3y ago
        When my computer can't even stay up for a full week without crashing, or suffer an inexplicable lag spike, etc., I become more motivated to closely examine everything to see what's causing the crash.
        callalex 3y ago
        Are you plugging/unplugging monitors or other hardware? With macs it’s always that for some reason.
        MichaelZuo 3y ago
        No, literally just sitting there with no peripherals attached, doing nothing more complex then playing a 4k video and web browsing.
        astrange 3y ago
        File a bug in Feedback Assistant, if you want.
        MichaelZuo 3y ago
        Okay, so?
        Moomoomoo309 3y ago
        If there's an issue, how are they supposed to know about it? Via osmosis? Even if they should have found it earlier, they didn't, so now what? You should tell them, so they can fix it.
      - nvy 3y ago
        You mean from dmesg or something? Not once have I opened console.app and seen random OS errors just sitting at the shell prompt. edit: derp I confused console/terminal please disregard
        dbmnt 3y ago
        No he means from the Console app on macOS. It's not the same as the Terminal app, which it sounds like you're referencing. It's a log viewer that's installed by default but tucked away in Applications/Utilities. It's similar to dmesg I guess.
        nvy 3y ago
        Yes, major derp moment there.
      - tambourine_man 3y ago
        Apple made logging “cheap” in Mavericks, IIRC. They made a big deal that by coalescing and compressing writes, using a database instead of flat text files, it would use little memory and battery, or something like that. Anyway, Console.app got way more verbose after that. I don’t like it, it’s too noise for me, but it doesn’t necessarily mean that lots of things are breaking under the hood.
        newaccount74 3y ago
        They added a way to efficiently generate hundreds of log messages per second, but never bothered to think about what to do with all these messages.
  - lIIllIIllIIllII 3y ago
    For a firewall I'd ideally want some degree of formal verification. It's not something you fuck around with if you're serious about creating a secure product - like an operating system.
    - thghtihadanacct 3y ago
      [flagged]
    - rcme 3y ago
      But how do you verify the formal verification?
      - mejutoco 3y ago
        Proving in Math that the building blocks have certain properties.
        tialaramex 3y ago
        While this is worthwhile, there is an opportunity for the math to violate your assumptions in a way you didn't understand. Let's talk briefly about TLS 1.3 Selfie attacks. TLS 1.3 is the first TLS in which experts built a formally verified model of the protocol before they shipped the standard. There are models of earlier TLS versions, but nothing learned from them could be incorporated into the corresponding standard because they were an afterthought - like writing unit tests after shipping version 1.0 TLS 1.3 has a mode intended for applications where some nodes which have a pre-existing relationship communicate, in this case we don't need the Web PKI ("SSL certificates") instead we use Pre-shared Keys (PSKs) - that is all the parties know the keys anyway, which clearly couldn't work for the open web but is fine for my boiler and its remote thermostat since duh, I don't want some random other person's thermostat talking to my boiler. PSKs are also used when you connect to the same server again later, but that doesn't matter here. The mathematical proof seemed to say exactly what the designers wanted, and it passed. So TLS 1.3 is exactly what we wanted... right? Well, almost. Designers were thinking of symmetries like "Alice talks to Bob" and "Bob talks to Alice" as one conversation, but the proof thinks that's two conversations. So the proof thinks Alice and Bob need two keys, the Alice->Bob key and the Bob->Alice key, while humans assumed they only need a single key, an Alice-x-Bob key. And so the RFC documents (prior to errata) the human assumption but the protocol actually requires the machine assumption. The result is the Selfie Attack. In our scenario Alice, and Bob have a Cat. The Cat, like many cats, would like more food, but Alice and Bob use a secure protocol to ensure they only feed the cat once. Bob has fed the cat and left. The cat is sat by an empty food bowl looking plaintive. Alice sends Bob an encrypted message, "Did you feed the cat?". The Cat intercepts the message, it doesn't have the Alice-x-Bob key so it can't decrypt this message nor directly answer it, but it doesn't need to. The Cat simply sends Alice's own message back to her. Receiving a message encrypted with the Alice-x-Bob key, Alice concludes it's from Bob. "Did you feed the cat?". So she replies "No, I did not feed the cat". The cat receives this message too, and directs it back to Alice as well, now Alice has what appears to be a reply to her first question, "No, I did not feed the cat" encrypted with the Alice-x-Bob key. So, Alice feeds the cat again because the cat was able to trick her into answering her own question.
  - skhr0680 3y ago
    from my past experience with Mac OS, the users of the .0 version are the test suite
    - rjzzleep 3y ago
      I think this pretty accurate. After messing around with the .0 versions of macOS a few times I also learned that the common wisdom is to only install OS updates on .1 versions.
      - detourdog 3y ago
        It has been true since the NeXT days. .3 is for speed. Anything above .3 is a usually a bug fix.
isodev 3y ago
It would have been even better if they had included the rdar/Feedback number.
voytec 3y ago
[flagged]
- jxf 3y ago
  If Apple is shipping that implementation as part of their OS, doesn't it make sense to let Apple know they should pick a different upstream target?
  - jojobas 3y ago
    No, you see grasshopper, Apple is infallible. If there's a problem with any of their devices or software, it's always someone else's fault. /s
- eptcyka 3y ago
  To the best of my knowledge, OpenBSD's pf wouldn't exhibit such pathological behavior.
  - voytec 3y ago
    Exactly - it's a screwup on Apple's part, not OpenBSD's.
    - alpaca128 3y ago
      And Mullvad very clearly indicated it's connected to Apple/macOS: > bug in the macOS firewall, packet filter (PF)
      - voytec 3y ago
        It's not macOS firewall, but Apple's implementation of OpenBSD's pf used in Apple's macOS. Mullvad is clearly pointing at a bug in OpenBSD's "packet filter", mentioning that it's used in macOS. Mullvad's article lacks proper wording and shits on the wrong target.
        3y ago
tiffanyh 3y ago
macOS has attempted to progress its networking stack for years but would run into regressions and then revert back. Old article on the topic. https://9to5mac.com/2015/05/26/apple-drops-discoveryd-in-latest-os-x-beta-following-months-of-complaints-about-network-issues-with-yosemite/ https://9to5mac.com/2015/05/26/apple-drops-discoveryd-in-lat...
- cptcobalt 3y ago
  Drudging up 8 year old architectural decisions that Apple rightfully reverted is hardly a charitable comment. A bug can just be a bug.
keepamovin 3y ago
If you're concerned about these kinds of bugs on your local OS platform you may consider "abstracting away" your local connection point via a remote browser. This way, whatever your local machine and OS, you can have a dedicated server that you run your browsing through. Granted it doesn't enclose your entire network connection: only your browsing, but what it does there is change your IP address, mask your location, and add protection from browser 0 days. We're constantly adding new features add BrowserBox to respect and protect privacy and improve the overall experience. It's open source so you can change it how you want too. If you don't like AGPL-3.0 you can get a commercial license. Come take us for a spin: https://github.com/dosyago/BrowserBoxPro https://github.com/dosyago/BrowserBoxPro If you don't want something open source, but prefer the joy of a large company I think Mullvad also has their Mullvad Browser which does something similar!
- scarmig 3y ago
  I've always imagined RBI as something primarily targeting enterprise; thanks for making it clearer the advantages for a personal end user. Do many websites end up blocking traffic if it's originating from e.g. an EC2 instance?
  - keepamovin 3y ago
    Yeah, thank you. It does actually have use cases for regular folk! About the IP: No actually. I'm not sure why that is, but I assume because a lot of premium VPN services and proxies need to end up going through IP blocks used by cloud providers as well these days. Most of the blocking I've seen comes down to how well the browser presents as a regular browser. If it looks like an automated bot browser (just raw headless, or whatever) it's more likely to get blocked in my experience than from the IP address. Although you certainly get some IP blocks or individual IPs that are more likely to get blocked. No offence to DO, but I've noticed their IPs are less reliable. I've never really had an issue with AWS, GCP, Vultr, Linode or Hetzner, except for the odd machine that somehow has an IP address that must be on a ban list, I'd estimate it around 5% probability of getting such an IP from the regular providers tho.

nhubbard 3y ago

The rewrite was definitely influenced by the mandatory migration from kernel extensions to userspace System Extensions, specifically NetworkExtension, between Catalina and Big Sur: https://developer.apple.com/documentation/networkextension https://developer.apple.com/documentation/networkextension

hulitu 3y ago
One would expect they have a test suite.
- dclowd9901 3y ago
  Oh you poor sweet summer child
- londons_explore 3y ago
  One would suspect their test suite is lacking...
  - MichaelZuo 3y ago
    There are a lot of errors and faults that show up in Console on a brand new MacBook just sitting on the desktop. And with every version the number seems to increase. So it's not even their test suite that's the issue.
    - dmix 3y ago
      I'm curious, why do you look at random OS errors in Console to the point you noticed such a thing?
      - MichaelZuo 3y ago
        When my computer can't even stay up for a full week without crashing, or suffer an inexplicable lag spike, etc., I become more motivated to closely examine everything to see what's causing the crash.
        callalex 3y ago
        Are you plugging/unplugging monitors or other hardware? With macs it’s always that for some reason.
        MichaelZuo 3y ago
        No, literally just sitting there with no peripherals attached, doing nothing more complex then playing a 4k video and web browsing.
        astrange 3y ago
        File a bug in Feedback Assistant, if you want.
        MichaelZuo 3y ago
        Okay, so?
        Moomoomoo309 3y ago
        If there's an issue, how are they supposed to know about it? Via osmosis? Even if they should have found it earlier, they didn't, so now what? You should tell them, so they can fix it.
    - nvy 3y ago
      You mean from dmesg or something? Not once have I opened console.app and seen random OS errors just sitting at the shell prompt. edit: derp I confused console/terminal please disregard
      - dbmnt 3y ago
        No he means from the Console app on macOS. It's not the same as the Terminal app, which it sounds like you're referencing. It's a log viewer that's installed by default but tucked away in Applications/Utilities. It's similar to dmesg I guess.
        nvy 3y ago
        Yes, major derp moment there.
    - tambourine_man 3y ago
      Apple made logging “cheap” in Mavericks, IIRC. They made a big deal that by coalescing and compressing writes, using a database instead of flat text files, it would use little memory and battery, or something like that. Anyway, Console.app got way more verbose after that. I don’t like it, it’s too noise for me, but it doesn’t necessarily mean that lots of things are breaking under the hood.
      - newaccount74 3y ago
        They added a way to efficiently generate hundreds of log messages per second, but never bothered to think about what to do with all these messages.
- lIIllIIllIIllII 3y ago
  For a firewall I'd ideally want some degree of formal verification. It's not something you fuck around with if you're serious about creating a secure product - like an operating system.
  - thghtihadanacct 3y ago
    [flagged]
  - rcme 3y ago
    But how do you verify the formal verification?
    - mejutoco 3y ago
      Proving in Math that the building blocks have certain properties.
      - tialaramex 3y ago
        While this is worthwhile, there is an opportunity for the math to violate your assumptions in a way you didn't understand. Let's talk briefly about TLS 1.3 Selfie attacks. TLS 1.3 is the first TLS in which experts built a formally verified model of the protocol before they shipped the standard. There are models of earlier TLS versions, but nothing learned from them could be incorporated into the corresponding standard because they were an afterthought - like writing unit tests after shipping version 1.0 TLS 1.3 has a mode intended for applications where some nodes which have a pre-existing relationship communicate, in this case we don't need the Web PKI ("SSL certificates") instead we use Pre-shared Keys (PSKs) - that is all the parties know the keys anyway, which clearly couldn't work for the open web but is fine for my boiler and its remote thermostat since duh, I don't want some random other person's thermostat talking to my boiler. PSKs are also used when you connect to the same server again later, but that doesn't matter here. The mathematical proof seemed to say exactly what the designers wanted, and it passed. So TLS 1.3 is exactly what we wanted... right? Well, almost. Designers were thinking of symmetries like "Alice talks to Bob" and "Bob talks to Alice" as one conversation, but the proof thinks that's two conversations. So the proof thinks Alice and Bob need two keys, the Alice->Bob key and the Bob->Alice key, while humans assumed they only need a single key, an Alice-x-Bob key. And so the RFC documents (prior to errata) the human assumption but the protocol actually requires the machine assumption. The result is the Selfie Attack. In our scenario Alice, and Bob have a Cat. The Cat, like many cats, would like more food, but Alice and Bob use a secure protocol to ensure they only feed the cat once. Bob has fed the cat and left. The cat is sat by an empty food bowl looking plaintive. Alice sends Bob an encrypted message, "Did you feed the cat?". The Cat intercepts the message, it doesn't have the Alice-x-Bob key so it can't decrypt this message nor directly answer it, but it doesn't need to. The Cat simply sends Alice's own message back to her. Receiving a message encrypted with the Alice-x-Bob key, Alice concludes it's from Bob. "Did you feed the cat?". So she replies "No, I did not feed the cat". The cat receives this message too, and directs it back to Alice as well, now Alice has what appears to be a reply to her first question, "No, I did not feed the cat" encrypted with the Alice-x-Bob key. So, Alice feeds the cat again because the cat was able to trick her into answering her own question.
- skhr0680 3y ago
  from my past experience with Mac OS, the users of the .0 version are the test suite
  - rjzzleep 3y ago
    I think this pretty accurate. After messing around with the .0 versions of macOS a few times I also learned that the common wisdom is to only install OS updates on .1 versions.
    - detourdog 3y ago
      It has been true since the NeXT days. .3 is for speed. Anything above .3 is a usually a bug fix.

isodev 3y ago

It would have been even better if they had included the rdar/Feedback number.

voytec 3y ago

[flagged]

jxf 3y ago
If Apple is shipping that implementation as part of their OS, doesn't it make sense to let Apple know they should pick a different upstream target?
- jojobas 3y ago
  No, you see grasshopper, Apple is infallible. If there's a problem with any of their devices or software, it's always someone else's fault. /s
eptcyka 3y ago
To the best of my knowledge, OpenBSD's pf wouldn't exhibit such pathological behavior.
- voytec 3y ago
  Exactly - it's a screwup on Apple's part, not OpenBSD's.
  - alpaca128 3y ago
    And Mullvad very clearly indicated it's connected to Apple/macOS: > bug in the macOS firewall, packet filter (PF)
    - voytec 3y ago
      It's not macOS firewall, but Apple's implementation of OpenBSD's pf used in Apple's macOS. Mullvad is clearly pointing at a bug in OpenBSD's "packet filter", mentioning that it's used in macOS. Mullvad's article lacks proper wording and shits on the wrong target.
      - 3y ago

tiffanyh 3y ago

macOS has attempted to progress its networking stack for years but would run into regressions and then revert back. Old article on the topic. https://9to5mac.com/2015/05/26/apple-drops-discoveryd-in-latest-os-x-beta-following-months-of-complaints-about-network-issues-with-yosemite/ https://9to5mac.com/2015/05/26/apple-drops-discoveryd-in-lat...

cptcobalt 3y ago
Drudging up 8 year old architectural decisions that Apple rightfully reverted is hardly a charitable comment. A bug can just be a bug.

keepamovin 3y ago

If you're concerned about these kinds of bugs on your local OS platform you may consider "abstracting away" your local connection point via a remote browser. This way, whatever your local machine and OS, you can have a dedicated server that you run your browsing through. Granted it doesn't enclose your entire network connection: only your browsing, but what it does there is change your IP address, mask your location, and add protection from browser 0 days. We're constantly adding new features add BrowserBox to respect and protect privacy and improve the overall experience. It's open source so you can change it how you want too. If you don't like AGPL-3.0 you can get a commercial license. Come take us for a spin: https://github.com/dosyago/BrowserBoxPro https://github.com/dosyago/BrowserBoxPro If you don't want something open source, but prefer the joy of a large company I think Mullvad also has their Mullvad Browser which does something similar!

scarmig 3y ago
I've always imagined RBI as something primarily targeting enterprise; thanks for making it clearer the advantages for a personal end user. Do many websites end up blocking traffic if it's originating from e.g. an EC2 instance?
- keepamovin 3y ago
  Yeah, thank you. It does actually have use cases for regular folk! About the IP: No actually. I'm not sure why that is, but I assume because a lot of premium VPN services and proxies need to end up going through IP blocks used by cloud providers as well these days. Most of the blocking I've seen comes down to how well the browser presents as a regular browser. If it looks like an automated bot browser (just raw headless, or whatever) it's more likely to get blocked in my experience than from the IP address. Although you certainly get some IP blocks or individual IPs that are more likely to get blocked. No offence to DO, but I've noticed their IPs are less reliable. I've never really had an issue with AWS, GCP, Vultr, Linode or Hetzner, except for the odd machine that somehow has an IP address that must be on a ban list, I'd estimate it around 5% probability of getting such an IP from the regular providers tho.