7 ms·

> We really should not run different security domains on different processor threads of the same core. Unfortunately changing our scheduler to take this into a

> We really should not run different security domains on different processor threads of the same core. Unfortunately changing our scheduler to take this into account is far from trivial. This suggests a long-term compromise solution where threads within a process can use hyperthreading to share a core, but threads in different processes can't. Given that hyperthreads share L1 cache, this might also be better for performance.

classichasclass 8y ago
I'm not sure that would necessarily fix the problem definitively. Say you had a browser running web-exposed JavaScript on a thread. You could still finagle a Spectre-type information leak that way by having the JavaScript thread snoop other browser threads, assuming no other mitigations.
- endianswap 8y ago
  Don't most browsers run one process per page/tab nowadays?
  - greglindahl 8y ago
    Chrome does, Firefox does not (I've got 5 processes for a billion tabs.)
    - valarauca1 8y ago
      Firefox process per tab is behind a feature flag as it’s in testing still
      - timvisee 8y ago
        I don't think the plan is to ever enable this in the comming few years. The current approach with a few tabs is much more memory efficient, which is why they've chosen it.
        ams6110 8y ago
        Indeed. I cannot open Google Drive in chrome on my OpenBSD box without crashing the tab from exhausting memory, but Firefox handles it no problem.
        mtgx 8y ago
        And it's a mistake. Just recently I noticed that when Firefox loads multiple tabs of the same wordpress site, it starts hanging not unlike Firefox always used to hang. That's likely because it groups all of those same site pages under one process. I've never experienced that with Chrome. This is why I hope Firefox eventually (ASAP) switches to one process per tab, too. I can handle the browser using an extra GB of RAM. I can't handle it hanging on me and frustrating me. Instead of pushing for 30-40% lower memory than Chrome, I say they should push for 10% lower memory with the same sanboxed process per tab model.
        greglindahl 8y ago
        So, you're saying because you think you've discovered one case where there might be a problem, Firefox should completely change their architecture? And you're saying this in a discussion which frequently mentions how extremely varied workloads are?
        abugheratwork 8y ago
        No, man. He said Firefox should change their architecture, and he gave some kind of example. After the way you seized on the word "typical", I kind of expected you to take words at face value. I didn't see any text to the effect that he thinks his say so is good enough. Also, you're the one frequently mentioning how varied workloads are, and you don't constitute the discussion. I'm going to go drink some cocoa to wash down this hook, line, and sinker I swallowed with your flame bait.
        greglindahl 8y ago
        Not intended as flame bait, sorry if that's how it comes across to you. It just so happens that people complaining about Firefox doing it wrong is a pretty common thing in Firefox threads on HN. And they usually have an example where it's really unclear if it's a problem for more than them or not. But, usually, they have a lot of advice about what the Firefox team should do. Whereas the Firefox team has telemetry data from most of their users.
      - craftyguy 8y ago
        The only thing I could find is a set of 'browser.tabs.remote.*' options, that are all enabled by default in FF 60. That seems to indicate it is enabled, since the old option was 'browser.tabs.remote'? Or has it changed to something else now?
    - ComputerGuru 8y ago
      No, Chrome used to but now uses a heuristic to determine whether new tabs should be launched in their own process or share an existing process, as a memory usage mitigation strategy. I believe tabs from the same origin have a preference of sharing processes.
- stormbrew 8y ago
  In theory marking threads even within the same process as part of a different 'security domain' shouldn't be impossible, though obviously it'd involve proprietary interfaces to the kernel at first.
  - krylon 8y ago
    .Net supports this (at least on Windows), it is called AppDomains: https://docs.microsoft.com/en-us/dotnet/framework/app-domains/application-domains https://docs.microsoft.com/en-us/dotnet/framework/app-domain...
- Tepix 8y ago
  Once operating systems offered this mitigation mechanism, I'm sure browser vendors would use them.
mww09 8y ago
>This suggests a long-term compromise solution where threads within a process can use hyperthreading to share a core, but threads in different processes can't. Given that hyperthreads share L1 cache, this might also be better for performance. Intuitively this may sound logical, however in practice it's often not the case. For many workloads putting two threads of the same program on a core ends up being worse than co-locating with threads from different programs. The reason is that two threads of the same program will often end up executing similar instruction streams (a really good example is when both are using vector instructions (these registers are shared between the two hyperthreads)).
- amelius 8y ago
  > The reason is that two threads of the same program will often end up executing similar instruction streams Why is that bad?
  - xeroaura 8y ago
    Taking a guess, but since they are running similar streams, they have similar loads at a specific time. Competition between main thread and hyperthread could hurt performance instead?
  - uryga 8y ago
    "both are using vector instructions (these registers are shared between the two hyperthreads)" So I guessing GP meant there's going to be contention for those registers, and thus no speedup?
  - jfoutz 8y ago
    Different instruction streams use different registers. It’s like sharing a bathroom. I can shower while you brush your teeth. There’s more contention when we’re both trying to shower.
  - CHY872 8y ago
    Your processor has a certain number of execution units which can actually execute individual instructions, maybe like 4 floating point units, maybe 8 arithmetic ones, and maybe 1 that can do vector processing (these numbers are not real, but are like, good enough for sake of message). So the idea with SMT is that most of the time, lots of the execution units are unused because the thread a) isn't using them at all (e.g. a process to do encryption won't use the floating point units) and/or b) can't use them all because of how the program's written (for example, if I say 'load a random memory address, then add it to a register, then load another random memory address, then add it, etc' I'm going to be spending most of my time waiting for memory to be loaded. SMT basically means that you run another program at the same time, so even if the encryption process can't use the floating point units, maybe there's another process that we can schedule that will. However, imagine my encryption process can use 6 of the 8 arithmetic units. If I have 2 encryption processes scheduled on the same core, I have demand for 12 when there are only 8. So now I have contention for resources, and I won't see a speedup from using SMT. Other comments mention registers and not execution units: I'm suspicious of this, since modern processors have many registers (for Skylake, 250+) which they remap between aggressively as part of pipelining. Maybe this is different for the SIMD units. That said, I haven't looked at this stuff since university so could well be wrong on the execution unit vs register comparison.
the8472 8y ago
> Given that hyperthreads share L1 cache, this might also be better for performance. If userspace thread writes something into a buffer, does some syscall initiating asynchronous work in the kernel wouldn't it be better for the kernel thread to be located on the same core instead of shuffling the data into another cache?
chasil 8y ago
Perhaps it makes more sense to require that all processes on an individual core share the same UID. Browsers are particularly problematic, and it would be nice to alert the scheduler that a particular process is untrusted and extra care should be taken to sanitize caches before and after its time slice.

classichasclass 8y ago

I'm not sure that would necessarily fix the problem definitively. Say you had a browser running web-exposed JavaScript on a thread. You could still finagle a Spectre-type information leak that way by having the JavaScript thread snoop other browser threads, assuming no other mitigations.

endianswap 8y ago
Don't most browsers run one process per page/tab nowadays?
- greglindahl 8y ago
  Chrome does, Firefox does not (I've got 5 processes for a billion tabs.)
  - valarauca1 8y ago
    Firefox process per tab is behind a feature flag as it’s in testing still
    - timvisee 8y ago
      I don't think the plan is to ever enable this in the comming few years. The current approach with a few tabs is much more memory efficient, which is why they've chosen it.
      - ams6110 8y ago
        Indeed. I cannot open Google Drive in chrome on my OpenBSD box without crashing the tab from exhausting memory, but Firefox handles it no problem.
      - mtgx 8y ago
        And it's a mistake. Just recently I noticed that when Firefox loads multiple tabs of the same wordpress site, it starts hanging not unlike Firefox always used to hang. That's likely because it groups all of those same site pages under one process. I've never experienced that with Chrome. This is why I hope Firefox eventually (ASAP) switches to one process per tab, too. I can handle the browser using an extra GB of RAM. I can't handle it hanging on me and frustrating me. Instead of pushing for 30-40% lower memory than Chrome, I say they should push for 10% lower memory with the same sanboxed process per tab model.
        greglindahl 8y ago
        So, you're saying because you think you've discovered one case where there might be a problem, Firefox should completely change their architecture? And you're saying this in a discussion which frequently mentions how extremely varied workloads are?
        abugheratwork 8y ago
        No, man. He said Firefox should change their architecture, and he gave some kind of example. After the way you seized on the word "typical", I kind of expected you to take words at face value. I didn't see any text to the effect that he thinks his say so is good enough. Also, you're the one frequently mentioning how varied workloads are, and you don't constitute the discussion. I'm going to go drink some cocoa to wash down this hook, line, and sinker I swallowed with your flame bait.
        greglindahl 8y ago
        Not intended as flame bait, sorry if that's how it comes across to you. It just so happens that people complaining about Firefox doing it wrong is a pretty common thing in Firefox threads on HN. And they usually have an example where it's really unclear if it's a problem for more than them or not. But, usually, they have a lot of advice about what the Firefox team should do. Whereas the Firefox team has telemetry data from most of their users.
    - craftyguy 8y ago
      The only thing I could find is a set of 'browser.tabs.remote.*' options, that are all enabled by default in FF 60. That seems to indicate it is enabled, since the old option was 'browser.tabs.remote'? Or has it changed to something else now?
  - ComputerGuru 8y ago
    No, Chrome used to but now uses a heuristic to determine whether new tabs should be launched in their own process or share an existing process, as a memory usage mitigation strategy. I believe tabs from the same origin have a preference of sharing processes.
stormbrew 8y ago
In theory marking threads even within the same process as part of a different 'security domain' shouldn't be impossible, though obviously it'd involve proprietary interfaces to the kernel at first.
- krylon 8y ago
  .Net supports this (at least on Windows), it is called AppDomains: https://docs.microsoft.com/en-us/dotnet/framework/app-domains/application-domains https://docs.microsoft.com/en-us/dotnet/framework/app-domain...
Tepix 8y ago
Once operating systems offered this mitigation mechanism, I'm sure browser vendors would use them.

mww09 8y ago

>This suggests a long-term compromise solution where threads within a process can use hyperthreading to share a core, but threads in different processes can't. Given that hyperthreads share L1 cache, this might also be better for performance. Intuitively this may sound logical, however in practice it's often not the case. For many workloads putting two threads of the same program on a core ends up being worse than co-locating with threads from different programs. The reason is that two threads of the same program will often end up executing similar instruction streams (a really good example is when both are using vector instructions (these registers are shared between the two hyperthreads)).

amelius 8y ago
> The reason is that two threads of the same program will often end up executing similar instruction streams Why is that bad?
- xeroaura 8y ago
  Taking a guess, but since they are running similar streams, they have similar loads at a specific time. Competition between main thread and hyperthread could hurt performance instead?
- uryga 8y ago
  "both are using vector instructions (these registers are shared between the two hyperthreads)" So I guessing GP meant there's going to be contention for those registers, and thus no speedup?
- jfoutz 8y ago
  Different instruction streams use different registers. It’s like sharing a bathroom. I can shower while you brush your teeth. There’s more contention when we’re both trying to shower.
- CHY872 8y ago
  Your processor has a certain number of execution units which can actually execute individual instructions, maybe like 4 floating point units, maybe 8 arithmetic ones, and maybe 1 that can do vector processing (these numbers are not real, but are like, good enough for sake of message). So the idea with SMT is that most of the time, lots of the execution units are unused because the thread a) isn't using them at all (e.g. a process to do encryption won't use the floating point units) and/or b) can't use them all because of how the program's written (for example, if I say 'load a random memory address, then add it to a register, then load another random memory address, then add it, etc' I'm going to be spending most of my time waiting for memory to be loaded. SMT basically means that you run another program at the same time, so even if the encryption process can't use the floating point units, maybe there's another process that we can schedule that will. However, imagine my encryption process can use 6 of the 8 arithmetic units. If I have 2 encryption processes scheduled on the same core, I have demand for 12 when there are only 8. So now I have contention for resources, and I won't see a speedup from using SMT. Other comments mention registers and not execution units: I'm suspicious of this, since modern processors have many registers (for Skylake, 250+) which they remap between aggressively as part of pipelining. Maybe this is different for the SIMD units. That said, I haven't looked at this stuff since university so could well be wrong on the execution unit vs register comparison.

the8472 8y ago

> Given that hyperthreads share L1 cache, this might also be better for performance. If userspace thread writes something into a buffer, does some syscall initiating asynchronous work in the kernel wouldn't it be better for the kernel thread to be located on the same core instead of shuffling the data into another cache?

chasil 8y ago

Perhaps it makes more sense to require that all processes on an individual core share the same UID. Browsers are particularly problematic, and it would be nice to alert the scheduler that a particular process is untrusted and extra care should be taken to sanitize caches before and after its time slice.