4 ms·

(Mill team) Afraid this answer is quite late, but hopefully it finds the curious passing through anyway :) The really big gains from splitting protection from

(Mill team) Afraid this answer is quite late, but hopefully it finds the curious passing through anyway :) The really big gains from splitting protection from translation are twofold: First, you can do the protection lookup in parallel to the cache lookup, rather than before it. This allows the PLB to be much bigger than a conventional TLB could ever be. Second, the PLB entries do not need to be fixed power-of-two sizes. Each PLB entry has a start and stop offset so protection can be as granular as protecting one byte. In general, though, a single PLB entry will be quite large, covering the whole mapped region. The Mill still has a TLB but it sits between chip and the off-chip memory. This allows it to be larger than a conventional TLB, as it taking slightly longer will not have the same impact. Additionally, as a small implementation optimisation, the TLB on the bigger Mill family members will do a lot of things like allocating-from-a-pool dynamically, so you only trap to the OS when things get dire. The Mill address has a special bit to signify if it is a global address, or whether it is "local" to the process (using a special register which is XORed into the address). This allows the Mill to fork() and all the other things that traditionally have been hard on a SASOS. We are porting Linux, and it will be Linux. Only a very very small part of the HAL will need to know anything Mill-specific :)

aidenn0 11y ago
> Second, the PLB entries do not need to be fixed power-of-two sizes. Each PLB entry has a start and stop offset so protection can be as granular as protecting one byte. In general, though, a single PLB entry will be quite large, covering the whole mapped region. This seems orthogonal to the choice of PLB vs. TLB. You could make a TLB with a start and stop offset if you really wanted to (though I suppose the increased size per entry would be problematic with TLBs needing to be faster than PLBs (which only need to be as fast as an l1 cache hit). > The Mill address has a special bit to signify if it is a global address, or whether it is "local" to the process (using a special register which is XORed into the address). This allows the Mill to fork() and all the other things that traditionally have been hard on a SASOS. Would shared-memory (in the unix sense) be considered global or local? Would read-only text sections be marked global to improve fork performance? I assume CoW is out-of-the-question for non-read-only sections and fork()? If there's a wiki page relating to fork, just point me to it :) > We are porting Linux, and it will be Linux. Only a very very small part of the HAL will need to know anything Mill-specific :) The feasibility of this seems obvious to me, as I've seen linux run other SAS platforms that are far more constrained than The Mill.
- willvarfar 11y ago
  (Mill team) Re non-paged TLB: The Mill has non-paged protection in the PLB and paged translation in the TLB. It is true that a 'conventional' could have non-paged translation, but non-paged translation would be slow. Really slow. Shudder to imagine how it would work internally :) Re SAS: Conventional machines are Multi-address-space (MAS). In a MAS, addresses can alias each another. Not so on the Mill (at least, if you deliberately alias in the Mill TLB you'll have to know that reads and writes will merge as they go off-chip etc; tricky, but can be useful in select low-level scenarios). On the Mill, everything that is shared between processes is global. On a Unix port, everything that is local to a process is 'local'. So MAP_SHARED gets addresses without the local bit set and MAP_PRIVATE does. When we fork(), we have some choices. The OS picks a new local offset for the new process, meaning that local pointers don't alias one another. We could straight away clone the protection entries and page tables. Or, as an implementation detail, depending on the extent to which the PLB self-manages on the particular Mill member, just let the PLB trap. As the new forked process accesses memory ranges for the first time the PLB will miss and there will be the classic trap. In the trap, the handler can set up COW lazily and for the memory regions that are actually touched by the child. I am simplifying it a bit ;) Most fork() is just going to exec(), and the latter will win. Some people are still using fork() in other ways, and they may benefit from an aggressive COW up front on fork(). Swings and roundabouts.
  - aidenn0 11y ago
    Thanks for your answers!

aidenn0 11y ago

> Second, the PLB entries do not need to be fixed power-of-two sizes. Each PLB entry has a start and stop offset so protection can be as granular as protecting one byte. In general, though, a single PLB entry will be quite large, covering the whole mapped region. This seems orthogonal to the choice of PLB vs. TLB. You could make a TLB with a start and stop offset if you really wanted to (though I suppose the increased size per entry would be problematic with TLBs needing to be faster than PLBs (which only need to be as fast as an l1 cache hit). > The Mill address has a special bit to signify if it is a global address, or whether it is "local" to the process (using a special register which is XORed into the address). This allows the Mill to fork() and all the other things that traditionally have been hard on a SASOS. Would shared-memory (in the unix sense) be considered global or local? Would read-only text sections be marked global to improve fork performance? I assume CoW is out-of-the-question for non-read-only sections and fork()? If there's a wiki page relating to fork, just point me to it :) > We are porting Linux, and it will be Linux. Only a very very small part of the HAL will need to know anything Mill-specific :) The feasibility of this seems obvious to me, as I've seen linux run other SAS platforms that are far more constrained than The Mill.

willvarfar 11y ago
(Mill team) Re non-paged TLB: The Mill has non-paged protection in the PLB and paged translation in the TLB. It is true that a 'conventional' could have non-paged translation, but non-paged translation would be slow. Really slow. Shudder to imagine how it would work internally :) Re SAS: Conventional machines are Multi-address-space (MAS). In a MAS, addresses can alias each another. Not so on the Mill (at least, if you deliberately alias in the Mill TLB you'll have to know that reads and writes will merge as they go off-chip etc; tricky, but can be useful in select low-level scenarios). On the Mill, everything that is shared between processes is global. On a Unix port, everything that is local to a process is 'local'. So MAP_SHARED gets addresses without the local bit set and MAP_PRIVATE does. When we fork(), we have some choices. The OS picks a new local offset for the new process, meaning that local pointers don't alias one another. We could straight away clone the protection entries and page tables. Or, as an implementation detail, depending on the extent to which the PLB self-manages on the particular Mill member, just let the PLB trap. As the new forked process accesses memory ranges for the first time the PLB will miss and there will be the classic trap. In the trap, the handler can set up COW lazily and for the memory regions that are actually touched by the child. I am simplifying it a bit ;) Most fork() is just going to exec(), and the latter will win. Some people are still using fork() in other ways, and they may benefit from an aggressive COW up front on fork(). Swings and roundabouts.
- aidenn0 11y ago
  Thanks for your answers!