4 ms·

"As soon as I write to the SATP register and set the mode, it is up and running since we’re in supervisor mode" While this is true on some CPUs it's not true o

• pavehawk2007 6y ago
  I don't think it's as cut and dry as "execute sfence.vma every time you write to SATP" as you say for the following reasons. 1. The sfence.vma is much like a TLB flush. Just like a normal fence, it ensures that orderings are done properly. The privileged spec says this in section 4.2.1. In fact, a parenthetical tells us that "The SFENCE.VMA is used to flush any local hardware caches related to address translation." 2. The sfence.vma has three formats. sfence.vma x0, x0, sfence.vma x? (!= 0), x0, and sfence.vma x0, x? (!= 0). This allows us to flush certain addresses from the translation lookaside buffer (TLB) so that the newly updated page tables are updated in this buffer. 3. You don't have to call sfence.vma every time you write to the SATP. This would thrash the TLB every time you context switch. Please see page 66 of the privileged spec (20190608 and more recent versions). 4. If you see the original tutorial this blog post is based on, you can see that I use sfence.vma with the ASID which is the process ID. That way there, the address space and process ID are equivalent so that when I transfer the process to another hart, I can execute sfence.vma x? where x? contains the ASID. If I add a mapping, such as sbrk or mmap, then I can call sfence.vma x0, x? where x? contains the specific page I want updated. 5. If we use PMP (physical memory protection), the specification tells us to use sfence.vma x0, x0 to update in case the hart speculates before we use those particular addresses (see section 3.6.2 in the privileged SPEC). 6. The code I show updating the SATP is the same code we used in the original tutorial: https://osblog.stephenmarz.com https://osblog.stephenmarz.com. In this case, I use sfence.vma when I update a mapping, transfer a process to another hart, or when I create a new process. Maybe I'm wrong, but my reading of the SFENCE.VMA instruction combined with the fact that we have multiple versions and the ability to TLB shootdown an entire address space or a single page tells me that it isn't "execute sfence.vma every time SATP is written." In conclusion, I don't use sfence.vma every time I write to SATP. Instead, I use it when there is a memory ordering issue, such as starting a new process, updating a mapping, or transferring a process to another hart. In this case, we wouldn't do this in the code that switches us to a userspace process, since this is invoked every time we do a context switch.
  • pavehawk2007 6y ago
    Here's the specification's recommended use of SFENCE.VMA, which is just a fancier version of what I wrote above: Page 66 and 67: The following common situations typically require executing an SFENCE.VMA instruction: 1. When software recycles an ASID (i.e., reassociates it with a different page table), it should first change satp to point to the new page table using the recycled ASID, then execute SFENCE.VMA with rs1=x0 and rs2 set to the recycled ASID. Alternatively, software can execute the same SFENCE.VMA instruction while a different ASID is loaded into satp, provided the next time satp is loaded with the recycled ASID, it is simultaneously loaded with the new page table. 2. If the implementation does not provide ASIDs, or software chooses to always use ASID 0, then after every satp write, software should execute SFENCE.VMA with rs1=x0. In the common case that no global translations have been modified, rs2 should be set to a register other than x0 but which contains the value zero, so that global translations are not flushed. 3. If software modifies a non-leaf PTE, it should execute SFENCE.VMA with rs1=x0. If any PTE along the traversal path had its G bit set, rs2 must be x0; otherwise, rs2 should be set to the ASID for which the translation is being modified. 4. If software modifies a leaf PTE, it should execute SFENCE.VMA with rs1 set to a virtual address within the page. If any PTE along the traversal path had its G bit set, rs2 must be x0; otherwise, rs2 should be set to the ASID for which the translation is being modified. 5. For the special cases of increasing the permissions on a leaf PTE and changing an invalid PTE to a valid leaf, software may choose to execute the SFENCE.VMA lazily. After modifying the PTE but before executing SFENCE.VMA, either the new or old permissions will be used. In the latter case, a page fault exception might occur, at which point software should execute SFENCE.VMA in accordance with the previous bullet point.
    • Taniwha 6y ago
      But note: "A consequence of this specification is that an implementation may use any translation for an address that was valid at any time since the most recent SFENCE.VMA that subsumes that address. In particular, if a leaf PTE is modified but a subsuming SFENCE.VMA is not executed, either the old translation or the new translation will be used, but the choice is unpredictable. The behavior is otherwise well-defined." This is not the "particular" case mentioned - essentially "subsumed" for satp is the entire space (you have to think of satp as simply being a non-leaf PTE) To be fair I think that there's issues in this area that may force a double pipe flush in some implementations
• sdbbp 6y ago
  This was clarified this year. I think the author's context is moving from Bare to one of the Sv modes. https://github.com/riscv/riscv-isa-manual/commit/866326d938cc6b33dfe051ebbe207c36ac384511 https://github.com/riscv/riscv-isa-manual/commit/866326d938c...
  • deleted 6y ago
    [deleted]