4 ms·

I believe the 0.28% are only for CPUs that support PCID. Earlier CPUs (which is a lot still) will get a much harder hit since you'll have to flush the entire TL

by tscs37 9y ago

I believe the 0.28% are only for CPUs that support PCID. Earlier CPUs (which is a lot still) will get a much harder hit since you'll have to flush the entire TLB.

my123 9y ago
Even with PCID, the hit is 29% in a tight syscall loop, but is a complete disaster above 50% with PCID off...
- BeeOnRope 9y ago
  In fact, the tight syscall loop isn't even necessarily the worst case: the primary cost of this change isn't the a direct cost in the syscall, but the CR3 switch which invalidates the TLB and incurs an ongoing cost for some time following the syscall. The worse case would be something like a frequent syscall followed by code that touches a number of distinct cache lines, which all now require a TLB reload and page-walk (even here the cost is tricky to evaluate since there are various levels where the paging structures can be cached beyond the TLB, so the cost of a page-walk various a lot depending on locality of the paging structures used).
  - bonzini 9y ago
    PCIDs avoid the TLB invalidation.
    - BeeOnRope 9y ago
      Yup, I think so. In that case (PCID hardware on a PCID-enabled kernel), the performance effect should be more limited to the syscall itself. That said, why is the hit still so big with PCID? Surely just the CR3-swap by itself shouldn't be so slow?
      - bonzini 9y ago
        MOV-to-CR3 is pretty slow, yes. In the ballpark of a hundred clock cycles, and you have to do 2 of them. The cost of a system call was about 1000 cycles, maybe less on newer processors---both OSes and processors optimize the hell out of SYSCALL/SYSRET.
walterbell 9y ago
Which Intel CPUs support PCID? Based on this Linus message, it was introduced in 2015, so Broadwell and later? http://lkml.iu.edu/hypermail/linux/kernel/1504.3/02961.html http://lkml.iu.edu/hypermail/linux/kernel/1504.3/02961.html
- Freaky 9y ago
  PCID's been around for a long time - even my old Westmere Xeons have it. INVPCID is more recent.

by tscs37 9y ago

I believe the 0.28% are only for CPUs that support PCID. Earlier CPUs (which is a lot still) will get a much harder hit since you'll have to flush the entire TLB.

my123 9y ago

Even with PCID, the hit is 29% in a tight syscall loop, but is a complete disaster above 50% with PCID off...

BeeOnRope 9y ago
In fact, the tight syscall loop isn't even necessarily the worst case: the primary cost of this change isn't the a direct cost in the syscall, but the CR3 switch which invalidates the TLB and incurs an ongoing cost for some time following the syscall. The worse case would be something like a frequent syscall followed by code that touches a number of distinct cache lines, which all now require a TLB reload and page-walk (even here the cost is tricky to evaluate since there are various levels where the paging structures can be cached beyond the TLB, so the cost of a page-walk various a lot depending on locality of the paging structures used).
- bonzini 9y ago
  PCIDs avoid the TLB invalidation.
  - BeeOnRope 9y ago
    Yup, I think so. In that case (PCID hardware on a PCID-enabled kernel), the performance effect should be more limited to the syscall itself. That said, why is the hit still so big with PCID? Surely just the CR3-swap by itself shouldn't be so slow?
    - bonzini 9y ago
      MOV-to-CR3 is pretty slow, yes. In the ballpark of a hundred clock cycles, and you have to do 2 of them. The cost of a system call was about 1000 cycles, maybe less on newer processors---both OSes and processors optimize the hell out of SYSCALL/SYSRET.

walterbell 9y ago

Which Intel CPUs support PCID? Based on this Linus message, it was introduced in 2015, so Broadwell and later? http://lkml.iu.edu/hypermail/linux/kernel/1504.3/02961.html http://lkml.iu.edu/hypermail/linux/kernel/1504.3/02961.html

Freaky 9y ago
PCID's been around for a long time - even my old Westmere Xeons have it. INVPCID is more recent.