3 ms·

What do you mean by "not happening very frequently?" The default timeslice is something on the order of 100ms, isn't it? And that's if the process doesn't yield

by ollien 6y ago

What do you mean by "not happening very frequently?" The default timeslice is something on the order of 100ms, isn't it? And that's if the process doesn't yield. Clearing L1d every 100ms (at worst) seems pretty frequent to me.

lmilcin 6y ago
The cost of clearing cache on context switch has to be put in context (hey, pun intended:) 100ms is a huge amount of time and 48kB is a tiny, tiny part of what processor does during 100ms. Gigabytes of data can be transferred during that time, 48kB isn't really much. As I have pointed out, that cache has very little value over context switch anyway. The cost is removing data from cache that would be usable after we have returned to the original context. But it is already very likely the data in the cache is already for a completely different context and hence completely unusable. Say you have apps A and B and OS. You are running A which has 48kB of data in L1d. It switches to OS which causes some of L1d to be evicted and puts its own data there. Then it switches to B which is likely another process, this causes very likely entire L1d to be evicted unless this is extremely small process. Then we come to OS and again to A. By the time you are at A, there is no data from the original L1d state. Cleaning L1d upfront on context switch is likely not hurting anything.
- MaulingMonkey 6y ago
  To further back this up with some math - L2 cache hits (what you'll hit on a L1D cache miss caused by clearing L1D cache) - are still in the mid/low single digit nanosecond ranges[1]. Say flushing the L1D causes another 1000 L2 cache misses[2] - maybe we got really lucky and the next thread was hashing all the exact same data at the exact same time or something equally unlikely? That'd still put us in the mid/low single digit microseconds range. On par with DDR4-1600 (12.8GB/s)'s 3.75us to read 48KB [3][4]. Let's more than double that and say it takes 10 microseconds = 0.01 milliseconds = 0.01% of 100 milliseconds. Any noticable perf overhead is going to be from the act of cache flushing taking some super slow path for some reason, or much more frequent context switching than 100ms timeslices. [1] https://stackoverflow.com/a/4087331 https://stackoverflow.com/a/4087331 [2] 1000x 32-128B cachelines = 32-128KB, definitely in the ballpark to completely refill a 48KB L1D cache. [3] https://en.wikipedia.org/wiki/DDR4_SDRAM#Modules https://en.wikipedia.org/wiki/DDR4_SDRAM#Modules [4] https://www.wolframalpha.com/input/?i=48KB+%2F+12800+MB%2Fs https://www.wolframalpha.com/input/?i=48KB+%2F+12800+MB%2Fs
deleted 6y ago
[deleted]
magicalhippo 6y ago
> The default timeslice is something on the order of 100ms, isn't it? Consider that a single core on a modern CPU running at 2 GHz can execute over 20k instructions in those 100ms.
- sgerenser 6y ago
  20 Million instructions in 100ms. More if IPC is >1.
  - magicalhippo 6y ago
    And that's why I shouldn't do math at night... Anyway, 100ms is quite a lot in the life of a modern CPU.
    - lmilcin 6y ago
      Even 1ms is a lot. I have some experience with algorithmic trading. The application took messages off the network, processed them and responded to market within 5 microseconds. That's 1/200th of 1ms. This measured on a special type of switch (https://en.wikipedia.org/wiki/Cut-through_switching https://en.wikipedia.org/wiki/Cut-through_switching ). Lots of stuff happens during those 5us. The message is read from the network device (directly by the application, no Linux or syscalls anywhere during those 5us). Then it is parsed, deduplicated (multiple multicast channels carry redundant copies of the messages), uncompressed (the payload is compressed with zlib), the uncompressed payload is parsed, interpreted (multiple types of messages). Business logic is executed to update state of the market in memory then to generate signals to listening algorithms. The algorithm is run to figure out whether it wants to execute an order. The order is verified against decision tree (for example to check whether it does not exceed available budget). The market order packet is created and sent over TCP. Now imagine, all that stuff happens in 1/200th of 1ms. In comparison, transferring 48kB from L2 or L3 to L1 is pretty damn insignificant.
  - pansa2 6y ago
    200 Million.
ken 6y ago
That sounds way too high. It's CONFIG_HZ, right? According to this[1], it can be 1ms to 10ms, with the default being 4ms. [1]: https://github.com/torvalds/linux/blob/master/kernel/Kconfig.hz https://github.com/torvalds/linux/blob/master/kernel/Kconfig...
mehrdadn 6y ago
On Windows it's around 1ms-15ms; I doubt on Linux it's very different. For reference 60fps gives 16.6ms.

by ollien 6y ago

lmilcin 6y ago

The cost of clearing cache on context switch has to be put in context (hey, pun intended:) 100ms is a huge amount of time and 48kB is a tiny, tiny part of what processor does during 100ms. Gigabytes of data can be transferred during that time, 48kB isn't really much. As I have pointed out, that cache has very little value over context switch anyway. The cost is removing data from cache that would be usable after we have returned to the original context. But it is already very likely the data in the cache is already for a completely different context and hence completely unusable. Say you have apps A and B and OS. You are running A which has 48kB of data in L1d. It switches to OS which causes some of L1d to be evicted and puts its own data there. Then it switches to B which is likely another process, this causes very likely entire L1d to be evicted unless this is extremely small process. Then we come to OS and again to A. By the time you are at A, there is no data from the original L1d state. Cleaning L1d upfront on context switch is likely not hurting anything.

MaulingMonkey 6y ago
To further back this up with some math - L2 cache hits (what you'll hit on a L1D cache miss caused by clearing L1D cache) - are still in the mid/low single digit nanosecond ranges[1]. Say flushing the L1D causes another 1000 L2 cache misses[2] - maybe we got really lucky and the next thread was hashing all the exact same data at the exact same time or something equally unlikely? That'd still put us in the mid/low single digit microseconds range. On par with DDR4-1600 (12.8GB/s)'s 3.75us to read 48KB [3][4]. Let's more than double that and say it takes 10 microseconds = 0.01 milliseconds = 0.01% of 100 milliseconds. Any noticable perf overhead is going to be from the act of cache flushing taking some super slow path for some reason, or much more frequent context switching than 100ms timeslices. [1] https://stackoverflow.com/a/4087331 https://stackoverflow.com/a/4087331 [2] 1000x 32-128B cachelines = 32-128KB, definitely in the ballpark to completely refill a 48KB L1D cache. [3] https://en.wikipedia.org/wiki/DDR4_SDRAM#Modules https://en.wikipedia.org/wiki/DDR4_SDRAM#Modules [4] https://www.wolframalpha.com/input/?i=48KB+%2F+12800+MB%2Fs https://www.wolframalpha.com/input/?i=48KB+%2F+12800+MB%2Fs

deleted 6y ago

[deleted]

magicalhippo 6y ago

> The default timeslice is something on the order of 100ms, isn't it? Consider that a single core on a modern CPU running at 2 GHz can execute over 20k instructions in those 100ms.

sgerenser 6y ago
20 Million instructions in 100ms. More if IPC is >1.
- magicalhippo 6y ago
  And that's why I shouldn't do math at night... Anyway, 100ms is quite a lot in the life of a modern CPU.
  - lmilcin 6y ago
    Even 1ms is a lot. I have some experience with algorithmic trading. The application took messages off the network, processed them and responded to market within 5 microseconds. That's 1/200th of 1ms. This measured on a special type of switch (https://en.wikipedia.org/wiki/Cut-through_switching https://en.wikipedia.org/wiki/Cut-through_switching ). Lots of stuff happens during those 5us. The message is read from the network device (directly by the application, no Linux or syscalls anywhere during those 5us). Then it is parsed, deduplicated (multiple multicast channels carry redundant copies of the messages), uncompressed (the payload is compressed with zlib), the uncompressed payload is parsed, interpreted (multiple types of messages). Business logic is executed to update state of the market in memory then to generate signals to listening algorithms. The algorithm is run to figure out whether it wants to execute an order. The order is verified against decision tree (for example to check whether it does not exceed available budget). The market order packet is created and sent over TCP. Now imagine, all that stuff happens in 1/200th of 1ms. In comparison, transferring 48kB from L2 or L3 to L1 is pretty damn insignificant.
- pansa2 6y ago
  200 Million.

ken 6y ago

That sounds way too high. It's CONFIG_HZ, right? According to this[1], it can be 1ms to 10ms, with the default being 4ms. [1]: https://github.com/torvalds/linux/blob/master/kernel/Kconfig.hz https://github.com/torvalds/linux/blob/master/kernel/Kconfig...

mehrdadn 6y ago

On Windows it's around 1ms-15ms; I doubt on Linux it's very different. For reference 60fps gives 16.6ms.