4 ms·

the limit isn't the processor, it's how fast memory is. With DDR3-1600, it should be 11.97 GiB/s (12.8 GB/s) I don't understand this reasoning. Why is it being

the limit isn't the processor, it's how fast memory is. With DDR3-1600, it should be 11.97 GiB/s (12.8 GB/s) I don't understand this reasoning. Why is it being limited to main memory speed? Surely the yes program, the fragments of the OS being used, and the program reading the data, all fit within the L2 cache?

pandaman 9y ago
In general, only the CPU itself sees the L2 cache. Anything you see on another device (screen, disk, NIC etc) has been flushed out of cache.
- joosters 9y ago
  Sure, but this is a pipe between two tiny processes, hopefully with very little else being run on the computer at the time (otherwise all bets are off for any benchmarking). There's no kind of 'real' I/O going on (in terms of stuff like screen, disk, NIC, and so on) There's no reason that the L2 cache needed to be flushed at any point - the caches are all dealing with physical memory, rather than virtualised address space, so the fact that there are two processes here shouldn't stop the caching from working.
  - pandaman 9y ago
    I am probably way behind the current state of the CPU judging by the downvotes I got so if you are saying there is no reason and the data can be written into a device without leaving the CPU I will just concede my ignorance.
    - joosters 9y ago
      Don't fret about the downvotes, these magic internet points aren't redeemable anywhere :) It's completely possible for the data to not (all) leave the CPU. If the caches are large enough, then the pages full of "y\n" will be still resident in the cache when the next iteration of the program overwrites the same pages again. Then the CPU has no need to send the original page out to main memory.
      - pandaman 9y ago
        If you did for(;;) buffer[(i++)%size] = 'y'; then you'd be correct. However, you do i/o. And the 'y's appear at the i/o driver and have to be made visible for the device it's driving, which can be anything, including a process on another CPU core in another socket. If they remained in the issuing CPU cache, I fail to see how the destination device could possibly see them. There are some devices which can snoop the cache (like SoC GPUs and other CPU sockets on some archs) but the snooping is much slower than memory bus. Writing to memory is the only way which a) guarantees the data is available elsewhere and b) is the fastest.
- deleted 9y ago
  [deleted]
qb45 9y ago
It is NOT limited to the external RAM speed and the best proof is that it actually uses over 40GB/s of memory bandwidth. For each byte of data passing through pv: 1. the byte is read from yes memory to CPU registers by the write syscall handler in the kernel 2. the byte is written to kernel's internal buffer associated with the pipe 3. the byte is read back in the read syscall called by pv 4. the byte is written to a buffer in pv memory 5. and thas's the end because write syscall executed by pv on /dev/null very likely doesn't actually bother reading the submitted buffer at all edit: Actually it might only be 20GB/s because on Linux pv seems to use the splice syscall to transfer data from stdin to stdout without copying to userspace. This is also the reason why further "optimization" of this program in assembly was a fool's errand: the bulk of CPU load is in the kernel.
- joosters 9y ago
  All good valid points, but I'm still a bit surprised that the limit is not higher, I thought that the L2 cache was over an order of magnitude faster than main memory (plus, as someone pointed out in the reddit thread, the peak memory performance should really be double the quoted 12GB/s due to dual channel memory). The actual throughput then, once you include OS copying, is either 2 or 4 times the quoted speed (depending on splice usage), so we're either at main memory theoretical speeds, or double main memory speeds. Intuitively, I'd still have expected that it should be a larger multiple. (A quick search can't find me any reliable Intel L1/L2 cache speeds/multiples to quote, so I admit this comment is more speculation than it should be!)
  - maxhou 9y ago
    yes and pv processes are not scheduled on the same CPU core, so different L2 cache.
    - daveguy 9y ago
      I wonder if `taskset -c1 yes | taskset -c1 pv > /dev/null` would significantly change the throughput.
alexeldeib 9y ago
As another point in your favor, one of the commentors reached 123 GB/s modifying both yes and pv. https://www.reddit.com/r/unix/comments/6gxduc/how_is_gnu_yes_so_fast/diua761/ https://www.reddit.com/r/unix/comments/6gxduc/how_is_gnu_yes...

pandaman 9y ago

In general, only the CPU itself sees the L2 cache. Anything you see on another device (screen, disk, NIC etc) has been flushed out of cache.

joosters 9y ago
Sure, but this is a pipe between two tiny processes, hopefully with very little else being run on the computer at the time (otherwise all bets are off for any benchmarking). There's no kind of 'real' I/O going on (in terms of stuff like screen, disk, NIC, and so on) There's no reason that the L2 cache needed to be flushed at any point - the caches are all dealing with physical memory, rather than virtualised address space, so the fact that there are two processes here shouldn't stop the caching from working.
- pandaman 9y ago
  I am probably way behind the current state of the CPU judging by the downvotes I got so if you are saying there is no reason and the data can be written into a device without leaving the CPU I will just concede my ignorance.
  - joosters 9y ago
    Don't fret about the downvotes, these magic internet points aren't redeemable anywhere :) It's completely possible for the data to not (all) leave the CPU. If the caches are large enough, then the pages full of "y\n" will be still resident in the cache when the next iteration of the program overwrites the same pages again. Then the CPU has no need to send the original page out to main memory.
    - pandaman 9y ago
      If you did for(;;) buffer[(i++)%size] = 'y'; then you'd be correct. However, you do i/o. And the 'y's appear at the i/o driver and have to be made visible for the device it's driving, which can be anything, including a process on another CPU core in another socket. If they remained in the issuing CPU cache, I fail to see how the destination device could possibly see them. There are some devices which can snoop the cache (like SoC GPUs and other CPU sockets on some archs) but the snooping is much slower than memory bus. Writing to memory is the only way which a) guarantees the data is available elsewhere and b) is the fastest.
deleted 9y ago
[deleted]

qb45 9y ago

It is NOT limited to the external RAM speed and the best proof is that it actually uses over 40GB/s of memory bandwidth. For each byte of data passing through pv: 1. the byte is read from yes memory to CPU registers by the write syscall handler in the kernel 2. the byte is written to kernel's internal buffer associated with the pipe 3. the byte is read back in the read syscall called by pv 4. the byte is written to a buffer in pv memory 5. and thas's the end because write syscall executed by pv on /dev/null very likely doesn't actually bother reading the submitted buffer at all edit: Actually it might only be 20GB/s because on Linux pv seems to use the splice syscall to transfer data from stdin to stdout without copying to userspace. This is also the reason why further "optimization" of this program in assembly was a fool's errand: the bulk of CPU load is in the kernel.

joosters 9y ago
All good valid points, but I'm still a bit surprised that the limit is not higher, I thought that the L2 cache was over an order of magnitude faster than main memory (plus, as someone pointed out in the reddit thread, the peak memory performance should really be double the quoted 12GB/s due to dual channel memory). The actual throughput then, once you include OS copying, is either 2 or 4 times the quoted speed (depending on splice usage), so we're either at main memory theoretical speeds, or double main memory speeds. Intuitively, I'd still have expected that it should be a larger multiple. (A quick search can't find me any reliable Intel L1/L2 cache speeds/multiples to quote, so I admit this comment is more speculation than it should be!)
- maxhou 9y ago
  yes and pv processes are not scheduled on the same CPU core, so different L2 cache.
  - daveguy 9y ago
    I wonder if `taskset -c1 yes | taskset -c1 pv > /dev/null` would significantly change the throughput.

alexeldeib 9y ago

As another point in your favor, one of the commentors reached 123 GB/s modifying both yes and pv. https://www.reddit.com/r/unix/comments/6gxduc/how_is_gnu_yes_so_fast/diua761/ https://www.reddit.com/r/unix/comments/6gxduc/how_is_gnu_yes...