6 ms·

Pretty sure that he's just rediscovering what I did in lmbench back in the early 1990's. All written up and open source (and got best paper in the 1995 Usenix)

Pretty sure that he's just rediscovering what I did in lmbench back in the early 1990's. All written up and open source (and got best paper in the 1995 Usenix): http://mcvoy.com/lm/bitmover/lmbench/lmbench-usenix.pdf http://mcvoy.com/lm/bitmover/lmbench/lmbench-usenix.pdf Edit: which now that I think about it, was over 20 years ago. Reminds me of something a friend (Ron Minnich) said long ago: people rediscover stuff in computer science over and over, nobody wants to read old papers. Or something like that. He said it better, I think it was something like "want to look smart? Just take any 5 year old paper and redo whatever it did, our attention span seems to be very short."

digi_owl 10y ago
Everything old is new again it seems. I guess this is a variant of CADT, based on ignorance of the past rather than a chase of shinies.
- cuspycode 10y ago
  People have always obsessed over "news", since humans are information-hungry by nature. But the modern tech crowd seems to have forgotten that there are two kinds of news: the kind that is objectively new (i.e. recent data), and the kind that is subjectively new (i.e. I didn't know that before). I think the situation is a bit similar to the distinction between objectively established frequencies and subjective Bayesian priors.
chillydawg 10y ago
One reason people are reluctant to trust old sources (regardless of how well received they were at the time) is that software, operating systems and hardware all move at lightning pace. I've not read your paper, but I'm willing to bet enough has changed to warrant a new writeup talking about modern systems.
- luckydude 10y ago
  I very much doubt it. If you read the paper, it took each of the things it could measure (memory, disk, network, etc) and measured bandwidth and latency. Those are pretty basic measurements and I suspect that they will still be interesting 100 years from now and measured in the same way. I'm running the benchmark right now on a modern system just to see what it says. Yep, still works fine: "size=0k ovr=0.90 2 7.63 4 6.51 8 7.78 16 7.07 24 7.78 32 9.02 64 10.35 96 10.49 That says that a hot potato (I send a byte to you, you send it back to me, the time is the round trip for one pair) context switch is ~7 usecs. Do it in a ring with 96 processes and it goes up to ~10 usecs. I think almost 100% of that benchmark is still valid (I say almost because there must be something that has bit rotted in 20+ years but I don't know of anything). It's worth a read. Edit: formatting.
- LoSboccacc 10y ago
  > operating systems and hardware all move at lightning pace not really, last big leap was the ability to unlock gpu for stream processing, and that's it. it's quite faster today as ten or one year ago, but at it's core it's still a von neumann machine and not much has been discovered ever since, mostly we're repackaging under a new name things that have been discovered in the 70s', like the thick vs thin client debate, which played more or less the same over and over again as the bottleneck switched from bandwidth to latency to client performances, the last iteration of which was ember and angular precompilation.
  - noir_lord 10y ago
    Some of the stuff goes back even further than that. The thread on the Intel Optane stuff the other day had an interesting conversation about how you would program for such systems and someone said we already did, mainframes 50 years ago. It's nice to see old ideas in a fresh context.
- jlg23 10y ago
  > I've not read your paper, but I'm willing to bet [...] Offloading all burden of proof onto the partner in a conversation strikes me as.. odd - esp. if said partner already provided a paper on the very same topic that you did not even read.
kirse 10y ago
You're like the auto mechanic I talked to that was bemoaning this generation and how automotive tech has basically "stayed the same" since the 50s... "MacPherson struts, you ever heard of those boy!? Yeah - invented in the 50s!" Nothing wrong with a fresh take on an old topic.
marcosdumay 10y ago
Your link is about 1990's hardware, the article is about 2010 hardware. The article was revisiting the same topic exactly because the hardware changed (the author said so multiple times), and cheap context switches were a big marketing topic at the time. It would even be interesting to get some newer study of that. You can bet the relative costs of a context switch changed a lot since 2010, as did the waiting latency.
- luckydude 10y ago
  Did you read the posted article? It spends a lot of time trying to figure out how to measure context switches. Which I carefully wrote up around 1995 and still works to this day. That's independent of the hardware and was sort of my point. This is a solved problem. And if you look at Table 10, the time for a context switch is remarkably similar to what it was 20 years ago (which means the OS is fatter, not surprising, it's scales up on SMPs now).
  - marcosdumay 10y ago
    I got the impression you were talking about the numbers. In that I agree. You've got basically the same procedure written there for anybody to learn. And a nice tool that runs it.
    - luckydude 10y ago
      The numbers are just a snapshot into history but what I measured and how it was measured is far more interesting. For example your comment "you've got basically the same procedure" isn'tquite right. If you read the paper, much effort was spent explaining to people that the baseline context switch is just a floor, what you will see in real life will be bigger since in real life a context switch is more that just a register file save/restore. Very few benchmark writers get that point but that's sort of the main point. Not getting it means you are doomed to be mislead.
      - luckydude 10y ago
luckydude 10y ago
Full results from mcvoy.com i7@3.5ghz: L M B E N C H 2 . 0 S U M M A R Y ------------------------------------ Basic system parameters ---------------------------------------------------- Host OS Description Mhz --------- ------------- ----------------------- ---- slovax.mc Linux 4.4.0-5 x86_64-glibc223-linux 3497 Processor, Processes - times in microseconds - smaller is better ---------------------------------------------------------------- Host OS Mhz null null open selct sig sig fork exec sh call I/O stat clos TCP inst hndl proc proc proc --------- ------------- ---- ---- ---- ---- ---- ----- ---- ---- ---- ---- ---- slovax.mc Linux 4.4.0-5 3497 0.09 0.12 0.28 1.76 2.170 0.12 0.58 86.4 249. 644. Context switching - times in microseconds - smaller is better ------------------------------------------------------------- Host OS 2p/0K 2p/16K 2p/64K 8p/16K 8p/64K 16p/16K 16p/64K ctxsw ctxsw ctxsw ctxsw ctxsw ctxsw ctxsw --------- ------------- ----- ------ ------ ------ ------ ------- ------- slovax.mc Linux 4.4.0-5 7.630 8.5200 10.5 10.9 15.5 10.8 19.0 *Local* Communication latencies in microseconds - smaller is better ------------------------------------------------------------------- Host OS 2p/0K Pipe AF UDP RPC/ TCP RPC/ TCP ctxsw UNIX UDP TCP conn --------- ------------- ----- ----- ---- ----- ----- ----- ----- ---- slovax.mc Linux 4.4.0-5 7.630 13.6 14.7 20.8 19.4 25.8 24.5 25.8 File & VM system latencies in microseconds - smaller is better -------------------------------------------------------------- Host OS 0K File 10K File Mmap Prot Page Create Delete Create Delete Latency Fault Fault --------- ------------- ------ ------ ------ ------ ------- ----- ----- slovax.mc Linux 4.4.0-5 2.7920 2.0480 8.2600 4.0840 17.6K 0.158 5.00000 *Local* Communication bandwidths in MB/s - bigger is better ----------------------------------------------------------- Host OS Pipe AF TCP File Mmap Bcopy Bcopy Mem Mem UNIX reread reread (libc) (hand) read write --------- ------------- ---- ---- ---- ------ ------ ------ ------ ---- ----- slovax.mc Linux 4.4.0-5 1928 11.K 5674 43009M 11.0K 11.2K 7686.9 14.K 10.3K Memory latencies in nanoseconds - smaller is better (WARNING - may not be correct, check graphs) --------------------------------------------------- Host OS Mhz L1 $ L2 $ Main mem Guesses --------- ------------- ---- ----- ------ -------- ------- slovax.mc Linux 4.4.0-5 3497 0.889 2.6670 57.4
drudru11 10y ago
Kind of like hacker news posts.
js2 10y ago
Author somewhat addresses that in a comment: @Adrian Cockcroft: I didn't use lmbench because I was wondering how to write such a benchmark in the first place, and you always learn more by building things yourself than by using something already existing. My little benchmarks aren't supposed to be alternatives to lmbench, I simply provided the source code so others could unambiguously see how I did the benchmarks and reproduce them.

digi_owl 10y ago

Everything old is new again it seems. I guess this is a variant of CADT, based on ignorance of the past rather than a chase of shinies.

cuspycode 10y ago
People have always obsessed over "news", since humans are information-hungry by nature. But the modern tech crowd seems to have forgotten that there are two kinds of news: the kind that is objectively new (i.e. recent data), and the kind that is subjectively new (i.e. I didn't know that before). I think the situation is a bit similar to the distinction between objectively established frequencies and subjective Bayesian priors.

chillydawg 10y ago

One reason people are reluctant to trust old sources (regardless of how well received they were at the time) is that software, operating systems and hardware all move at lightning pace. I've not read your paper, but I'm willing to bet enough has changed to warrant a new writeup talking about modern systems.

luckydude 10y ago
I very much doubt it. If you read the paper, it took each of the things it could measure (memory, disk, network, etc) and measured bandwidth and latency. Those are pretty basic measurements and I suspect that they will still be interesting 100 years from now and measured in the same way. I'm running the benchmark right now on a modern system just to see what it says. Yep, still works fine: "size=0k ovr=0.90 2 7.63 4 6.51 8 7.78 16 7.07 24 7.78 32 9.02 64 10.35 96 10.49 That says that a hot potato (I send a byte to you, you send it back to me, the time is the round trip for one pair) context switch is ~7 usecs. Do it in a ring with 96 processes and it goes up to ~10 usecs. I think almost 100% of that benchmark is still valid (I say almost because there must be something that has bit rotted in 20+ years but I don't know of anything). It's worth a read. Edit: formatting.
LoSboccacc 10y ago
> operating systems and hardware all move at lightning pace not really, last big leap was the ability to unlock gpu for stream processing, and that's it. it's quite faster today as ten or one year ago, but at it's core it's still a von neumann machine and not much has been discovered ever since, mostly we're repackaging under a new name things that have been discovered in the 70s', like the thick vs thin client debate, which played more or less the same over and over again as the bottleneck switched from bandwidth to latency to client performances, the last iteration of which was ember and angular precompilation.
- noir_lord 10y ago
  Some of the stuff goes back even further than that. The thread on the Intel Optane stuff the other day had an interesting conversation about how you would program for such systems and someone said we already did, mainframes 50 years ago. It's nice to see old ideas in a fresh context.
jlg23 10y ago
> I've not read your paper, but I'm willing to bet [...] Offloading all burden of proof onto the partner in a conversation strikes me as.. odd - esp. if said partner already provided a paper on the very same topic that you did not even read.

kirse 10y ago

You're like the auto mechanic I talked to that was bemoaning this generation and how automotive tech has basically "stayed the same" since the 50s... "MacPherson struts, you ever heard of those boy!? Yeah - invented in the 50s!" Nothing wrong with a fresh take on an old topic.

marcosdumay 10y ago

Your link is about 1990's hardware, the article is about 2010 hardware. The article was revisiting the same topic exactly because the hardware changed (the author said so multiple times), and cheap context switches were a big marketing topic at the time. It would even be interesting to get some newer study of that. You can bet the relative costs of a context switch changed a lot since 2010, as did the waiting latency.

luckydude 10y ago
Did you read the posted article? It spends a lot of time trying to figure out how to measure context switches. Which I carefully wrote up around 1995 and still works to this day. That's independent of the hardware and was sort of my point. This is a solved problem. And if you look at Table 10, the time for a context switch is remarkably similar to what it was 20 years ago (which means the OS is fatter, not surprising, it's scales up on SMPs now).
- marcosdumay 10y ago
  I got the impression you were talking about the numbers. In that I agree. You've got basically the same procedure written there for anybody to learn. And a nice tool that runs it.
  - luckydude 10y ago
    The numbers are just a snapshot into history but what I measured and how it was measured is far more interesting. For example your comment "you've got basically the same procedure" isn'tquite right. If you read the paper, much effort was spent explaining to people that the baseline context switch is just a floor, what you will see in real life will be bigger since in real life a context switch is more that just a register file save/restore. Very few benchmark writers get that point but that's sort of the main point. Not getting it means you are doomed to be mislead.
    - luckydude 10y ago

luckydude 10y ago

Full results from mcvoy.com i7@3.5ghz: L M B E N C H 2 . 0 S U M M A R Y ------------------------------------ Basic system parameters ---------------------------------------------------- Host OS Description Mhz --------- ------------- ----------------------- ---- slovax.mc Linux 4.4.0-5 x86_64-glibc223-linux 3497 Processor, Processes - times in microseconds - smaller is better ---------------------------------------------------------------- Host OS Mhz null null open selct sig sig fork exec sh call I/O stat clos TCP inst hndl proc proc proc --------- ------------- ---- ---- ---- ---- ---- ----- ---- ---- ---- ---- ---- slovax.mc Linux 4.4.0-5 3497 0.09 0.12 0.28 1.76 2.170 0.12 0.58 86.4 249. 644. Context switching - times in microseconds - smaller is better ------------------------------------------------------------- Host OS 2p/0K 2p/16K 2p/64K 8p/16K 8p/64K 16p/16K 16p/64K ctxsw ctxsw ctxsw ctxsw ctxsw ctxsw ctxsw --------- ------------- ----- ------ ------ ------ ------ ------- ------- slovax.mc Linux 4.4.0-5 7.630 8.5200 10.5 10.9 15.5 10.8 19.0 *Local* Communication latencies in microseconds - smaller is better ------------------------------------------------------------------- Host OS 2p/0K Pipe AF UDP RPC/ TCP RPC/ TCP ctxsw UNIX UDP TCP conn --------- ------------- ----- ----- ---- ----- ----- ----- ----- ---- slovax.mc Linux 4.4.0-5 7.630 13.6 14.7 20.8 19.4 25.8 24.5 25.8 File & VM system latencies in microseconds - smaller is better -------------------------------------------------------------- Host OS 0K File 10K File Mmap Prot Page Create Delete Create Delete Latency Fault Fault --------- ------------- ------ ------ ------ ------ ------- ----- ----- slovax.mc Linux 4.4.0-5 2.7920 2.0480 8.2600 4.0840 17.6K 0.158 5.00000 *Local* Communication bandwidths in MB/s - bigger is better ----------------------------------------------------------- Host OS Pipe AF TCP File Mmap Bcopy Bcopy Mem Mem UNIX reread reread (libc) (hand) read write --------- ------------- ---- ---- ---- ------ ------ ------ ------ ---- ----- slovax.mc Linux 4.4.0-5 1928 11.K 5674 43009M 11.0K 11.2K 7686.9 14.K 10.3K Memory latencies in nanoseconds - smaller is better (WARNING - may not be correct, check graphs) --------------------------------------------------- Host OS Mhz L1 $ L2 $ Main mem Guesses --------- ------------- ---- ----- ------ -------- ------- slovax.mc Linux 4.4.0-5 3497 0.889 2.6670 57.4

drudru11 10y ago

Kind of like hacker news posts.

js2 10y ago

Author somewhat addresses that in a comment: @Adrian Cockcroft: I didn't use lmbench because I was wondering how to write such a benchmark in the first place, and you always learn more by building things yourself than by using something already existing. My little benchmarks aren't supposed to be alternatives to lmbench, I simply provided the source code so others could unambiguously see how I did the benchmarks and reproduce them.