5 ms·

No, I don't deny it's important to know that cache misses exist, and what they can do: to the contrary, it's vital. However, in 90% of applications, it's not g

No, I don't deny it's important to know that cache misses exist, and what they can do: to the contrary, it's vital. However, in 90% of applications, it's not going to matter, because those applications are spending hundreds of cycles waiting anyways: Disk or network IO, user input, all that stuff is way slower than a cache miss. If you're writing a video game, or a database, or other software with very high soft-realtime speed requirements, or heavy data access, by all means, optimize to avoid cache miss. But if you're writing a company-internal Rails app, nobody's going to notice, even if you're getting cache miss after cache miss. Which you probably won't. Actually, if your language isn't compiled, a cache miss is the least of your worries, perf-wise. And now I've got to see if I can optimize my code to avoid cache misses. But the code's in Scheme, so unless the initial access cost is amortized, I'm already doomed...

yxhuvud 10y ago
Considering the Ruby object and memory models, you probably will get cache miss after cache miss. You just won't have the tools to do anything about them.
- qwertyuiop924 10y ago
  You're probably right. I was just thinking, "It can't possibly be THAT bad," especially seeing as there are so many objects that are referenced frequently.
Koromix 10y ago
You're looking at this problem backward. For example, you mention user input. Users may need a second to click or touch a button, but when they do the software should react instantly, and that does not leave you that many cycles. My smartphone's lock screen is my go-to example: most times it fails to follow my finger, and I barely have anything running on it. Most of the dynamic languages are data and instruction cache-miss machines. They chase objects and pointers all around the memory.
- qwertyuiop924 10y ago
  >My smartphone's lock screen is my go-to example: most times it fails to follow my finger, and I barely have anything running on it. ...That doesn't sound like a cache miss. Knowing Android, A cache miss is probably the least of your worries. >Users may need a second to click or touch a button, but when they do the software should react instantly, and that does not leave you that many cycles. You raise a good point... >Most of the dynamic languages are data and instruction cache-miss machines. They chase objects and pointers all around the memory. ...and this is part of my point. If you look at a problem and think, "A high-level language is fast enough," then you are implicitly saying that the latency of a cache miss is acceptable. And IME, in most cases that's true. I mean, heck, I'm using Scheme, so while I may have pointer chases like the Amazon has trees, but I CAN optimize them into array lookups, and my code is compiled: not great, but better than most HLLs. It's the same argument as always: perf vs. development speed. You can be in the C and FP loop, or the Lisp and JS loop.
  - Koromix 10y ago
    > That doesn't sound like a cache miss. Knowing Android, A cache miss is probably the least of your worries. My example was meant to illustrate the user input problem. From what I know about Android, the absymal performance is very much a case of "death from a thousand cuts". > It's the same argument as always: perf vs. development speed. You can be in the C and FP loop, or the Lisp and JS loop. The fast(er) languages we have are old and full of warts, and that makes them slow to develop in. The heavily used HLLs such as Python and Ruby were made by people who did not care much (at all?) about performance, and it shows in many design decisions. But here's the thing: we could have both at the same time. I don't buy this dichotomy.
    - deleted 10y ago
      [deleted]
    - qwertyuiop924 10y ago
      >But here's the thing: we could have both at the same time. I don't buy this dichotomy. That's actually not true: OO, dynamism, late binding, and a lot of the other things that HLLs have to offer require a lot of pointer chasing and non-consecutive datastructures. I'm mostly a Schemer, and Scheme and Lisp have had decades of research put into making them compile and run fast. Most dynamic languages aren't so lucky. But the required pointer chasing and garbage collection mean they'll never be as fast as C. Functional programming languages, however, are rarely late-binding, and don't expose as much about their implementation, so some of the pointer chasing can be avoided. Rust doesn't need a GC, and is fairly C-like - or rather, ALGOL-like and BLISS-like - with added memory safety. So with a programmer who knows what they're doing, it can be pretty fast. But here's the rub: the faster a language is, the closer it has to be to the metal, and the less it can do with high-level features. So yes, you can make HLLs faster, but you can't take the cache misses out of an HLL, and you can't make a systems language wearing an HLL's clothing - although Rust is making an admirable attempt.

yxhuvud 10y ago

Considering the Ruby object and memory models, you probably will get cache miss after cache miss. You just won't have the tools to do anything about them.

qwertyuiop924 10y ago
You're probably right. I was just thinking, "It can't possibly be THAT bad," especially seeing as there are so many objects that are referenced frequently.

Koromix 10y ago

You're looking at this problem backward. For example, you mention user input. Users may need a second to click or touch a button, but when they do the software should react instantly, and that does not leave you that many cycles. My smartphone's lock screen is my go-to example: most times it fails to follow my finger, and I barely have anything running on it. Most of the dynamic languages are data and instruction cache-miss machines. They chase objects and pointers all around the memory.

qwertyuiop924 10y ago
>My smartphone's lock screen is my go-to example: most times it fails to follow my finger, and I barely have anything running on it. ...That doesn't sound like a cache miss. Knowing Android, A cache miss is probably the least of your worries. >Users may need a second to click or touch a button, but when they do the software should react instantly, and that does not leave you that many cycles. You raise a good point... >Most of the dynamic languages are data and instruction cache-miss machines. They chase objects and pointers all around the memory. ...and this is part of my point. If you look at a problem and think, "A high-level language is fast enough," then you are implicitly saying that the latency of a cache miss is acceptable. And IME, in most cases that's true. I mean, heck, I'm using Scheme, so while I may have pointer chases like the Amazon has trees, but I CAN optimize them into array lookups, and my code is compiled: not great, but better than most HLLs. It's the same argument as always: perf vs. development speed. You can be in the C and FP loop, or the Lisp and JS loop.
- Koromix 10y ago
  > That doesn't sound like a cache miss. Knowing Android, A cache miss is probably the least of your worries. My example was meant to illustrate the user input problem. From what I know about Android, the absymal performance is very much a case of "death from a thousand cuts". > It's the same argument as always: perf vs. development speed. You can be in the C and FP loop, or the Lisp and JS loop. The fast(er) languages we have are old and full of warts, and that makes them slow to develop in. The heavily used HLLs such as Python and Ruby were made by people who did not care much (at all?) about performance, and it shows in many design decisions. But here's the thing: we could have both at the same time. I don't buy this dichotomy.
  - deleted 10y ago
    [deleted]
  - qwertyuiop924 10y ago
    >But here's the thing: we could have both at the same time. I don't buy this dichotomy. That's actually not true: OO, dynamism, late binding, and a lot of the other things that HLLs have to offer require a lot of pointer chasing and non-consecutive datastructures. I'm mostly a Schemer, and Scheme and Lisp have had decades of research put into making them compile and run fast. Most dynamic languages aren't so lucky. But the required pointer chasing and garbage collection mean they'll never be as fast as C. Functional programming languages, however, are rarely late-binding, and don't expose as much about their implementation, so some of the pointer chasing can be avoided. Rust doesn't need a GC, and is fairly C-like - or rather, ALGOL-like and BLISS-like - with added memory safety. So with a programmer who knows what they're doing, it can be pretty fast. But here's the rub: the faster a language is, the closer it has to be to the metal, and the less it can do with high-level features. So yes, you can make HLLs faster, but you can't take the cache misses out of an HLL, and you can't make a systems language wearing an HLL's clothing - although Rust is making an admirable attempt.