5 ms·

Why is this even a thing? Copying and the like is such a common operation. Why don't chip providers offer a single instruction that gets decoded to the absolute

Why is this even a thing? Copying and the like is such a common operation. Why don't chip providers offer a single instruction that gets decoded to the absolute fastest way the chip can do? That'd even allow them to, maybe, do some behind-the-scenes optimization, bypassing caches or something. It's painful that such a common operation needs highly specialized code. I know you can just REP an operation but apparently CPUs don't optimize this the same way. This is too obvious an issue, so there must be a solid reason. What is it?

corysama 10y ago
http://stackoverflow.com/questions/8858778/why-are-complicated-memcpy-memset-superior http://stackoverflow.com/questions/8858778/why-are-complicat... counter claims https://news.ycombinator.com/item?id=8683565 https://news.ycombinator.com/item?id=8683565
- MichaelGG 10y ago
  Ah that thread is great and answers my question. TL;DR seems to be that rep+mov is as fast as anything now.
Tloewald 10y ago
This seems like something that compilers should do and CPU instruction sets should not.
- nostrademons 10y ago
  It's really something the stdlib should do, which, when I've seen it implemented, is usually what happens. The compiler should just provide good inlining support, so that if eg. you include the short-string optimization in your stdlib, the compiler can optimize it down to a couple bit operations, a test, and a word copy. If the test fails and your string is more than 7 bytes, it's perfectly fine to call a function - the function call overhead is usually dwarfed by the copy loop for large strings. And then if new hardware comes out and you vectorize it differently, you can get away with replacing that one function in the stdlib instead of recompiling every single program in existence.
- MichaelGG 10y ago
  Why? It's a common op that requires internal knowledge of every microarchitecture, isn't it? Seems like something that should be totally offloaded to the CPU so you're guaranteed best performance.
  - anjc 10y ago
    The message you were referring to was talking about code for copying strings. If you wanted an instruction to copy lots of strings, the CPU would need to know what a character is (which could be 7 bits, 8 bits, 16 or 32), what a string is, how it's terminated, what ascii and unicode is, be able to allow new character encoding standards etc etc. Then you would need other instructions for other high level datatypes. That's not what CPUs do, because you're limited by how much more logic/latency you can add to an architecture, how many distinct instructions you can implement with the bits available per instruction, how many addressing modes you want etc. So instead, this information/knowledge about high level data types is encapsulated by standard libraries and then the compiler below that. Most CPUs have single instructions to copy a chunk of data from somewhere to somewhere else and a nice basic way to repeat this process efficiently, and it's up to the compiler to use this.
taneq 10y ago
I remember way back in the DOS days there were a lot of clever things you could do with setting up DMA transfers which would then run in the background... With CPU/memory latency and bandwidth being so much more of an issue these days I can't see why this isn't still the case. Maybe it's all done automatically in the background now?
- creshal 10y ago
  DMA is done automatically by modern hardware, PCIe(/Thunderbolt) and SATA just do it. Somewhat related is e.g. the sendfile() syscall that's used by web servers/frameworks to pass a file directly from your fcgi application to the outgoing socket.
wbl 10y ago
The CPU has a limited amount of silicon to spend on instruction decoding. That silicon is also part of every single instructions issue latency. Machines like the Cray with complex internal operations paid the cost in issue delays, and most workloads won't earn them back.
nickzoic 10y ago
In the olden days of x86: "REPNZ SCASB" to get the length of a zero-terminated string and "REP MOVSB" to copy bytes from place to place. But I think more modern CPUs actually work faster with the RISCier equivalents.
kragen 10y ago
See http://yosefk.com/blog/its-done-in-hardware-so-its-cheap.html http://yosefk.com/blog/its-done-in-hardware-so-its-cheap.htm..., "It's done in hardware so it's cheap". Summary: 'Why doesn't "hardware support" automatically translate to "low cost"/"efficiency"? The short answer is, hardware is an electric circuit and you can't do magic with that, there are rules.'

corysama 10y ago

http://stackoverflow.com/questions/8858778/why-are-complicated-memcpy-memset-superior http://stackoverflow.com/questions/8858778/why-are-complicat... counter claims https://news.ycombinator.com/item?id=8683565 https://news.ycombinator.com/item?id=8683565

MichaelGG 10y ago
Ah that thread is great and answers my question. TL;DR seems to be that rep+mov is as fast as anything now.

Tloewald 10y ago

This seems like something that compilers should do and CPU instruction sets should not.

nostrademons 10y ago
It's really something the stdlib should do, which, when I've seen it implemented, is usually what happens. The compiler should just provide good inlining support, so that if eg. you include the short-string optimization in your stdlib, the compiler can optimize it down to a couple bit operations, a test, and a word copy. If the test fails and your string is more than 7 bytes, it's perfectly fine to call a function - the function call overhead is usually dwarfed by the copy loop for large strings. And then if new hardware comes out and you vectorize it differently, you can get away with replacing that one function in the stdlib instead of recompiling every single program in existence.
MichaelGG 10y ago
Why? It's a common op that requires internal knowledge of every microarchitecture, isn't it? Seems like something that should be totally offloaded to the CPU so you're guaranteed best performance.
- anjc 10y ago
  The message you were referring to was talking about code for copying strings. If you wanted an instruction to copy lots of strings, the CPU would need to know what a character is (which could be 7 bits, 8 bits, 16 or 32), what a string is, how it's terminated, what ascii and unicode is, be able to allow new character encoding standards etc etc. Then you would need other instructions for other high level datatypes. That's not what CPUs do, because you're limited by how much more logic/latency you can add to an architecture, how many distinct instructions you can implement with the bits available per instruction, how many addressing modes you want etc. So instead, this information/knowledge about high level data types is encapsulated by standard libraries and then the compiler below that. Most CPUs have single instructions to copy a chunk of data from somewhere to somewhere else and a nice basic way to repeat this process efficiently, and it's up to the compiler to use this.

taneq 10y ago

I remember way back in the DOS days there were a lot of clever things you could do with setting up DMA transfers which would then run in the background... With CPU/memory latency and bandwidth being so much more of an issue these days I can't see why this isn't still the case. Maybe it's all done automatically in the background now?

creshal 10y ago
DMA is done automatically by modern hardware, PCIe(/Thunderbolt) and SATA just do it. Somewhat related is e.g. the sendfile() syscall that's used by web servers/frameworks to pass a file directly from your fcgi application to the outgoing socket.

wbl 10y ago

The CPU has a limited amount of silicon to spend on instruction decoding. That silicon is also part of every single instructions issue latency. Machines like the Cray with complex internal operations paid the cost in issue delays, and most workloads won't earn them back.

nickzoic 10y ago

In the olden days of x86: "REPNZ SCASB" to get the length of a zero-terminated string and "REP MOVSB" to copy bytes from place to place. But I think more modern CPUs actually work faster with the RISCier equivalents.

kragen 10y ago

See http://yosefk.com/blog/its-done-in-hardware-so-its-cheap.html http://yosefk.com/blog/its-done-in-hardware-so-its-cheap.htm..., "It's done in hardware so it's cheap". Summary: 'Why doesn't "hardware support" automatically translate to "low cost"/"efficiency"? The short answer is, hardware is an electric circuit and you can't do magic with that, there are rules.'