4 ms·
What's the cost of shuttling data in and out of SIMD land?
by jancsika 2y ago
What's the cost of shuttling data in and out of SIMD land?
- School-Cotton 2y agoSIMD doesn’t operate on a separate memory space or anything like that. You just load data from normal memory into the SIMD registers, just like you would have to load it into the scalar registers if you wanted to operate on it with normal instructions.
- aidenn0 2y agoOn some targets you need to overalign data for vectorization.
- astrange 2y agoIt is slow to move data from SIMD to scalar registers, or can be.
- TinkersW 2y agoIt depends, for SIMD float-> scalar floats it is fast as they operate on the same registers. If pulling out of lane 0 you don't even need to do anything(just a type cast). For other lanes you need a shuffle. For SIMD integer to scalar integer, it has to move into separate register, so there is some short penalty(3 cycles iir).
- kccqzy 2y agoIt's pretty cheap. You can easily find the latency and throughput numbers on different Intel architectures. Here's an example for movdqa: https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html#ig_expand=4029,0,4029&text=_mm_load_si128 https://www.intel.com/content/www/us/en/docs/intrinsics-guid... which is a basic 128-bit load. Even a 512-bit load isn't much more expensive: https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html#text=_mm512_load_epi64&ig_expand=4006,3993 https://www.intel.com/content/www/us/en/docs/intrinsics-guid...