2 ms·

The precision doesn't matter much on the CPU (roughly half the throughout if you're doing a lot of them). You have a data dependency on the arctangent, and tran

by hansvm 4y ago

The precision doesn't matter much on the CPU (roughly half the throughout if you're doing a lot of them). You have a data dependency on the arctangent, and transcendentals are slow anyway, so you're not going to have a good time. You can even compute the sin and cos together and still expect a bare minimum latency of 50 clock cycles for the arctangent followed by 50 for the sincos, usually implemented with a variable speed algorithm that might be slower (i.e., lower bound of 30ns of latency on a moderately fast single core, just for the trig). Throughput can be a bit higher with vectorization and pipelining, but more or less any such optimization you can make to the transcendentals you can also make on the proposed solution.

dahart 4y ago
Thanks, good to know! So that does suggest to me that if there are other threads in progress, the actual impact to the runtime in practice probably would be lower with fp32 - the half throughput rate does tell us how long we wait for any instructions dependent on the operation, right? I’m saying I suspect the total latency can be filled with other work, so this isn’t really 40ns that is fully devoted to atan2+sincos and nothing else. Is that accurate? (On the GPUs I use, trig functions use a different math pipe than FMA, and so they can be ‘free’ if you don’t use too many of them and there is enough logic or FMA to hide the trig latency.) Aren’t the benefits of using fp32 bigger than the trig instruction throughput too? Doubles eat more register space, and double the bandwidth to memory if anything ends up in memory. I just don’t see any reason whatsoever to use doubles for this operation. If we’re going to worry about nanoseconds, then these extra bytes matter too, no?
- hansvm 4y ago
  The throughput comment was just that we have fixed-width vector instructions which support being filled with and doing trig on either 32-bit or 64-bit floats. They both occupy the same floating point execution units (mostly, see the AMD bulldozer/steamdriver/... fiasco from a few years ago), so you're either filling them with N doubles or 2N floats. You'd be hard-pressed to squeeze other useful work into that space. The actual difference between float/double in the amount of work done is otherwise miniscule and mostly related to teasing out the extra precision. With fast-converging algorithms it doesn't wind up mattering much in terms of total latency. "Latency" is what I was referring to with respect to how long you have to wait to execute a data-dependant instruction. "Throughput" refers to how many you can have in flight and how fast those execute. Latency _can_ be filled with other work, but as I mentioned, it can be filled better with other work using the faster non-trig solution. Rule of thumb, you won't squeeze in more than 4 in-flight instructions per core, so you're probably not going to eek out more 10ns average throughput for the trig solution, worse than the proposed optimization even assuming it couldn't be pipelined at all. You're right that float/int execution units are usually separate, but they do have some shared resources. If you're doing enough logic relative to these computations then yes these could sometimes be basically free. The register/cache/RAM/disk space and bandwidth very well might wind up dominating every other consideration. As always, it depends what else you're doing and what exactly you're doing with these computations. To give a useful counter-example, consider matrix multiplication. It can be structured such that even if you have to read/write the matrices from disk because of their size it's still a CPU-bound operation because you can efficiently use each data access many times. In that case you really do just have typically a 2x slowdown for doubles because you have half the elements per vector. Might f32 be sufficient for this problem? Almost certainly IMO. It might not matter though (or it might -- as always, measure).
  - dahart 4y ago
    FWIW, for a game controller, I’m absolutely certain that f32 is already overkill and that a half float or even a byte per axis, u16/u8, is sufficient. The only reason to need f32 or f64 is if you’re rounding direction vectors for some other purpose than a game controller. But in graphics we frequently use f16 for direction vectors even when f32 is needed for positions. Normals and unit sphere directions usually require less precision than other operations. Using f64 for a game controller stick input isn’t justifiable, even if it isn’t measurable in code speed, I don’t think it would fly or pass code review in a real game engine. My next concern would be whether using doubles willy-nilly leads to the compiler spilling registers to memory more often. That could easily undo the 40ns optimization, and it would be difficult to measure or blame the direction rounding code, it would diffuse the cost to other parts of the code.

by hansvm 4y ago

dahart 4y ago

Thanks, good to know! So that does suggest to me that if there are other threads in progress, the actual impact to the runtime in practice probably would be lower with fp32 - the half throughput rate does tell us how long we wait for any instructions dependent on the operation, right? I’m saying I suspect the total latency can be filled with other work, so this isn’t really 40ns that is fully devoted to atan2+sincos and nothing else. Is that accurate? (On the GPUs I use, trig functions use a different math pipe than FMA, and so they can be ‘free’ if you don’t use too many of them and there is enough logic or FMA to hide the trig latency.) Aren’t the benefits of using fp32 bigger than the trig instruction throughput too? Doubles eat more register space, and double the bandwidth to memory if anything ends up in memory. I just don’t see any reason whatsoever to use doubles for this operation. If we’re going to worry about nanoseconds, then these extra bytes matter too, no?

hansvm 4y ago
The throughput comment was just that we have fixed-width vector instructions which support being filled with and doing trig on either 32-bit or 64-bit floats. They both occupy the same floating point execution units (mostly, see the AMD bulldozer/steamdriver/... fiasco from a few years ago), so you're either filling them with N doubles or 2N floats. You'd be hard-pressed to squeeze other useful work into that space. The actual difference between float/double in the amount of work done is otherwise miniscule and mostly related to teasing out the extra precision. With fast-converging algorithms it doesn't wind up mattering much in terms of total latency. "Latency" is what I was referring to with respect to how long you have to wait to execute a data-dependant instruction. "Throughput" refers to how many you can have in flight and how fast those execute. Latency _can_ be filled with other work, but as I mentioned, it can be filled better with other work using the faster non-trig solution. Rule of thumb, you won't squeeze in more than 4 in-flight instructions per core, so you're probably not going to eek out more 10ns average throughput for the trig solution, worse than the proposed optimization even assuming it couldn't be pipelined at all. You're right that float/int execution units are usually separate, but they do have some shared resources. If you're doing enough logic relative to these computations then yes these could sometimes be basically free. The register/cache/RAM/disk space and bandwidth very well might wind up dominating every other consideration. As always, it depends what else you're doing and what exactly you're doing with these computations. To give a useful counter-example, consider matrix multiplication. It can be structured such that even if you have to read/write the matrices from disk because of their size it's still a CPU-bound operation because you can efficiently use each data access many times. In that case you really do just have typically a 2x slowdown for doubles because you have half the elements per vector. Might f32 be sufficient for this problem? Almost certainly IMO. It might not matter though (or it might -- as always, measure).
- dahart 4y ago
  FWIW, for a game controller, I’m absolutely certain that f32 is already overkill and that a half float or even a byte per axis, u16/u8, is sufficient. The only reason to need f32 or f64 is if you’re rounding direction vectors for some other purpose than a game controller. But in graphics we frequently use f16 for direction vectors even when f32 is needed for positions. Normals and unit sphere directions usually require less precision than other operations. Using f64 for a game controller stick input isn’t justifiable, even if it isn’t measurable in code speed, I don’t think it would fly or pass code review in a real game engine. My next concern would be whether using doubles willy-nilly leads to the compiler spilling registers to memory more often. That could easily undo the 40ns optimization, and it would be difficult to measure or blame the direction rounding code, it would diffuse the cost to other parts of the code.