3 ms·

> 'The' flop is a misnomer because it's almost always the multiply-and-accumulate instruction: X = A + B * C. Which... Is two operations per instruction (per sh

> 'The' flop is a misnomer because it's almost always the multiply-and-accumulate instruction: X = A + B * C. Which... Is two operations per instruction (per shader/SIMD lane). Eeehhh whatever. Who cares about these details? If FMA is supported, it can either be counted as one or two operations, depending on the rule of the benchmark involved or the marketing of the processor. The marketing specification of a processor's theoretical peak performance sometimes counts a single-instruction FMA as two operations. On the other hand, for the purpose of code profiling, counting FMA as one operation is more realistic... As you said, who cares about these details?

fluoridation 3y ago
Given that the point of the FLOPS unit is to compare processors, it does make more sense to count complex instructions as more than a single floating-point operation. If one CPU could multiply a 4x4 matrix by a vector in a single instruction that can run a million times per second, and another CPU needed ~32 instructions and so can only multiply 500k matrices per second but retires 16 million instructions in that same second, it would be silly to compare instructions instead of multiplications and additions.
- dragontamer 3y ago
  As a computer-engineer, the circuit design needed to make a fast multiplication operation (ie: Wallace Tree, and similar) are an order-of-magnitude larger than the circuit design needed for fast addition (ie: a Kogge-Stone Carry lookahead Adder). This idea that additions and multiplications can be combined like this as "equivalent operations" is kinda bullshit. But hey, if its "how its done" (and its done this way because multiply-then-add is how you do matrix-multiplications...) then so be it. Just remember that this is an arbitrary subdivision of a matrix multiplication operation, that may not have much relevance as a benchmark outside of matrix multiplications.
  - fluoridation 3y ago
    It was just an example, not necessarily a realistic one. The point is that we want to compare how quickly a processor will compute our problem, not how many instructions it's going to execute. If it was a car you want to compare things like its top speed and acceleration, not something inane like engine revolutions per kilometer. You measure and compare things that are relevant to the user, not implementation details.
    - dragontamer 3y ago
      Okay, I think I see what you're saying now. I misinterpreted your statements earlier.
    - magicalhippo 3y ago
      > not something inane like engine revolutions per kilometer Unrelated to the thread but I just read that Top Fuel dragster engines only do about 240 revolutions over the 1000 foot race[1]. Never thought of it like that before so blew my mind a little. [1]: https://en.wikipedia.org/wiki/Top_Fuel#Performance https://en.wikipedia.org/wiki/Top_Fuel#Performance
- Findecanor 3y ago
  FLOPS is a measure of throughput. The time-difference between a `fmul` and a `fmadd` is often in latency. A modern processor is typically superscalar, with multiple individually pipelined floating point units that can each be issued one instruction per cycle: regardless of whether that is a `fmadd`, a `fadd or something else. If you'd issue a `fmadd` to a unit, then you are prevented from issuing a `fadd` to it in that same cycle, and vice versa. That's one op, not two. However, floating point units are sometimes heterogeneous. I think the question should be, rather: should you count the throughput of any ops, or of arbitrary ops. In other words, if a CPU runs at 1 MHz and has two units but where only one can run `fmul`, should you count the CPU as having one or two MFLOPS?
  - fluoridation 3y ago
    >In other words, if a CPU runs at 1 MHz and has two units but where only one can run `fmul`, should you count the CPU as having one or two MFLOPS? I don't think there's enough information to answer that question. For starters, even if the CPU was issued a long sequence of nothing but additions, it's possible that it might spend some fraction of that million cycles waiting on memory. I don't think FLOPS are counted from first principles, I think they're measured empirically using benchmarks. It's possible one benchmark will yield 1.25 MFLOPS and another 1.8 MFLOPS. Split the difference and call it a day.

fluoridation 3y ago

Given that the point of the FLOPS unit is to compare processors, it does make more sense to count complex instructions as more than a single floating-point operation. If one CPU could multiply a 4x4 matrix by a vector in a single instruction that can run a million times per second, and another CPU needed ~32 instructions and so can only multiply 500k matrices per second but retires 16 million instructions in that same second, it would be silly to compare instructions instead of multiplications and additions.

dragontamer 3y ago
As a computer-engineer, the circuit design needed to make a fast multiplication operation (ie: Wallace Tree, and similar) are an order-of-magnitude larger than the circuit design needed for fast addition (ie: a Kogge-Stone Carry lookahead Adder). This idea that additions and multiplications can be combined like this as "equivalent operations" is kinda bullshit. But hey, if its "how its done" (and its done this way because multiply-then-add is how you do matrix-multiplications...) then so be it. Just remember that this is an arbitrary subdivision of a matrix multiplication operation, that may not have much relevance as a benchmark outside of matrix multiplications.
- fluoridation 3y ago
  It was just an example, not necessarily a realistic one. The point is that we want to compare how quickly a processor will compute our problem, not how many instructions it's going to execute. If it was a car you want to compare things like its top speed and acceleration, not something inane like engine revolutions per kilometer. You measure and compare things that are relevant to the user, not implementation details.
  - dragontamer 3y ago
    Okay, I think I see what you're saying now. I misinterpreted your statements earlier.
  - magicalhippo 3y ago
    > not something inane like engine revolutions per kilometer Unrelated to the thread but I just read that Top Fuel dragster engines only do about 240 revolutions over the 1000 foot race[1]. Never thought of it like that before so blew my mind a little. [1]: https://en.wikipedia.org/wiki/Top_Fuel#Performance https://en.wikipedia.org/wiki/Top_Fuel#Performance
Findecanor 3y ago
FLOPS is a measure of throughput. The time-difference between a `fmul` and a `fmadd` is often in latency. A modern processor is typically superscalar, with multiple individually pipelined floating point units that can each be issued one instruction per cycle: regardless of whether that is a `fmadd`, a `fadd or something else. If you'd issue a `fmadd` to a unit, then you are prevented from issuing a `fadd` to it in that same cycle, and vice versa. That's one op, not two. However, floating point units are sometimes heterogeneous. I think the question should be, rather: should you count the throughput of any ops, or of arbitrary ops. In other words, if a CPU runs at 1 MHz and has two units but where only one can run `fmul`, should you count the CPU as having one or two MFLOPS?
- fluoridation 3y ago
  >In other words, if a CPU runs at 1 MHz and has two units but where only one can run `fmul`, should you count the CPU as having one or two MFLOPS? I don't think there's enough information to answer that question. For starters, even if the CPU was issued a long sequence of nothing but additions, it's possible that it might spend some fraction of that million cycles waiting on memory. I don't think FLOPS are counted from first principles, I think they're measured empirically using benchmarks. It's possible one benchmark will yield 1.25 MFLOPS and another 1.8 MFLOPS. Split the difference and call it a day.