5 ms·
What is a flop?
- jjgreen 3y agoI'm usually a fan of Higham, but the last few posts have been weak.
- deleted 3y ago[deleted]
- namirez 3y agoFloating point operations per …?
- GenericDev 3y agoFloating Point Operations Per Second [1] [1] https://academickids.com/encyclopedia/index.php/FLOPS https://academickids.com/encyclopedia/index.php/FLOPS
- Zambyte 3y ago> One should speak in the singular of a FLOPS and not of a FLOP, although the latter is frequently encountered. The final S stands for second and does not indicate a plural. The author of this post seems to have fallen for this error.
- dataflow 3y agoI think their point was that the p stood for "per", not for the second letter of "operation".
- FabHK 3y agoFLOP = FLoating point OPeration, akin to the work that needs to be done (which could be measured in Joule) FLOPS = FLoating point Operations Per Second, akin to the power of a given machine (which could be measured in Watt = Joule per second) A task of W FLOP on a machine capable of P FLOPS will take T=W/P seconds. Or, a machine capable of executing a task of W FLOP in T seconds is capable of P=W/T FLOPS. TL;DR: 1 FLOPS = 1 FLOP/sec
- dragontamer 3y agoMy personal notes on this subject: * MIPS was perhaps the integer equivalent to FLOP, still used in modern microcontrollers because the 8051 at 12MHZ would only execute 1MIPS (12 clocks per instruction). Modern 8051 chips obviously have sped up to 1 clock per instruction, but MIPS (and Dhrystone MIPS in particular) are still a common benchmark today. * FLOPs is very difficult to calculate in theory because modern CPUs have vector units, and multiple pipelines per core. You could have 3x AVX512 instructions in parallel on today's CPUs on a single core. * FLOPs we're traditionally a 64-bit operation for the supercomputer community. Today, most FLOPs are 32-bit for video games. Finally, the deep learning / neural net guys have popularized 16-bit flops, and even 8-bit iops. * 'The' flop is a misnomer because it's almost always the multiply-and-accumulate instruction: X = A + B * C. Which... Is two operations per instruction (per shader/SIMD lane). Eeehhh whatever. Who cares about these details? * As 'Dhrystone' is the benchmark for MIPS, the benchmark for 64-bit flops is Linpack.
- segfaultbuserr 3y ago> 'The' flop is a misnomer because it's almost always the multiply-and-accumulate instruction: X = A + B * C. Which... Is two operations per instruction (per shader/SIMD lane). Eeehhh whatever. Who cares about these details? If FMA is supported, it can either be counted as one or two operations, depending on the rule of the benchmark involved or the marketing of the processor. The marketing specification of a processor's theoretical peak performance sometimes counts a single-instruction FMA as two operations. On the other hand, for the purpose of code profiling, counting FMA as one operation is more realistic... As you said, who cares about these details?
- fluoridation 3y agoGiven that the point of the FLOPS unit is to compare processors, it does make more sense to count complex instructions as more than a single floating-point operation. If one CPU could multiply a 4x4 matrix by a vector in a single instruction that can run a million times per second, and another CPU needed ~32 instructions and so can only multiply 500k matrices per second but retires 16 million instructions in that same second, it would be silly to compare instructions instead of multiplications and additions.
- yujian 3y agothis
- nightmonkey 3y agoMy last two startups.
- HPsquared 3y agoYou could use it as a unit of accounting for investing losses. A gigaflop would be a loss of 1 billion, etc.
- AlbertCory 3y agoDisappointed this is not about basketball.
- Dwedit 3y agoBASEketball?
- paulddraper 3y agoFloating Point Operations Per
- MattyDub 3y agoCan somebody explain why a square root is also considered a flop? Surely that involves more work than the other four operations the article listed. Is there some hardware algorithm for the square root that is as fast as (e.g.) division?
- jjgreen 3y agoOn x86-64, there are instructions, see https://stackoverflow.com/a/54642811/468334 https://stackoverflow.com/a/54642811/468334
- stephencanon 3y agoSquare root is pretty much equivalent to division in complexity, and computed by similar techniques (digit-by-digit methods or newton-raphson or goldschmidt iterations). Division is often a little more efficient, but square root has fewer messy edge cases (it never overflows nor underflows). Division and square root are generally slower than the other arithmetic operations, in both latency and throughput. They are finally partially pipelined in recent CPUs (a result every two or three cycles), but were totally unpipelined in mainstream designs for many years before that. A decade ago, they might take a few tens of cycles, now they’re generally somewhere around ten cycles latency on “real” CPUs, vs 3-5 cycles latency for the other floating point arithmetic instructions.
- herodoturtle 3y agoRelevant Quake engine trivia, for those that are interested: https://en.m.wikipedia.org/wiki/Fast_inverse_square_root https://en.m.wikipedia.org/wiki/Fast_inverse_square_root
- somat 3y agoA question that I am not quite smart enough to figure out. The fast inverse square root works via a giant hack. hand waving a bit here but it has to do with because floats are stored as an exponential structure bit shifting one does magic math things to it. however note that bit shifting a float is not an implemented instruction. so you have to hack it by casting to an int. Anyway the question is. If you implemented fast inverse square root with out the hack. (i don't know, perhaps packing your own bits, or it might be screwball enough you only assembly could do it.) would it be as fast?
- satvikpendem 3y agoThere is no flop, it's flops even in the singular because it's an acronym.
- hinkley 3y agoA flop is something footballers do. flops are what CPUs do.
- ggm 3y agoIt was always an approximation to the real world computing need. a single instruction is how most people thought of flop or mips but it had to be a blend over the range of cost behind implementing a task, the ALU single thread/lock cost, and the problemspace. If you do exclusively square roots over a giant matrix, being told the FLOP is based on a single 64 bit float division might not help you. (I cannot conceive of a problem solved only in square roots. This is a hyperbolic/reductive argument) Given the distinction of single/parallel a lot of FLOPS stories and MIPS stories were somewhat mythic anyway. Then manufacturers started gaming the test suites.. it was always very notional. I ask, if IOPS is in fact any better? or SQL transaction models? I might be stuck in a problem which is a LEFT OUTER JOIN do I not need to know that cost distinct from a well optimised single B Tree index cost? I worked on an 11/780. Many in fact, both as an operator and as a user. It was awesome. But, I reminded myself the Dec-10 alongside had happily been supporting 70-80 users in parallel, running MUD, being a sort-of mainframe, I had no conscious sense of how "big" an IBM 360 was by comparison. A Vax 11/780 was however, really quite fast at processing the same FORTRAN jobs which ran on the Dec-10. The Dec-10 had a floating-point systems "ice cream cabinet" glued on the side as an array processor. It ran tasks compiled to run on it very very fast but that was a narrow problemspace. This was 1982-3.
- fragmede 3y agoLooking at today, the Tensor-TFLOPS is the unit of measurement that Nvidia is trying to push as the relevant benchmark for hardware performance with ML models. https://blog.paperspace.com/understanding-tensor-cores/amp/ https://blog.paperspace.com/understanding-tensor-cores/amp/
- mensetmanusman 3y agoHow many flops for a full qcd atomistic simulation of a human in real time?