3 ms·
It’s better to compare against an ML framework than to maximum theoretical flops because sometimes it’s not possible to reach. These models are often limited by
by tehsauce 4y ago
It’s better to compare against an ML framework than to maximum theoretical flops because sometimes it’s not possible to reach. These models are often limited by memory bandwidth rather than flop capability.
- pklausler 3y agoSomething more than 30 years ago, I had the privilege of working as a young compiler writer for [a supercomputer designer]'s penultimate start-up. I once naively asked him why the machine he was working on couldn't have more memory bandwidth, since the floating-point functional units were sometimes starved for operand data and it was hard to hit the peak Flop/sec figures. And his response has stuck with me ever since; basically, it makes better sense for the memory to be fully utilized, not the floating-point units, because the memory paths were way more expensive than the floating-point units. And this was something you could actually physically see through the transparent top of the system's case. I guess the lesson would be: Don't let a constraint that would be fairly cheap to overdesign be the limiting factor in a system's performance.