> I just know that model optimization gains are much less guaranteed than say, FLOP density, even though model optimization has so far provided way more gains t

> I just know that model optimization gains are much less guaranteed than say, FLOP density, even though model optimization has so far provided way more gains than hardware advancements. Performance gained from model improvements has outpaced performance gained from hardware improvements for decades.