What's interesting to note is that the "intelligence" labs can squeeze out of an H100, an almost 4 year old GPU, is dramatically higher than what they got out o

What's interesting to note is that the "intelligence" labs can squeeze out of an H100, an almost 4 year old GPU, is dramatically higher than what they got out of it in 2022. It hints that once these labs get a good enough "everyday model", they can work on efficiency so they can serve these models on old hardware. Which is almost certainly already happening.