3 ms·
I think that costs have scaled with N^2 on every version of GPT meriting a new major version number and that Altman’s public statements around seven trillion ar
by benreesman 3y ago
I think that costs have scaled with N^2 on every version of GPT meriting a new major version number and that Altman’s public statements around seven trillion are almost exactly what the computer science would say it would cost to be on the wrong side of a polynomial at one more turn of the crank.
- HarHarVeryFunny 3y agoAltman's attempts to raise money for a new "$7T" chip consortium is exactly that - for a new chip consortium - this is NOT him trying to raise money for OpenAI's next training run (which is anyways already paid for and likely complete at this stage). I'm curious what "N" (what model measure is this?) you think spending is scaling in N^2 fashion with? And, what values of this N are you using for GPT-3, 4 and 5?