3 ms·
The cost of the next number in a GPT (3>4>5) seems to be in 2 ways: 1) $$$ 2) data The second (data) also isn't cheap. As it seems we've already gotten throu
by Balgair 1y ago
The cost of the next number in a GPT (3>4>5) seems to be in 2 ways:
1) $$$
2) data
The second (data) also isn't cheap. As it seems we've already gotten through all the 'cheap' data out there. So much so that synthetic data (fart huffing) is a big thing now. People tell it's real and useful and passes the glenn-horf theore... blah blah blah.
So it really more so comes down to just:
1) $$$^2 (but really pick any exponent)
In that, I'm not sure this thing is a true sigmoid curve (see: biology all the time). I think it's more a logarithmic cost here. In that, it never really goes away, but it gets really expensive to carry out for large N.
[To be clear, lots of great shit happens out there in large N. An AI god still may lurk in the long slow slope of $N, the cure for boredom too, or knowing why we yawn, etc.]