3 ms·
Transformers were known that they kept scaling up with more parameters and more training data so if the Open AI hit the limits of this scaling that would be a v
by bitL 4y ago
Transformers were known that they kept scaling up with more parameters and more training data so if the Open AI hit the limits of this scaling that would be a very important milestone in AI.
- fastball 4y agoI think performance will keep scaling, the main problem is cost. If you want to feed longer strings of tokens in, the price is going to keep going up.