4 ms·
So would it be a 42 trillion parameter model, because that's how many tokens there are in the training data?
by alightsoul 16d ago
So would it be a 42 trillion parameter model, because that's how many tokens there are in the training data?
- k__ 16d agoDoes that mean, it's not compressed anymore?
- robotresearcher 16d agoNo, the parameters are updated online as new run-time data comes in. Infinite different param values over time, but a constant number of params. It's a stretch really, since by this usage a one-parameter model that updates itself is also infinite over time...