3 ms·
> Is this the current standard in the field? It’s becoming pretty common, yeah. The two things you mentioned: training particulars and dataset mixture are also
by ShamelessC 3y ago
> Is this the current standard in the field?
It’s becoming pretty common, yeah. The two things you mentioned: training particulars and dataset mixture are also basically the only competitive advantage companies have. Since the code/architecture is trivial to reproduce, anyone with enough money can make a competing model “easily”.
OpenAI started this trend and cemented it with GPT4’s “technical report” which didn’t even specify the number of parameters in the model. They’ve been historically vague about their dataset for far longer than that though.