3 ms·
Depends. The only paper they cite for training: https://arxiv.org/pdf/2310.11453.pdf https://arxiv.org/pdf/2310.11453.pdf doesn't improve training costs much an
by llm_trw 3y ago
Depends. The only paper they cite for training: https://arxiv.org/pdf/2310.11453.pdf https://arxiv.org/pdf/2310.11453.pdf doesn't improve training costs much and most models are already training constrained. Not everyone has $200m to throw at training another model from scratch.