There’s also articles for pre-training BERT models on hardware resources a small lab could afford. Those are still useful, too, even if not highly competitive.

There’s also articles for pre-training BERT models on hardware resources a small lab could afford. Those are still useful, too, even if not highly competitive. So, they could still have value for low-cost, small, model development.