https://www.mosaicml.com/blog/mosaicbert https://www.mosaicml.com/blog/mosaicbert I think an useful article for people who want to train BERT from scratch with

https://www.mosaicml.com/blog/mosaicbert https://www.mosaicml.com/blog/mosaicbert I think an useful article for people who want to train BERT from scratch with 20$ (in this case by renting GPUs). This model has also an actual good GLUE score.