> We also come up with a new version of BERT, called NumBERT, with improved numerical reasoning by replacing numbers in the pretraining text corpus with their s

> We also come up with a new version of BERT, called NumBERT, with improved numerical reasoning by replacing numbers in the pretraining text corpus with their scientific notation, which more readily exposes the magnitude to the model, and demonstrate that NumBERT representations capture scale significantly better than all those previous text representations. That's really cool. I like the writing in this post. Concise and dense with info.