3 ms·

Microsoft releases code to enable training BERT at large scale on commodity GPUs

• TheIronYuppie 7y ago
  Disclosure: I work at Microsoft/Azure on ML Really excited to announce the release of the code we use to train/fine-tune BERT at Azure & Bing. Please let me know if you have any questions!
• nonfamous 7y ago
  The actual code is in this repo: https://github.com/microsoft/AzureML-BERT https://github.com/microsoft/AzureML-BERT
• maxluk 7y ago
  If you have any questions, please let us know. maxluk [MSFT]
• kdatta1 7y ago
  seems like the scripts have a dependency on pytorch_pretrained_bert library. Is there a way to run the dataprep/create_pretraining.py or train.py scripts on-premise?
  • aagarg 7y ago
    The recipes are optimized for AzureML but the implementation can be adapted to any cluster. --aagarg [MSFT]