From 2019: https://heartbeat.fritz.ai/deep-learning-has-a-size-problem-ea601304cd8 https://heartbeat.fritz.ai/deep-learning-has-a-size-problem-... > Earlier th

From 2019: https://heartbeat.fritz.ai/deep-learning-has-a-size-problem-ea601304cd8 https://heartbeat.fritz.ai/deep-learning-has-a-size-problem-... > Earlier this year, researchers at NVIDIA announced MegatronLM, a massive transformer model with 8.3 billion parameters (24 times larger than BERT) > The parameters alone weigh in at just over 33 GB on disk. Training the final model took 512 V100 GPUs running continuously for 9.2 days. Running this model on a "regular" machine at some useful rate is probably not possible at this time.