There isn't a lot of technical information in the Bloomberg article but I found this info from Tom's Hardware: Deepseek trained its DeepSeek-V3 Mixture-of-Expe

There isn't a lot of technical information in the Bloomberg article but I found this info from Tom's Hardware: Deepseek trained its DeepSeek-V3 Mixture-of-Experts (MoE) language model with 671 billion parameters using a cluster containing 2,048 Nvidia H800 GPUs in just two months, which means 2.8 million GPU hours, according to its paper. For comparison, it took Meta 11 times more compute power (30.8 million GPU hours) to train its Llama 3 with 405 billion parameters using a cluster containing 16,384 H100 GPUs over the course of 54 days. https://www.tomshardware.com/tech-industry/artificial-intelligence/chinese-ai-company-says-breakthroughs-enabled-creating-a-leading-edge-ai-model-with-11x-less-compute-deepseeks-optimizations-highlight-limits-of-us-sanctions https://www.tomshardware.com/tech-industry/artificial-intell...