3 ms·
> Seems like wishful thinking to think decentralized open source can beat centralized models that cost 100M+ to train! Because surely price = quality. Solid ar
by lapinot 3y ago
> Seems like wishful thinking to think decentralized open source can beat centralized models that cost 100M+ to train!
Because surely price = quality. Solid argumentation there.
- Mike_12345 3y agoYes, price = quality because they require supercomputing resources to train. GPT-3 required hundreds of Tesla GPUs running for several weeks. That's millions of dollars just for hardware, not including power (the GPUs cost $15k each)
- sashank_1509 3y agoYou’re right but I’d just like to add, GPT-3 probably required 1000’s of GPUs. OpenAI is known to have the largest cluster 16k+ A100 GPUs and most of them were used for the major model training.