2 ms·
My gut has the same feeling, but to put numbers on it, for a 1.76 trillion parameter model... - It takes 10s of millions of dollars in GPU time for training?
by consumer451 3y ago
My gut has the same feeling, but to put numbers on it, for a 1.76 trillion parameter model...
- It takes 10s of millions of dollars in GPU time for training?
- Curation of data to train on
- Maybe 10s of thousands of man hours for reinforcement?
- How many lines of code are written for the nets and data pipelines?
Does anyone have any insight on these numbers?
- adastra22 3y ago100M for GPT-4 I think. They would be in a strong position to negotiate a discount on that GPU bill though.