3 ms·

~50-300 million sounds like an over-estimate, and the investment in GPT-3 is miniscule compared to SDC. Plus, GPT-3 generates value even without direct product

~50-300 million sounds like an over-estimate, and the investment in GPT-3 is miniscule compared to SDC. Plus, GPT-3 generates value even without direct product impact. I'm sure MSFT sales reps are already folding tons of nonsense about gpt-3 into their Azure pitches. Industry labs like OpenAI and DeepMind replace/augment the "Research & Development" model with a "Research & Marketing" model.

sillysaurusx 6y ago
Seems spot-on. One trick to estimate a startup's burn rate is to multiply their number of employees by $200k. It's not too accurate, but it's within the ballpark. So how many employees does OpenAI have? Supposing they have 500, that's a burn rate of $100M/yr. 250 employees, $50M/yr. 100 employees, $20M/yr.
- tehsauce 6y ago
  Gpt-3 is just one of openai's many projects. Theres ~20 authors on the paper, and they all almost certainly did not spend a full year on this project. So 300m is completely wrong.
- coolspot 6y ago
  I believe significant share of GPT-3 cost is machine-hours that were spent training this model - months of hundreds top-tier NVidia machines. Edit: estimates range from $2-$5MM to $15MM https://www.reddit.com/r/MachineLearning/comments/hwfjej/d_the_cost_of_training_gpt3/ https://www.reddit.com/r/MachineLearning/comments/hwfjej/d_t...
  - zaptrem 6y ago
    I thought MS was giving them Azure GPU instances for free?
    - coolspot 6y ago
      “Free”, which is basically Microsoft, as an investor, has paid those $4-$15MM minus ~30% of margin they make over pure operational costs. Electricity was burned and NVidia GPUs that could have been doing paid job were doing GPT-3 training instead.
      - jychang 6y ago
        $15mm is a rounding error for MSFT