3 ms·

Allegedly, based on info from a DALL-E 2 paper, it takes about 100k-200k GPU hours to train the model: https://github.com/lucidrains/DALLE2-pytorch/issues/22#i

Allegedly, based on info from a DALL-E 2 paper, it takes about 100k-200k GPU hours to train the model: https://github.com/lucidrains/DALLE2-pytorch/issues/22#issuecomment-1108171293 https://github.com/lucidrains/DALLE2-pytorch/issues/22#issue...

p1esk 4y ago
Assuming those are V100 GPUs at $3/hr, that’s at least $300k.
- manquer 4y ago
  Relative to compensation for the developers and researchers working on projects like these this looks a small component. GPT was rumored to cost in the millions, perhaps the hours estimate is conservative?
  - agnosticmantis 4y ago
    And presumably one should also factor in the cost of experimentation before a good setting for the final training is found.
    - zdkl 4y ago
      How feasible is it to evaluate the quality of chosen parameters/settings without going through an extensive training? I mean <BIGNUM> GPU hours to discover your results are meh, jesus. (Small timer with absolutely no idea of practical ML here)
  - starnger 4y ago
    4.6 millions