4 ms·

Manhattan Project, that can be copied and duplicated for practically free. Interesting times.

• janalsncm 3y ago
  That’s not actually true at all. The basic hardware required to run even a copy of GPT-3 is outside of the budgets of all but the wealthiest companies and individuals. The engineering talent required to wire it all up is also extremely scarce. Training an original GPT-4 sized model would also cost on the order of hundreds of millions of dollars.
  • fatherzine 3y ago
    GPT3 is 800GB, which is about as large as the largest torrent files out there. GPT4 size is unknown, so I can't comment. While I haven't run the actual numbers (life is short), assuming a GPT3 torrent, my sense is that the cost of running it would be under $1M, possibly under $100k. Compared to (hundreds of?) $B for a state-level nuclear weapons program. But yes, technically I was wrong. It is not 'practically free', it is 'within the budget of e.g. a race boat enthusiast'.
    • janalsncm 3y ago
      That would be the cost of running GPT3 as-is, not to train a new model or hook it up to any live information. GPT4 has a much larger context window (16x larger), which suggests its file size would be at least 16x as large.