4 ms·
To be fair, with a multi-terabyte model, the number of people who have the money to make use of the model is probably only a few hundred...
by londons_explore 6y ago
To be fair, with a multi-terabyte model, the number of people who have the money to make use of the model is probably only a few hundred...
- dx034 6y agoRather a few thousands at least. Most larger companies have that amount of data (if they can use the model is another question).
- londons_explore 6y agoYep, but running inference on it at any reasonable performance requires you to have all of it in GPU RAM - Ie. you need a cluster of ~100 high performance GPU's.
- trsohmers 6y agoThe largest version of GPT-3 is 175B parameters, which is ~350GB. I frequently use 8x and 10x RTX8000 boxes (and can access a 16x JBOG system as well), and the 8x system would have 384GB of VRAM. These sort of systems start off at only ~$60k.