4 ms·
Microsoft, in this case, is pouring billions into this, and the industry at large, from Alphabet to LAION is in an arms race. They will optimize it beyond belie
by ly3xqhl8g9 4y ago
Microsoft, in this case, is pouring billions into this, and the industry at large, from Alphabet to LAION is in an arms race. They will optimize it beyond belief, x100 is rather low, come to think of it. I fully expect to be able to run your own ChatGPT on one DGX H100 unit (640GB GPU memory) in 18 months or less.
- visarga 4y ago> GLM-130B, a model comparable with GPT-3, has 130 billion parameters in FP16 precision, a total of 260G of GPU memory is required to store model weights. The DGX-A100 server has 8 A100s and provides an amount of 320G of GPU memory (640G for 80G A100 version) so it suits GLM-130B well. https://github.com/THUDM/GLM-130B/blob/main/docs/low-resource-inference.md https://github.com/THUDM/GLM-130B/blob/main/docs/low-resourc...
- ly3xqhl8g9 4y agoThere is surely lots of room for improvement [1]. [1] "Use GPT-3 incorrectly: reduce costs 40x and increase speed by 5x", https://news.ycombinator.com/item?id=34702349 https://news.ycombinator.com/item?id=34702349