4 ms·
Yeah it wouldn't fit. GPT3 is 175B params, so even if you use 8 bit for each weight, you need 175×10^9÷2^30 = 163GiB of memory.
by madiator 4y ago
Yeah it wouldn't fit. GPT3 is 175B params, so even if you use 8 bit for each weight, you need 175×10^9÷2^30 = 163GiB of memory.
- cal85 4y agoWouldn’t that be possible with about 4 powerful GPUs? Or does it not work like that?
- taocoyote 4y agoPossibly, but that would be 10 of thousands of dollars worth of GPUs.
- kurtoid 4y agoSilly question: how does OpenAI host/serve it?
- magixx 4y agoI think on professional hardware you can get 80G of memory per GPU and they can likely do memory pooling.
- joshka 4y agohttps://www.reddit.com/r/ChatGPT/comments/zhzjpq/comment/izouwt0/?context=3 https://www.reddit.com/r/ChatGPT/comments/zhzjpq/comment/izo... >It's around 500gbs and requires around 300+gbs of vram from my understanding and runs on one of the largest super computers in the world. Sable diffusion has around 6 billion parameters gpt-3/chatgpt has 175 billion.