3 ms·
How much RAM/GPU/CPU is required to run the chatGPT / GPT3 model (aka text-davinci-003)?
by greenSunglass 4y ago
How much RAM/GPU/CPU is required to run the chatGPT / GPT3 model (aka text-davinci-003)?
- ilaksh 4y agoI'm using the text-davinci-003 model with the open AI API. It seems equivalent to chatGPT in terms of coding tasks. And the pricing is totally reasonable.
- infinityio 4y agoAs far as I know Davinci is supposedly a variant of gpt3-175B, so likely 175B parameters, so likely in the hundreds of gigabytes of ram unfortunately
- zeknife 4y agoIt is not available to run locally, but like the other comments said, an equivalent model would require an inordinate amount of VRAM.
- macrolime 4y agoIt used to be 2x number of parameters, so about 350GB. Facebook came with an optimization a couple months ago so it takes only half the memory, so about 175GB for GPT-3, not sure if OpenAI has implemented that yet. BLOOM, which is around the size of GPT-3, though not as good currently, works with these optimizations and should then be possible to run with 8x 24GB consumer GPUs. Huggin face has a nice blog post on this https://huggingface.co/blog/hf-bitsandbytes-integration https://huggingface.co/blog/hf-bitsandbytes-integration