3 ms·

So how 'big' is GPT-3? Is it anywhere near being able to be run on local consumer hardware? How long until we can have the GPT3 or 3.5 chatbot locally like we

So how 'big' is GPT-3? Is it anywhere near being able to be run on local consumer hardware? How long until we can have the GPT3 or 3.5 chatbot locally like we have StableDiffusion locally for image generation? I've been spoiled by having it accessible offline and with community built support/modifications to it. GPT-3 is super neat but feels like too many guard rails or the custom playground is too pricey.

lossolo 4y ago
> So how 'big' is GPT-3? For inference so basically running it you need multiple GPUs and hundreds of GBs of GPU memory. As to model size it's around 100x bigger than SD. You can forget about running it locally unless you have dozens of high-end GPUs or you want to wait hours/days/weeks (depending on your hardware) for a single response.