4 ms·
Insanely misleading title. The author is basically showing off their A100 GPUs. The ChatGPT part is just clickbait, the only relevance ia that ChatGPT supposedl
by hexomancer 4y ago
Insanely misleading title. The author is basically showing off their A100 GPUs. The ChatGPT part is just clickbait, the only relevance ia that ChatGPT supposedly is also using A100s (definitely not only eight of them though).
- yieldcrv 4y agothe real question is whether ChatGPT needs that kind of hardware for a single user. a client side version.
- speedgoose 4y agoYes it does.
- jpeter 4y agoWhen stable diffusion came out, my 8gb gpu could handle 1 image at 512x512. Now i can generate 1000x800 with batch size 4. I wonder if Chatgpt has the same unused optimisation potential
- yieldcrv 4y agowhat packages are you using? I think I have DiffusionBee installed but I haven't updated it for months
- fpgaminer 4y ago> I wonder if Chatgpt has the same unused optimisation potential Doubtful. As an example, OpenAI released Triton, a programming language explicitly for optimizing models. For the scale they're at they'd be crazy not to throw an engineer or two at optimizing the models they're going to send to production.
- speedgoose 4y agoStable Diffusion is a lot smaller than GPT3. It has a bit less than a billion parameters while GPT3 has 175 billion parameters. I would bet that we can do better than GPT3 with less computing resources but that would need a different model architecture.
- flangola7 4y agoMake a Google of "hardware overhang" Our brains outperform GPT-3 with only 30w of power. The potential for software optimization may be many orders of magnitude.