3 ms·
5KW Peak Power per system pictured. I wonder, how many watt hours per query are used on average? Is this 100k investment enough to roll your own ChatGPT level b
by pcdoodle 4y ago
5KW Peak Power per system pictured. I wonder, how many watt hours per query are used on average? Is this 100k investment enough to roll your own ChatGPT level bot using open source software?
- HPsquared 4y agoNot exactly... OpenAI apparently have the 5th most powerful supercomputer in the world. This is probably what they used for training: https://news.microsoft.com/source/features/ai/openai-azure-supercomputer/ https://news.microsoft.com/source/features/ai/openai-azure-s... https://www.zdnet.com/article/microsoft-builds-a-supercomputer-for-openai-for-training-massive-ai-models/ https://www.zdnet.com/article/microsoft-builds-a-supercomput...
- pcdoodle 4y agoThanks for the links. I wonder what percentage of the energy expenditure is the initial training vs. operating the service?
- HPsquared 4y agoThat depends how much it's used, I suppose. Ever-increasing, it looks like. At some point it might be worth investing in more training to reduce operating expense. Pretty standard CAPEX vs OPEX calculation. On the other hand, we're not counting energy saved by people using it.. What if some scientist uses it and ends up solving fusion power? Or, say, making copywriters and programmers redundant (who I assume would go on to become subsistence farmers) - many energy savings to be had!