3 ms·

I can't find the citation, but someone related to OpenAI said that ChatGpt cost a few cents per query. Anecdotally, I looked at running an open-source GPT comp

I can't find the citation, but someone related to OpenAI said that ChatGpt cost a few cents per query. Anecdotally, I looked at running an open-source GPT competitor from Meta (OPT), and the requirements for running that are vaguely on the scope of "8 GPUs with at least 400gb of GPU memory" which would be VERY expensive, even in the cloud. An 8gpu AWS EC2 instance costs $3.40/hr (p2.8xlarge), and if the model runs for 20s per request (roughly how long chatGPT takes me to get a response), that gives you $0.02 a request, not including things like storage or idle capacity. So a request could likely be $0.05-0.1, not including overhead like dev costs, training, etc.

chrisbaker98 4y ago
> I can't find the citation, but someone related to OpenAI said that ChatGpt cost a few cents per query. Here's the citation, from Sam Altman of OpenAI: https://twitter.com/sama/status/1599671496636780546 https://twitter.com/sama/status/1599671496636780546 Also, this person who doesn't appear to actually be affiliated with OpenAI estimates the running costs at $100k per day: https://twitter.com/tomgoldsteincs/status/1600196995389366274 https://twitter.com/tomgoldsteincs/status/160019699538936627...