3 ms·

OpenAI charges per-word. You get an initial credit, but the "setup" prompt is so long that each request will be far more expensive than "normal" requests, since

OpenAI charges per-word. You get an initial credit, but the "setup" prompt is so long that each request will be far more expensive than "normal" requests, since you'll be charged for the entire setup prompt with every single request.

mrtksn 4y ago
Yep, have been playing with the 18USD API credit and indeed the cost is not trivial enough for casual use cases. That said, these new developments give us a new reason to have powerful computers. The last 10 years computers were way more powerful for most of the task for their conventional use cases but now computers are yet again way less powerful for the new frontier use cases. Apparently GPT-3 runs on 350GB VRAM, so it requires specialised hardware, so the best we can do today is to sneak peak into the future of on-device GPT-3 like LLMs.
eknkc 4y ago
That input is 610 tokens. With the actual prompt and the response let's say you hit 1000 tokens. That would be 2 cents per question then. Not sure how often do people ask Siri questions but that seems like it would not be a significant cost for one user. If you were to build a service on top of it, that might be expensive though.
- PebblesRox 4y ago
  In a home with only responsible adults, it would be fine. With kids it would be a repeat of the olden days before unlimited texting when the parents would get the cellphone bill at the end of the month. But I wonder how soon we’ll have fine-tuned or otherwise customized specialized LLMs that are cheaper to run? In the mean time this kind of thing is a really neat proof-of-concept of how ChatGPT’s abilities can be harnessed.
KMnO4 4y ago
GPT-3 is really impressive with the Davinci model, but this seems like overkill for the tasks here. You can save 10x [0] by just switching to the Curie model. More information on the models: https://subscription.packtpub.com/book/data/9781800563193/2/ch02lvl1sec07/introducing-davinci-babbage-curie-and-ada https://subscription.packtpub.com/book/data/9781800563193/2/... [0]: https://openai.com/api/pricing/ https://openai.com/api/pricing/

mrtksn 4y ago

Yep, have been playing with the 18USD API credit and indeed the cost is not trivial enough for casual use cases. That said, these new developments give us a new reason to have powerful computers. The last 10 years computers were way more powerful for most of the task for their conventional use cases but now computers are yet again way less powerful for the new frontier use cases. Apparently GPT-3 runs on 350GB VRAM, so it requires specialised hardware, so the best we can do today is to sneak peak into the future of on-device GPT-3 like LLMs.

eknkc 4y ago

That input is 610 tokens. With the actual prompt and the response let's say you hit 1000 tokens. That would be 2 cents per question then. Not sure how often do people ask Siri questions but that seems like it would not be a significant cost for one user. If you were to build a service on top of it, that might be expensive though.

PebblesRox 4y ago
In a home with only responsible adults, it would be fine. With kids it would be a repeat of the olden days before unlimited texting when the parents would get the cellphone bill at the end of the month. But I wonder how soon we’ll have fine-tuned or otherwise customized specialized LLMs that are cheaper to run? In the mean time this kind of thing is a really neat proof-of-concept of how ChatGPT’s abilities can be harnessed.

KMnO4 4y ago

GPT-3 is really impressive with the Davinci model, but this seems like overkill for the tasks here. You can save 10x [0] by just switching to the Curie model. More information on the models: https://subscription.packtpub.com/book/data/9781800563193/2/ch02lvl1sec07/introducing-davinci-babbage-curie-and-ada https://subscription.packtpub.com/book/data/9781800563193/2/... [0]: https://openai.com/api/pricing/ https://openai.com/api/pricing/