3 ms·

The Implications of ChatGPT’s API Cost

35 points by breck 4y ago

demetrius 4y ago
> However, it does speak sections of thousands of languages (try Egyptian Hieroglyphics That was actually a test case I've tried. Asked it to write a text in Middle Egyptian. It wrote gibberish. Asked it to translate it. It admitted that it was just a random hieroglyph salad with no meaning at all. It's quite bad even at Belarusian, regularly producing impossible word forms, incorrectly using words and so on, mis-recognizing the language as Ukrainian, etc.
- quesomaster9000 4y ago
  While you may get braindead salad responses from GPT I've experienced much more frustration with non-technical support people politely avoiding the question and sticking to the script rather than saying they don't know what I'm even asking and shutting down every line of inquiry. If I could pay a dollar to be passed over to an AI Level-2 support that'd been trained on their documentation and supplier reference manuals (who knows what happens if that information gets released without signing 10 levels of NDAs, all hell would unleash) - I'd treat it no different to any other Level-2 (as in, with a big pinch of salt) and it'd be cheaper for all involved.
  - taskforcegemini 4y ago
    I can't help seeing this as anything but a race to the bottom. In call centers, those sticking to the script get promoted for doing what they are told, and because usually handle a case more "efficiently" than someone who tries to actually solve the problem. and chatgpt will just be the even cheaper version
sourcecodeplz 4y ago
I've read the whole reddit thread. One reply caught my attention. It said: openai will "bait and switch" by increasing the price once the new api becomes too popular. I believe not, before they have a chance to do this, we will be able to run something similar on home PCs.
- SturgeonsLaw 4y ago
  Yes, just like Stable Diffusion rugpulled DALL-E 2, I would expect (hope!) to see an LLM about as capable as GPT-3 that can run on a decent gaming box. ChatGPT supposedly uses 8x A100s to process a single request, so some advances in efficiency and GPU power might be needed first, but it's just a matter of time really.
credit_guy 4y ago
We are entering a period of Moore's law for GPT bots. The claim by NVIDIA that in 10 years a GPT bot will be one million times more powerful than ChatGPT may or may not pan out. But I can easily see a factor of one thousand, which means a doubling of capabilities each year. Five years from now the world will be unrecognizable.
- icapybara 4y ago
  I think one big difference might be that we are limited by hardware. For this to be true, the models and algorithms will need to become exponentially more efficient over time, because the hardware won't.
  - credit_guy 4y ago
    Well, right now we think performance comes from software plus hardware. I think going forward, performance will come from software, hardware and training. Chances are that ChatGPT in the current configuration (i.e. same NN architecture and number of parameters) run on the current hardware would be much more powerful if you could continue training it with 10 times more data than you used so far. Separate from that, I think you could also increase tremendously ChatGPT's power if you could pair it with some regular old-fashioned computer. For example, if you ask ChatGPT to write some python code, it produces code that is surprisingly close to correct. But why not give ChatGPT a separate module or sandbox where it can actually run and test that code?

35 points by breck 4y ago

> However, it does speak sections of thousands of languages (try Egyptian Hieroglyphics That was actually a test case I've tried. Asked it to write a text in Middle Egyptian. It wrote gibberish. Asked it to translate it. It admitted that it was just a random hieroglyph salad with no meaning at all. It's quite bad even at Belarusian, regularly producing impossible word forms, incorrectly using words and so on, mis-recognizing the language as Ukrainian, etc.

quesomaster9000 4y ago
While you may get braindead salad responses from GPT I've experienced much more frustration with non-technical support people politely avoiding the question and sticking to the script rather than saying they don't know what I'm even asking and shutting down every line of inquiry. If I could pay a dollar to be passed over to an AI Level-2 support that'd been trained on their documentation and supplier reference manuals (who knows what happens if that information gets released without signing 10 levels of NDAs, all hell would unleash) - I'd treat it no different to any other Level-2 (as in, with a big pinch of salt) and it'd be cheaper for all involved.
- taskforcegemini 4y ago
  I can't help seeing this as anything but a race to the bottom. In call centers, those sticking to the script get promoted for doing what they are told, and because usually handle a case more "efficiently" than someone who tries to actually solve the problem. and chatgpt will just be the even cheaper version

sourcecodeplz 4y ago

I've read the whole reddit thread. One reply caught my attention. It said: openai will "bait and switch" by increasing the price once the new api becomes too popular. I believe not, before they have a chance to do this, we will be able to run something similar on home PCs.

SturgeonsLaw 4y ago
Yes, just like Stable Diffusion rugpulled DALL-E 2, I would expect (hope!) to see an LLM about as capable as GPT-3 that can run on a decent gaming box. ChatGPT supposedly uses 8x A100s to process a single request, so some advances in efficiency and GPU power might be needed first, but it's just a matter of time really.

credit_guy 4y ago

We are entering a period of Moore's law for GPT bots. The claim by NVIDIA that in 10 years a GPT bot will be one million times more powerful than ChatGPT may or may not pan out. But I can easily see a factor of one thousand, which means a doubling of capabilities each year. Five years from now the world will be unrecognizable.

icapybara 4y ago
I think one big difference might be that we are limited by hardware. For this to be true, the models and algorithms will need to become exponentially more efficient over time, because the hardware won't.
- credit_guy 4y ago
  Well, right now we think performance comes from software plus hardware. I think going forward, performance will come from software, hardware and training. Chances are that ChatGPT in the current configuration (i.e. same NN architecture and number of parameters) run on the current hardware would be much more powerful if you could continue training it with 10 times more data than you used so far. Separate from that, I think you could also increase tremendously ChatGPT's power if you could pair it with some regular old-fashioned computer. For example, if you ask ChatGPT to write some python code, it produces code that is surprisingly close to correct. But why not give ChatGPT a separate module or sandbox where it can actually run and test that code?