5 ms·

>Through a series of system-wide optimizations, we’ve achieved 90% cost reduction for ChatGPT since December This is seriously impressive. A MILLION tokens for

by alexb_ 4y ago

>Through a series of system-wide optimizations, we’ve achieved 90% cost reduction for ChatGPT since December This is seriously impressive. A MILLION tokens for 2 dollars is absolutely fucking insane. I hope that the gains reached here can also be found by open source and non-controlled AI projects. If so, that could be huge for the advancement of AI.

machinekob 4y ago
If you are Microsoft as GigaScaler with almost unlimited cash and can ignore getting profit of your api/models its pretty easy to undercut all the other companies and offer it very cheap just to gain advantage in the future.
- alexb_ 4y ago
  What the cost cutting measures suggest is that AI like this could maybe soon be run on consumer hardware. That combined with actually open source language models could be huge. OpenAI won't allow for that for obvious reasons, but this confirms that the optimizations are there, and that's exciting enough news on its own.
  - alfalfasprout 4y ago
    I mean, meta's new LAMA model runs on a single A100 in the 13B parameter variant (which performs similar to GPT3 65B).
    - visarga 4y ago
      "Performs" on paper until they give a demo.
sva_ 4y ago
A lot of people assumed GPT4 would be an even bigger model, but I've been thinking it'll probably be more about more efficient compute. This is at least some evidence that they're working on that.
- m3kw9 4y ago
  GPT3s
ImprobableTruth 4y ago
It's tokens processed, not generated.
- visarga 4y ago
  If you have 10K tokens in your conversation, the next reply means 10K + len(reply) extra tokens. I estimate 125 rounds of conversation fit in 1M tokens, for $2.
  - IanCal 4y ago
    No it handles a max of 4k. That's context plus reply.
Tenoke 4y ago
To be fair, it also seems like it performs worse. question: "Who is Scott Alexander?" chatgpt-turbo: >As an AI language model, I cannot determine the specific Scott Alexander in question without further contextual information. There may be several people with this name, including writers, politicians, entrepreneurs, athletes, and more. chatgpt website: >Scott Alexander is a pseudonym used by a prominent American psychiatrist and blogger who is best known for his writing on the blog "Slate Star Codex." He has also written articles for other publications, including The New York Times, The Atlantic, and The Wall Street Journal. Scott Alexander's writing covers a wide range of topics, including medicine, psychiatry, economics, politics, and culture. He is known for his thoughtful, analytical approach to complex issues and for his ability to explain difficult concepts in a clear and accessible way. In 2020, Scott Alexander announced that he was deleting his blog after he was doxxed by a reporter for The New York Times who planned to write an article about him. However, he later started a new blog called "Astral Codex Ten" where he continues to write.
- nicky0 4y ago
  Perhaps a transitory issue. I just tried it with the API, `gpt-3.5-turbo`. I got: > Scott Alexander is the pen name of American psychiatrist and blogger, Scott Alexander Siskind. He is known for writing his blog, "Slate Star Codex", which covers a wide range of topics including science, medicine, politics, and culture. He has been praised for his clear and concise writing style and thoughtful analysis of various issues. In addition to his work as a blogger, Scott Alexander has also published a book titled "Unsong", which is a fantasy novel set in an alternate universe where the Bible is a magical text.
- CapsAdmin 4y ago
  Can we really draw any conclusions on LLMs based on 1 sample? Maybe you've tried multiple times and with different semi famous people, but in general I see people comparing ML models in this fashion.
  - Tenoke 4y ago
    Not really, I did try it with multiple attempts with multiple people and chatgpt had more issues. I just shared only one of them. If someone tests in a more systematic fashion that'd be great.
Havoc 4y ago
90% for a presumably already semi efficient setup is insane

by alexb_ 4y ago

machinekob 4y ago

If you are Microsoft as GigaScaler with almost unlimited cash and can ignore getting profit of your api/models its pretty easy to undercut all the other companies and offer it very cheap just to gain advantage in the future.

alexb_ 4y ago
What the cost cutting measures suggest is that AI like this could maybe soon be run on consumer hardware. That combined with actually open source language models could be huge. OpenAI won't allow for that for obvious reasons, but this confirms that the optimizations are there, and that's exciting enough news on its own.
- alfalfasprout 4y ago
  I mean, meta's new LAMA model runs on a single A100 in the 13B parameter variant (which performs similar to GPT3 65B).
  - visarga 4y ago
    "Performs" on paper until they give a demo.

sva_ 4y ago

A lot of people assumed GPT4 would be an even bigger model, but I've been thinking it'll probably be more about more efficient compute. This is at least some evidence that they're working on that.

m3kw9 4y ago
GPT3s

ImprobableTruth 4y ago

It's tokens processed, not generated.

visarga 4y ago
If you have 10K tokens in your conversation, the next reply means 10K + len(reply) extra tokens. I estimate 125 rounds of conversation fit in 1M tokens, for $2.
- IanCal 4y ago
  No it handles a max of 4k. That's context plus reply.

Tenoke 4y ago

To be fair, it also seems like it performs worse. question: "Who is Scott Alexander?" chatgpt-turbo: >As an AI language model, I cannot determine the specific Scott Alexander in question without further contextual information. There may be several people with this name, including writers, politicians, entrepreneurs, athletes, and more. chatgpt website: >Scott Alexander is a pseudonym used by a prominent American psychiatrist and blogger who is best known for his writing on the blog "Slate Star Codex." He has also written articles for other publications, including The New York Times, The Atlantic, and The Wall Street Journal. Scott Alexander's writing covers a wide range of topics, including medicine, psychiatry, economics, politics, and culture. He is known for his thoughtful, analytical approach to complex issues and for his ability to explain difficult concepts in a clear and accessible way. In 2020, Scott Alexander announced that he was deleting his blog after he was doxxed by a reporter for The New York Times who planned to write an article about him. However, he later started a new blog called "Astral Codex Ten" where he continues to write.

nicky0 4y ago
Perhaps a transitory issue. I just tried it with the API, `gpt-3.5-turbo`. I got: > Scott Alexander is the pen name of American psychiatrist and blogger, Scott Alexander Siskind. He is known for writing his blog, "Slate Star Codex", which covers a wide range of topics including science, medicine, politics, and culture. He has been praised for his clear and concise writing style and thoughtful analysis of various issues. In addition to his work as a blogger, Scott Alexander has also published a book titled "Unsong", which is a fantasy novel set in an alternate universe where the Bible is a magical text.
CapsAdmin 4y ago
Can we really draw any conclusions on LLMs based on 1 sample? Maybe you've tried multiple times and with different semi famous people, but in general I see people comparing ML models in this fashion.
- Tenoke 4y ago
  Not really, I did try it with multiple attempts with multiple people and chatgpt had more issues. I just shared only one of them. If someone tests in a more systematic fashion that'd be great.

Havoc 4y ago

90% for a presumably already semi efficient setup is insane