3 ms·

This isn't totally out of the blue - they have previously e-mailed API users claiming that the GPT-3.5-Turbo model is just as good/better as well as being faste

This isn't totally out of the blue - they have previously e-mailed API users claiming that the GPT-3.5-Turbo model is just as good/better as well as being faster & cheaper per token than their other models, and would only require a small change to prompts (claims of being better, of course, isn't true for all cases). I suspect the reason for this (especially given the extremely short timeline) is they're desperately trying to free up space in their clusters for GPT-4 nodes - they've clearly been struggling under the demand for GPT-4 (with ChatGPT Plus subscribers initially getting 100 GPT-4 messages every few hours, then 50, then 25 with a warning that the limit will be reduced further this week). I assume the limiting factor here is how many A100 80GB instances Microsoft is able to get and rack from nVidia.

karmasimida 4y ago
If it is for capacity issue, they can just say it. Also cushman is, allegedly, a 12B model. It can be served using much lesser demanding hardware. Honestly, they can just severely reduce the nodes allocate to those models, employing punishing rate limits, instead of announcing such a short noticed retirement, if resources are REALLY the issue.
- brookst 4y ago
  I don’t think they owe you an explanation. And resources aren’t just A100’s. They have engineers maintaining those older models, and support staff answering questions, and so on. OpenAI has found themselves at the center of a gold rush. I don’t blame them for not wanting to operate silver mines, and I don’t expect them to ask my permission or justify themselves to me. (I will be impacted by the codex shutdown. It’s fine. If I wanted to be in a perfectly stable space I’d go support cobol banking applications)
- garblegarble 4y ago
  If you're trying to convince people to replace humans with your tech then I could see a business not wanting to appear like they're struggling with demand (although, like you I don't think that's the right choice). Like the sibling comment said, it could also be limited by developer capacity - they reportedly have 350 employees (excluding contractors helping with training), with that elite size you wouldn't want to be supporting a large set of different clusters.
  - varenc 4y ago
    Riffing on that a bit, I just don't get the sense OpenAI wants to be in the business of hosting mission critical APIs for Fortune 500 companies. To your point, running AWS level services is an entirely different business that doesn't make sense for OpenAI's relatively small team. If you need rock solid stability and scale, OpenAI and Microsoft would probably rather have you just use "Azure OpenAI" services, which is still hosting Codex: https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/work-with-code https://learn.microsoft.com/en-us/azure/cognitive-services/o... Reading between the lines, I think OpenAI is signaling that if you want a more mature and stable product, go with Azure. If you want the latest cutting edge, and don't mind a little instability, keep using OpenAI API's.
    - garblegarble 4y ago
      That's a good take, I hadn't thought about it like that
chillfox 4y ago
Interesting… so finally a tech advantage to being in Australia. I still get the 100 messages/4h during the day here. Late at night (it’s about 1am now) it’s hard to get logged in due to congestion.

karmasimida 4y ago

If it is for capacity issue, they can just say it. Also cushman is, allegedly, a 12B model. It can be served using much lesser demanding hardware. Honestly, they can just severely reduce the nodes allocate to those models, employing punishing rate limits, instead of announcing such a short noticed retirement, if resources are REALLY the issue.

brookst 4y ago
I don’t think they owe you an explanation. And resources aren’t just A100’s. They have engineers maintaining those older models, and support staff answering questions, and so on. OpenAI has found themselves at the center of a gold rush. I don’t blame them for not wanting to operate silver mines, and I don’t expect them to ask my permission or justify themselves to me. (I will be impacted by the codex shutdown. It’s fine. If I wanted to be in a perfectly stable space I’d go support cobol banking applications)
garblegarble 4y ago
If you're trying to convince people to replace humans with your tech then I could see a business not wanting to appear like they're struggling with demand (although, like you I don't think that's the right choice). Like the sibling comment said, it could also be limited by developer capacity - they reportedly have 350 employees (excluding contractors helping with training), with that elite size you wouldn't want to be supporting a large set of different clusters.
- varenc 4y ago
  Riffing on that a bit, I just don't get the sense OpenAI wants to be in the business of hosting mission critical APIs for Fortune 500 companies. To your point, running AWS level services is an entirely different business that doesn't make sense for OpenAI's relatively small team. If you need rock solid stability and scale, OpenAI and Microsoft would probably rather have you just use "Azure OpenAI" services, which is still hosting Codex: https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/work-with-code https://learn.microsoft.com/en-us/azure/cognitive-services/o... Reading between the lines, I think OpenAI is signaling that if you want a more mature and stable product, go with Azure. If you want the latest cutting edge, and don't mind a little instability, keep using OpenAI API's.
  - garblegarble 4y ago
    That's a good take, I hadn't thought about it like that

chillfox 4y ago

Interesting… so finally a tech advantage to being in Australia. I still get the 100 messages/4h during the day here. Late at night (it’s about 1am now) it’s hard to get logged in due to congestion.