10 ms·

Is OpenAI still making any changes to make ChatGPT better and more accurate and correct, or are they only focusing on making it cheaper by giving us weaker/dumb

Is OpenAI still making any changes to make ChatGPT better and more accurate and correct, or are they only focusing on making it cheaper by giving us weaker/dumber responses in a faster way nowadays? I have cancelled my subscription recently because didn't think that the GPT-4 was much better than what I get for free at Claude or Gemini.

mritchie712 2y ago
as someone building a product with their API, GPT-4 is already smarter than we need for our use case. Request latency is my #1 concern right now.
- JimDabell 2y ago
  If request latency is your top concern and you don’t need it to be as smart as GPT-4, why are you using OpenAI and not a model hosted by Groq or similar?
  - mritchie712 2y ago
    We need function calling, which wasn't great in Groq when we last tried it, but they seem to have made progress on it. We'll be testing it again soon.
    - voiper1 2y ago
      groq just released: llama3-groq-70b-8192-tool-use-preview llama3-groq-8b-8192-tool-use-preview Sonnet3.5 also has tool usage, and is slightly cheaper than 4o ($3 input vs $5). If you _only_ need tool usage, then 4o-mini or others will be cheaper... mistal's Mixtral 8x22B has function calling for $2/$6, which is a bit cheaper than 4o.
  - d13 2y ago
    Groq’s rate limits are so bad they’re only good enough for one person using it very rarely with an under 4k context size. It’s just a marketing stunt at the moment.
    - fragmede 2y ago
      how much are you paying them?
j45 2y ago
There might be some changes, but OpenAI has consistently shown they have way more polished stuff sitting and getting ready to go out, based on the quality of the product or update on the day of the launch.
OsrsNeedsf2P 2y ago
I suspect model improvement costs exponentially more. Between the fact it's been over a year since 4's release and 4o is only marginally better, and that OpenAI is focusing on other things (Sora, mini models, voice, etc) suggests a "better and more accurate and correct" model is actually really hard.
fragmede 2y ago
> Is OpenAI still making any changes to make ChatGPT better and more accurate and correct a few prompts recently, it just answered for me instead of refusing, to my surprise, so if feels like it's been getting better, but unfortunately there's no hard data I have on that, it's more of a feeling, so it's hard to prove that's actually true to someone else. then again, ChatGPT-4o was recently found to be unable to do 9.11 - 9.9 correctly unless pressed, so there's still a long ways to go. https://chatgpt.com/share/47d260ae-0c62-48ab-8d3b-84767acd7fb5 https://chatgpt.com/share/47d260ae-0c62-48ab-8d3b-84767acd7f...
- JohnMakin 2y ago
  That is interesting for sure, but the subsequent questioning "Why did you get it wrong the first time?" and asking it to explain itself seems like it shows a misunderstanding about how these technologies actually work. Please stop treating these tools like they are conscious and actually understand what they are spitting back to you! /rant
  - fragmede 2y ago
    I've done spreadsheets are all you need. The questioning isn't based on a misunderstanding that this is autocomplete with an insane dataset, but given a black box, how else do you want it to explain how it works?
    - JohnMakin 2y ago
      > how else do you want it to explain how it works? Why do you think these tools understand how they work enough to explain it? If this were the case they wouldn't be a blackbox to begin with.
    - Atotalnoob 2y ago
      Because it’s making up how it works.
  - skeledrew 2y ago
    Actually, given that models are trained on data generated by humans, the ideal way to get decent responses are by treating them as such. Many times when I'm aware that a model gave an incorrect response, or I'm not sure and skeptical, a simple "Are you sure" or "Explain your answer" does wonders.
hidelooktropic 2y ago
Going out on a limb here that they're not focusing the entire company on one objective. Adding to this, reducing cost means they've reduced compute and improved quality per unit of computation. That matters! It matters that these systems currently require an embarassing amount of energy to run otherwise. It matters that these models could be portable to the point where they can be run locally on portable consumer electronics rather than being locked into remote compute and HTTP calls.
doctorpangloss 2y ago
They didn't strategize for the possibility that Anthropic would produce a better model than they do.
swyx 2y ago
anecdotally hearing they're pretty close to releasing the "AGI level 2" model. not sure if it'll be classed 4.5 or 5

mritchie712 2y ago

as someone building a product with their API, GPT-4 is already smarter than we need for our use case. Request latency is my #1 concern right now.

JimDabell 2y ago
If request latency is your top concern and you don’t need it to be as smart as GPT-4, why are you using OpenAI and not a model hosted by Groq or similar?
- mritchie712 2y ago
  We need function calling, which wasn't great in Groq when we last tried it, but they seem to have made progress on it. We'll be testing it again soon.
  - voiper1 2y ago
    groq just released: llama3-groq-70b-8192-tool-use-preview llama3-groq-8b-8192-tool-use-preview Sonnet3.5 also has tool usage, and is slightly cheaper than 4o ($3 input vs $5). If you _only_ need tool usage, then 4o-mini or others will be cheaper... mistal's Mixtral 8x22B has function calling for $2/$6, which is a bit cheaper than 4o.
- d13 2y ago
  Groq’s rate limits are so bad they’re only good enough for one person using it very rarely with an under 4k context size. It’s just a marketing stunt at the moment.
  - fragmede 2y ago
    how much are you paying them?

j45 2y ago

There might be some changes, but OpenAI has consistently shown they have way more polished stuff sitting and getting ready to go out, based on the quality of the product or update on the day of the launch.

OsrsNeedsf2P 2y ago

I suspect model improvement costs exponentially more. Between the fact it's been over a year since 4's release and 4o is only marginally better, and that OpenAI is focusing on other things (Sora, mini models, voice, etc) suggests a "better and more accurate and correct" model is actually really hard.

fragmede 2y ago

> Is OpenAI still making any changes to make ChatGPT better and more accurate and correct a few prompts recently, it just answered for me instead of refusing, to my surprise, so if feels like it's been getting better, but unfortunately there's no hard data I have on that, it's more of a feeling, so it's hard to prove that's actually true to someone else. then again, ChatGPT-4o was recently found to be unable to do 9.11 - 9.9 correctly unless pressed, so there's still a long ways to go. https://chatgpt.com/share/47d260ae-0c62-48ab-8d3b-84767acd7fb5 https://chatgpt.com/share/47d260ae-0c62-48ab-8d3b-84767acd7f...

JohnMakin 2y ago
That is interesting for sure, but the subsequent questioning "Why did you get it wrong the first time?" and asking it to explain itself seems like it shows a misunderstanding about how these technologies actually work. Please stop treating these tools like they are conscious and actually understand what they are spitting back to you! /rant
- fragmede 2y ago
  I've done spreadsheets are all you need. The questioning isn't based on a misunderstanding that this is autocomplete with an insane dataset, but given a black box, how else do you want it to explain how it works?
  - JohnMakin 2y ago
    > how else do you want it to explain how it works? Why do you think these tools understand how they work enough to explain it? If this were the case they wouldn't be a blackbox to begin with.
  - Atotalnoob 2y ago
    Because it’s making up how it works.
- skeledrew 2y ago
  Actually, given that models are trained on data generated by humans, the ideal way to get decent responses are by treating them as such. Many times when I'm aware that a model gave an incorrect response, or I'm not sure and skeptical, a simple "Are you sure" or "Explain your answer" does wonders.

hidelooktropic 2y ago

Going out on a limb here that they're not focusing the entire company on one objective. Adding to this, reducing cost means they've reduced compute and improved quality per unit of computation. That matters! It matters that these systems currently require an embarassing amount of energy to run otherwise. It matters that these models could be portable to the point where they can be run locally on portable consumer electronics rather than being locked into remote compute and HTTP calls.

doctorpangloss 2y ago

They didn't strategize for the possibility that Anthropic would produce a better model than they do.

swyx 2y ago

anecdotally hearing they're pretty close to releasing the "AGI level 2" model. not sure if it'll be classed 4.5 or 5