4 ms·

On the other hand gpt model are converging down. Gpt4 turbo degraded performance so much that now certain 13b produce more consistent results in reasoning. I've

On the other hand gpt model are converging down. Gpt4 turbo degraded performance so much that now certain 13b produce more consistent results in reasoning. I've a marathon test here for example https://chat.openai.com/share/dfd9b9ae-7214-4dd7-ad20-7ee07abed87c https://chat.openai.com/share/dfd9b9ae-7214-4dd7-ad20-7ee07a... with purposefully open ended and somewhat ambiguous request to see how models perform and gpt4 turbo chat is just not that good it confuses persons out, didn't pick the right one for abduction, didn't change topic when requested, when recalling persons picked the one from the wrong set, when asked to change language it didn't... It know a lot when asked zero shot questions, but when proving it's self consistency and attention it is nowhere near gpt4.

BoorishBears 3y ago
[flagged]
- avereveard 3y ago
  The point is exactly that the model people are experiencing is converging down with every subsequent update, and I even mentioned that it's nowhere near the orig gpt4, idk possibly read it again slower instead of jumping to credibility and whatnot.
  - BoorishBears 3y ago
    [flagged]
    - avereveard 3y ago
      > you can't compare it to the "original" via the web ui Good thing then that I was comparing the chat offering to 7b and 13b open models then, maybe you didn't catch that, try a third reading. Also, this kind of is a chatgpt thread. I know it's not a perfect comparison, but still. It's a discussion point I'm presenting, not a research paper.
      - BoorishBears 3y ago
        [flagged]
        kaycebasques 3y ago
        BoorishBear, please stop with the aggressive, condescending tone. You seem to have some productive counterarguments but they're getting lost in your disrespectful language.
        BoorishBears 3y ago
        [flagged]
        kaycebasques 3y ago
        I don't have a feature to flag your comments and your profile does not point to a way for me to message you privately so this was my only option to call you out What you call "tone policing" I call "violating numerous HN guidelines": > Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes. > When disagreeing, please reply to the argument instead of calling names. > Please don't sneer, including at the rest of the community. > Please don't post shallow dismissals, especially of other people's work.
        BoorishBears 3y ago
        It's not your job to call out anyone on HN, and it adds precious little to the conversation.
        kaycebasques 3y ago
        > It's not your job to call out anyone on HN You may not personally agree with it, but there's nothing in the HN guidelines or comments from mods saying that I can't do it > it adds precious little to the conversation The conversations would have ended a long time ago if you're vitriolic tone was considered acceptable and normal See dang's response to your comments: https://news.ycombinator.com/item?id=38493663 https://news.ycombinator.com/item?id=38493663
    - dang 3y ago
      Yikes, you broke the site guidelines really badly in this thread, not just in this comment but in several other places including these: https://news.ycombinator.com/item?id=38490123 https://news.ycombinator.com/item?id=38490123 https://news.ycombinator.com/item?id=38484632 https://news.ycombinator.com/item?id=38484632 We have to ban accounts that post this way. It's not what this site is for, and destroys what it is for. Moreover, we've had warn you about this more than once over the years. I don't want to ban you, so if you'd please review https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and stick to the rules from now on, we'd appreciate it. That means no swipes and no personal attacks, among other things. Edit: you've unfortunately been doing this in other recent threads too: https://news.ycombinator.com/item?id=38453843 https://news.ycombinator.com/item?id=38453843 https://news.ycombinator.com/item?id=38453381 https://news.ycombinator.com/item?id=38453381 https://news.ycombinator.com/item?id=38377068 https://news.ycombinator.com/item?id=38377068 We really are going to have to ban you if this keeps up, so if you'd please make whatever change is needed to not post like this again, that would be good.
- cooperaustinj 3y ago
  Your credibility is killed by thinking using an API can guarantee which model you're getting. It's entirely black box. If OpenAI wants to lie to you, they can.
  - BoorishBears 3y ago
    Your credibility is killed by thinking paying AWS for a c5.metal can guarantee which compute you're getting. It's entirely black box. If AWS wants to torpedo their business for marginal gain they can. And you won't be able to tell just because your workload falls on its face. Your comment is the textbook definition of FUD in a comical way.
infecto 3y ago
I don't think think using examples derived from ChatGPT are a fair comparison to the underlying models. OpenAI has many optimization tricks on the ChatGPT side that are unrelated to the underlying models being used. We do know of course that ChatGPT is most likely using 4-Turbo from the decrease in latency and increase in unhelpful answers. We cannot say that the models are "converging down" though. I don't remember the marketing materials but from the model side we all realize that the Turbo models have some type of quantization/optimization that makes them cheap and fast. 4-Turbo is 3x cheaper than 4, substantially quicker and provides better results than 3.5-Turbo. Amazing progress in my arena.
- Workaccount2 3y ago
  There were many rumors (and it probably was true) that OpenAI was hemorrhaging cash on GPT4 requests. So it makes tons of sense for them to sprint towards a turbo model at the expense of some ability. GPT4-turbo still is ridiculously powerful anyway.

BoorishBears 3y ago

[flagged]

avereveard 3y ago
The point is exactly that the model people are experiencing is converging down with every subsequent update, and I even mentioned that it's nowhere near the orig gpt4, idk possibly read it again slower instead of jumping to credibility and whatnot.
- BoorishBears 3y ago
  [flagged]
  - avereveard 3y ago
    > you can't compare it to the "original" via the web ui Good thing then that I was comparing the chat offering to 7b and 13b open models then, maybe you didn't catch that, try a third reading. Also, this kind of is a chatgpt thread. I know it's not a perfect comparison, but still. It's a discussion point I'm presenting, not a research paper.
    - BoorishBears 3y ago
      [flagged]
      - kaycebasques 3y ago
        BoorishBear, please stop with the aggressive, condescending tone. You seem to have some productive counterarguments but they're getting lost in your disrespectful language.
        BoorishBears 3y ago
        [flagged]
        kaycebasques 3y ago
        I don't have a feature to flag your comments and your profile does not point to a way for me to message you privately so this was my only option to call you out What you call "tone policing" I call "violating numerous HN guidelines": > Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes. > When disagreeing, please reply to the argument instead of calling names. > Please don't sneer, including at the rest of the community. > Please don't post shallow dismissals, especially of other people's work.
        BoorishBears 3y ago
        It's not your job to call out anyone on HN, and it adds precious little to the conversation.
        kaycebasques 3y ago
        > It's not your job to call out anyone on HN You may not personally agree with it, but there's nothing in the HN guidelines or comments from mods saying that I can't do it > it adds precious little to the conversation The conversations would have ended a long time ago if you're vitriolic tone was considered acceptable and normal See dang's response to your comments: https://news.ycombinator.com/item?id=38493663 https://news.ycombinator.com/item?id=38493663
  - dang 3y ago
    Yikes, you broke the site guidelines really badly in this thread, not just in this comment but in several other places including these: https://news.ycombinator.com/item?id=38490123 https://news.ycombinator.com/item?id=38490123 https://news.ycombinator.com/item?id=38484632 https://news.ycombinator.com/item?id=38484632 We have to ban accounts that post this way. It's not what this site is for, and destroys what it is for. Moreover, we've had warn you about this more than once over the years. I don't want to ban you, so if you'd please review https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and stick to the rules from now on, we'd appreciate it. That means no swipes and no personal attacks, among other things. Edit: you've unfortunately been doing this in other recent threads too: https://news.ycombinator.com/item?id=38453843 https://news.ycombinator.com/item?id=38453843 https://news.ycombinator.com/item?id=38453381 https://news.ycombinator.com/item?id=38453381 https://news.ycombinator.com/item?id=38377068 https://news.ycombinator.com/item?id=38377068 We really are going to have to ban you if this keeps up, so if you'd please make whatever change is needed to not post like this again, that would be good.
cooperaustinj 3y ago
Your credibility is killed by thinking using an API can guarantee which model you're getting. It's entirely black box. If OpenAI wants to lie to you, they can.
- BoorishBears 3y ago
  Your credibility is killed by thinking paying AWS for a c5.metal can guarantee which compute you're getting. It's entirely black box. If AWS wants to torpedo their business for marginal gain they can. And you won't be able to tell just because your workload falls on its face. Your comment is the textbook definition of FUD in a comical way.

infecto 3y ago

I don't think think using examples derived from ChatGPT are a fair comparison to the underlying models. OpenAI has many optimization tricks on the ChatGPT side that are unrelated to the underlying models being used. We do know of course that ChatGPT is most likely using 4-Turbo from the decrease in latency and increase in unhelpful answers. We cannot say that the models are "converging down" though. I don't remember the marketing materials but from the model side we all realize that the Turbo models have some type of quantization/optimization that makes them cheap and fast. 4-Turbo is 3x cheaper than 4, substantially quicker and provides better results than 3.5-Turbo. Amazing progress in my arena.

Workaccount2 3y ago
There were many rumors (and it probably was true) that OpenAI was hemorrhaging cash on GPT4 requests. So it makes tons of sense for them to sprint towards a turbo model at the expense of some ability. GPT4-turbo still is ridiculously powerful anyway.