4 ms·

Not really. They have a way of squaring this circle, by changing their inference code. Speculative sampling [1] would still make their first claim a lie – sure,

Not really. They have a way of squaring this circle, by changing their inference code. Speculative sampling [1] would still make their first claim a lie – sure, there'd still be the original GPT-4 model, plus a smaller draft worker. But early exit decoding [2] allows you to get almost as good results for much cheaper from exactly the same checkpoint. We know that this line of research for large-scale inference is going strong [3] so it stands to reason that OpenAI, with their wealth of talent focused on GPT-4 throughput&inference [4], large contexts and aggressive pricing policy, would also develop something like that. And of course it's "smarter" that way – in a very deceptive sense of the word. 1. https://arxiv.org/abs/2302.01318 https://arxiv.org/abs/2302.01318 2. https://arxiv.org/abs/2207.07061 https://arxiv.org/abs/2207.07061 3. https://arxiv.org/abs/2307.02628 https://arxiv.org/abs/2307.02628 4. https://openai.com/contributions/gpt-4 https://openai.com/contributions/gpt-4

BoorishBears 3y ago
I don't get why you're jumping to cloak and daggers style operations: OpenAI would not kneecap their commercial offering by randomly changing how it works. At the end of the day 99% of the confusion comes from people using the web interface, which undoubtedly does change much more often than the API versions they share. The web app they host isn't a simple API wrapper, it does summarization, has some sort of system prompt, and calls the moderation API. That's undoubtedly being updated all the time.
- Shank 3y ago
  > OpenAI would not kneecap their commercial offering by randomly changing how it works. > As of July 3, 2023, we’ve disabled the Browse with Bing beta feature out of an abundance of caution while we fix this in order to do right by content owners. We are working to bring the beta back as quickly as possible, and appreciate your understanding! https://help.openai.com/en/articles/8077698-how-do-i-use-chatgpt-browse-with-bing-to-search-the-web https://help.openai.com/en/articles/8077698-how-do-i-use-cha...
  - BoorishBears 3y ago
    Thank you for confirming my point? > At the end of the day 99% of the confusion comes from people using the web interface, which undoubtedly does change much more often than the API versions they share. The API does not offer any browsing features, that's the web app.
    - deleted 3y ago
      [deleted]
- airgapstopgap 3y ago
  No, it makes sense to secure engagement with the most expensive implementation and then cut costs, this kind of stuff is pervasive in the industry. Besides, we have Brockman on record saying that they do "a lot of quantization"[1][2] so it's not paranoia to suspect other optimization schemes when there's a clear performance drop, which they have also denied a few times. 1. https://chat.openai.com/share/44a0c5b6-c629-470a-992f-8cdbbecd64a2 https://chat.openai.com/share/44a0c5b6-c629-470a-992f-8cdbbe... 2. https://www.youtube.com/watch?v=_hpuPi7YZX8 https://www.youtube.com/watch?v=_hpuPi7YZX8
  - BoorishBears 3y ago
    Paranoia would be charitable: it's FUD. If you intentionally smear the line between their web app which is chock full of optimizations to even let it function as it does (the web app's max conversation length exceeds the context window) and the API which is versioned and iterated on in the open... it's either a lack of understanding or FUD.
- visarga 3y ago
  > OpenAI would not kneecap their commercial offering by randomly changing how it works. Have you seen the 25 messages/3 hours limitation for GPT-4? Why do you think they did that? Of course they would make more money scaling up the volume, but how to do that when compute is so limited? Of course, by using some kind of approximation - quantised model or speculative sampling come to mind. It's hard to pinpoint model regressions, but scaling up volume is great, one more incentive to do it.
  - BoorishBears 3y ago
    You realize that's a limitation in the web application right? The web app is a consumer app (B2C) the api is commercial (B2B). They tinker with the B2C app because it's already a lossy approximation of using the model between the summarization and system prompt. They cannot mess with the commercial offering willy-nilly: People are building businesses predicated on it behaving a certain way. That's why there are dated version that you can pin to with the API. The web app changes whenever they feel like it.
    - zo1 3y ago
      You keep repeating that. You don't even know if the people commenting to you use the API or the "web app". I use the API and I noticed the same stuff others have.
      - BoorishBears 3y ago
        > Have you seen the 25 messages/3 hours limitation for GPT-4? If can't tell if that's about the API or the web app, I don't think you're familiar enough with the subject to speak on it.
        zo1 3y ago
        I don't need to be familiar with it to see that you're not being genuine in your interpretation of peoples' comments and in the way you're responding to people in this thread. Case in point: your reply to me.

BoorishBears 3y ago

I don't get why you're jumping to cloak and daggers style operations: OpenAI would not kneecap their commercial offering by randomly changing how it works. At the end of the day 99% of the confusion comes from people using the web interface, which undoubtedly does change much more often than the API versions they share. The web app they host isn't a simple API wrapper, it does summarization, has some sort of system prompt, and calls the moderation API. That's undoubtedly being updated all the time.

Shank 3y ago
> OpenAI would not kneecap their commercial offering by randomly changing how it works. > As of July 3, 2023, we’ve disabled the Browse with Bing beta feature out of an abundance of caution while we fix this in order to do right by content owners. We are working to bring the beta back as quickly as possible, and appreciate your understanding! https://help.openai.com/en/articles/8077698-how-do-i-use-chatgpt-browse-with-bing-to-search-the-web https://help.openai.com/en/articles/8077698-how-do-i-use-cha...
- BoorishBears 3y ago
  Thank you for confirming my point? > At the end of the day 99% of the confusion comes from people using the web interface, which undoubtedly does change much more often than the API versions they share. The API does not offer any browsing features, that's the web app.
  - deleted 3y ago
    [deleted]
airgapstopgap 3y ago
No, it makes sense to secure engagement with the most expensive implementation and then cut costs, this kind of stuff is pervasive in the industry. Besides, we have Brockman on record saying that they do "a lot of quantization"[1][2] so it's not paranoia to suspect other optimization schemes when there's a clear performance drop, which they have also denied a few times. 1. https://chat.openai.com/share/44a0c5b6-c629-470a-992f-8cdbbecd64a2 https://chat.openai.com/share/44a0c5b6-c629-470a-992f-8cdbbe... 2. https://www.youtube.com/watch?v=_hpuPi7YZX8 https://www.youtube.com/watch?v=_hpuPi7YZX8
- BoorishBears 3y ago
  Paranoia would be charitable: it's FUD. If you intentionally smear the line between their web app which is chock full of optimizations to even let it function as it does (the web app's max conversation length exceeds the context window) and the API which is versioned and iterated on in the open... it's either a lack of understanding or FUD.
visarga 3y ago
> OpenAI would not kneecap their commercial offering by randomly changing how it works. Have you seen the 25 messages/3 hours limitation for GPT-4? Why do you think they did that? Of course they would make more money scaling up the volume, but how to do that when compute is so limited? Of course, by using some kind of approximation - quantised model or speculative sampling come to mind. It's hard to pinpoint model regressions, but scaling up volume is great, one more incentive to do it.
- BoorishBears 3y ago
  You realize that's a limitation in the web application right? The web app is a consumer app (B2C) the api is commercial (B2B). They tinker with the B2C app because it's already a lossy approximation of using the model between the summarization and system prompt. They cannot mess with the commercial offering willy-nilly: People are building businesses predicated on it behaving a certain way. That's why there are dated version that you can pin to with the API. The web app changes whenever they feel like it.
  - zo1 3y ago
    You keep repeating that. You don't even know if the people commenting to you use the API or the "web app". I use the API and I noticed the same stuff others have.
    - BoorishBears 3y ago
      > Have you seen the 25 messages/3 hours limitation for GPT-4? If can't tell if that's about the API or the web app, I don't think you're familiar enough with the subject to speak on it.
      - zo1 3y ago
        I don't need to be familiar with it to see that you're not being genuine in your interpretation of peoples' comments and in the way you're responding to people in this thread. Case in point: your reply to me.