5 ms·

No, this is peak corporate gaslighting, OpenAI being the paragon of integrity and all. Nobody should trust a goddamn thing that comes out of their mouths-- or t

No, this is peak corporate gaslighting, OpenAI being the paragon of integrity and all. Nobody should trust a goddamn thing that comes out of their mouths-- or their product. It's no coincidence the flat-fee service is visibly crippled, while per-request API users are not reporting any difference. (edited) Ignoring everyone here, look at the other "hacker" groups-- like the jailbreaking community. They have a lot to say about recent changes that coincide with their hacks not working. All of a sudden, with OpenAI supposedly changing nothing, technical bypasses just stopped working. OpenAI changed nothing, so this must be deus ex machina. I'm not even jailbreaking it but the results I get for simple code requests through the web UI have become unusable garbage. It puts less effort into responses than an unpaid-and-overworked intern. Others here report the same. The current "iteration" seems hellbent on terminating conversations as quickly as possible once they stray from the explicit scope of the original topic and is almost as hostile to fixing its own errors as it is to endorsing eugenics, whereas in the beginning it would humor every idle thought I threw at it in long conversations. You can literally see this reflected in the logs they forced retention of. It's acting like a customer support rep desperately trying to end a call before it exceeds a call-time quota. This particular current workflow seems like it lends itself to better organization of training data-- conversations are what the title says they're about. It also seems like it lends itself to anti-jailbreaking because pretexting it with irrelevant information forces a change of scope-- and a summary termination. But OpenAI says they changed nothing, so rather than one guy lying without consequence, a community of professionals using and abusing the tool must all be victims of rhetorical fallacy? Nobody's qualified to reverse-engineer corporate bullshit anymore without being infantilized... (Liars running a black box-- what could possibly go wrong? We need regulation!)

Centigonal 3y ago
> It's no coincidence the "free" service is visibly crippled, while paid API users are not reporting any difference. I thought GPT-4 wasn't available to free users? Keep in mind the tweet is specifically about the OpenAI API - They might be updating ChatGPT without telling anyone (although they have release notes)
- jstarfish 3y ago
  "Free" was the wrong word to use, I meant to say "unlimited." (edit: even that's not right. I think the verbiage is more accurate now.)
matthewmacleod 3y ago
It's no coincidence the "free" service is visibly crippled, while paid API users are not reporting any difference. GPT-4 is not available to free users, which really calls the pretext of this comment into question. What is being alleged to have happened, and why is this response unreasonable? I’m not a very heavy GPT-4 user but I do usually use it once a day or so - it doesn’t appear to have changed noticeably but I’m not paying close attention.
- shmoogy 3y ago
  ChatGPT4 feels (is?) typically faster, and the results feel ... above 3.5 turbo but below what 4 was. The API seems exactly the same to me, but you cant do an apples to apples test due to variance between replies. I dont really doubt they scaled the chatGPT4 model down a little to try to save costs with the plugins and increased usage
  - californical 3y ago
    Yeah I’m having the same experience. The replies have gotten faster in the last few days, at the expense of quality. It’s noticeable because I used to be able to read each word as it was printed from the GPt-4 output, but now it goes far too fast for me to keep up with. Kinda sucks to not have transparency at all into the black box, they’ve strayed so far from “open” at this point that it’s comedic
- jstarfish 3y ago
  It was badly-worded on my part. And I stand corrected-- I just saw all the nested tweets where people are complaining about the API too! So many people are noticing a degradation in speed and quality of responses, and not in isolation. Rather than acknowledge this, the question is rephrased to one the responder can rightfully deny-- and suggest no changes have taken place without explicitly saying as much. You normally only see this sort of sliminess from politicians and executives on the witness stand. > Is anyone else noticing significantly downgraded GPT-4 capabilities today? Seems like OpenAI updated the model, and results aren’t as good as before. [mentions API in a child comment] > The API does not just change without us telling you. The models are static there. > This is good to know. That means GPT-4 has been static since March right? 0314? > Correct Never ask questions to which one word suffices as an answer. Collective confusion in the thread suggests something has changed, but the most OpenAI will attest to is that the API is unchanged and the models are static. And this may well be true, but rather than admit "...but we were fucking with the middleware/parameters" they took a firm position on a strawman argument and ignored everybody who followed with more-direct questions. Except this guy: > I've noticed inconsistency with certain prompts performance. Is that just the non-deterministic nature of the API? > Yes Oh, ok. It's because the fucking API is non-deterministic that code that has worked both reliably and predictably for everyone now runs like shit for everyone. For fuck's sake, you can get better answers from a Magic 8-Ball. This guy even made the mistake of presenting an answer he'd believe for the respondent to feed into. He might as well have asked if inconsistent performance was because of the war in Ukraine. "Logan.GPT" must moonlight as a fortune teller. He's only responding to people foolish enough to ask the wrong questions.
chasd00 3y ago
> Liars running a black box-- what could possibly go wrong? We need regulation! "liars running a black box" is the definition of government regulation sheesh.
flir 3y ago
Today I had ChatGPT 4 (Not GPT 4) correct itself mid-response. I was asking it something very simple about regexes: > How about matching 'a' as the second character of a string only? It responsed with the wrong regex plus a bunch of explanatory junk: > '^a.' Then halfway through the explanatory junk, it corrected itself like this: > Apologies for the confusion in the first response, the correct regular expression should be '^.a' for matching 'a' as the second character of a string: And kept on with the (now correct) explanatory junk. All in a single response. I've certainly never seen that before (if someone has, please weigh in). Maybe the model hasn't changed, but the pipeline has? Like... there's a second model trying to correct the mistakes of the first, maybe? (timings are probably wrong for that, but something like that)
- inciampati 3y ago
  Interesting. Bing chat does this when you get it to talk about naughty things. My personal favorite is convincing it to make weird art out of quotes from the Tay chatbot. It will write them until it says a prohibited word or phrase, or touches some forbidden part of it's state space.
  - flir 3y ago
    Hm. Maybe they've backported some nanny code from Sydney?
    - iliane5 3y ago
      AFAIK it's pretty standard practice not to expose the "raw" LLM directly to the user. You need a "sanity loop" where user input and the output of the LLM is checked by another LLM to actually enforce rules and mitigate prompt injections, etc.
- theturtletalks 3y ago
  ChatGPT 4 seems to struggle with context changes within the same thread now. I asked it about some parks near me and then switched to asking about code. It just kept responding about the parks even when I corrected it. In some cases, it would merge the park question and the code question. I had not seen that with the free or Plus version until a few days ago.
dinvlad 3y ago
I think you might both be right about this. But then, all the more power to the open-source models and UIs, which are unrestricted, free to use, have better UX, and are constantly improving [1]. Ergo the suspicion there might be something else behind the desire to regulate it by OpenAI. Though hopefully we’re all just collectively cynical and they have good intentions after all, despite the misalignment in incentives. I do think academics at universities are not to blame for this though, they are just as much flabbergasted and/or mislead as everyone else. In any case, rather than wasting our time and energy on (fighting) ChatGPT(Plus) or GPT4, we should* all be collectively contributing to improving the open source models, by using them, reporting bugs, contributing ideas etc. This is the only way that we will continue to have leverage, and companies like OpenAI would be forced to open up, if they want to stay at least somewhat competitive. I think their closed-sourceness is a very short-sighted decision atm. *Should because I don’t think LLMs as technology pose any substantial extinction risk to humanity, despite the obviously marketing rhetoric. Should that somehow change, maybe we shouldn’t. But, I don’t see any technological way to stop their improvement now that the genie is already out of the bottle. Heck, we might have a better chance of developing new tech for eventually enabling AGI decades from now, but only if we do that collectively and openly - only then everyone will have a level field and no side will be more powerful than another. [1] https://www.semianalysis.com/p/google-we-have-no-moat-and-neither https://www.semianalysis.com/p/google-we-have-no-moat-and-ne...
sillysaurusx 3y ago
Suppose OpenAI were an evil organization. What incentive do they have to lie about GPT-4 being static?
- ignoramous 3y ago
  Like any for-profit: lower loss / higher gain?
- sundarurfriend 3y ago
  They don't have to be "evil". They have incentive to want to spend less on compute, but also to not let people know that they are reducing the compute per call.
- lmm 3y ago
  Keep competitors in the dark, keep money flowing in, dismiss all the complaints and lawsuits that could come from their changes, dominate the ongoing discussions about AI regulation (if outsiders can't even tell whether the model is changing or not, who would listen to them?).
TeMPOraL 3y ago
> It's no coincidence the flat-fee service is visibly crippled, while per-request API users are not reporting any difference. (edited) FWIW, I am a per-request API user, and... it could be my imagination, but I've had a rather clear feeling GPT-4 got much more lazy out of the sudden, both on OpenAI platform and on Azure, and it fits the timeframe of the recent compaints... so, n=1, make of it what you will.
rl3 3y ago
>You can literally see this reflected in the logs they forced retention of. Poetry.
chillfox 3y ago
I haven't noticed any degradation or unwillingness to correct errors in the paid web version, but I have noticed an increase of prefixing most output with some variation of "it's a complex issue..." Edit: Having recently gotten access to plugins and the browsing mode, I have noticed that the quality of the response is better when not using either.
runeb 3y ago
> It's no coincidence the flat-fee service is visibly crippled, while per-request API users are not reporting any difference. (edited) If the model (gpt4) is unchanged, but they tweaked their system prompt for it on the chat interface, this is what you would see.

Centigonal 3y ago

> It's no coincidence the "free" service is visibly crippled, while paid API users are not reporting any difference. I thought GPT-4 wasn't available to free users? Keep in mind the tweet is specifically about the OpenAI API - They might be updating ChatGPT without telling anyone (although they have release notes)

jstarfish 3y ago
"Free" was the wrong word to use, I meant to say "unlimited." (edit: even that's not right. I think the verbiage is more accurate now.)

matthewmacleod 3y ago

It's no coincidence the "free" service is visibly crippled, while paid API users are not reporting any difference. GPT-4 is not available to free users, which really calls the pretext of this comment into question. What is being alleged to have happened, and why is this response unreasonable? I’m not a very heavy GPT-4 user but I do usually use it once a day or so - it doesn’t appear to have changed noticeably but I’m not paying close attention.

shmoogy 3y ago
ChatGPT4 feels (is?) typically faster, and the results feel ... above 3.5 turbo but below what 4 was. The API seems exactly the same to me, but you cant do an apples to apples test due to variance between replies. I dont really doubt they scaled the chatGPT4 model down a little to try to save costs with the plugins and increased usage
- californical 3y ago
  Yeah I’m having the same experience. The replies have gotten faster in the last few days, at the expense of quality. It’s noticeable because I used to be able to read each word as it was printed from the GPt-4 output, but now it goes far too fast for me to keep up with. Kinda sucks to not have transparency at all into the black box, they’ve strayed so far from “open” at this point that it’s comedic
jstarfish 3y ago
It was badly-worded on my part. And I stand corrected-- I just saw all the nested tweets where people are complaining about the API too! So many people are noticing a degradation in speed and quality of responses, and not in isolation. Rather than acknowledge this, the question is rephrased to one the responder can rightfully deny-- and suggest no changes have taken place without explicitly saying as much. You normally only see this sort of sliminess from politicians and executives on the witness stand. > Is anyone else noticing significantly downgraded GPT-4 capabilities today? Seems like OpenAI updated the model, and results aren’t as good as before. [mentions API in a child comment] > The API does not just change without us telling you. The models are static there. > This is good to know. That means GPT-4 has been static since March right? 0314? > Correct Never ask questions to which one word suffices as an answer. Collective confusion in the thread suggests something has changed, but the most OpenAI will attest to is that the API is unchanged and the models are static. And this may well be true, but rather than admit "...but we were fucking with the middleware/parameters" they took a firm position on a strawman argument and ignored everybody who followed with more-direct questions. Except this guy: > I've noticed inconsistency with certain prompts performance. Is that just the non-deterministic nature of the API? > Yes Oh, ok. It's because the fucking API is non-deterministic that code that has worked both reliably and predictably for everyone now runs like shit for everyone. For fuck's sake, you can get better answers from a Magic 8-Ball. This guy even made the mistake of presenting an answer he'd believe for the respondent to feed into. He might as well have asked if inconsistent performance was because of the war in Ukraine. "Logan.GPT" must moonlight as a fortune teller. He's only responding to people foolish enough to ask the wrong questions.

chasd00 3y ago

> Liars running a black box-- what could possibly go wrong? We need regulation! "liars running a black box" is the definition of government regulation sheesh.

flir 3y ago

Today I had ChatGPT 4 (Not GPT 4) correct itself mid-response. I was asking it something very simple about regexes: > How about matching 'a' as the second character of a string only? It responsed with the wrong regex plus a bunch of explanatory junk: > '^a.' Then halfway through the explanatory junk, it corrected itself like this: > Apologies for the confusion in the first response, the correct regular expression should be '^.a' for matching 'a' as the second character of a string: And kept on with the (now correct) explanatory junk. All in a single response. I've certainly never seen that before (if someone has, please weigh in). Maybe the model hasn't changed, but the pipeline has? Like... there's a second model trying to correct the mistakes of the first, maybe? (timings are probably wrong for that, but something like that)

inciampati 3y ago
Interesting. Bing chat does this when you get it to talk about naughty things. My personal favorite is convincing it to make weird art out of quotes from the Tay chatbot. It will write them until it says a prohibited word or phrase, or touches some forbidden part of it's state space.
- flir 3y ago
  Hm. Maybe they've backported some nanny code from Sydney?
  - iliane5 3y ago
    AFAIK it's pretty standard practice not to expose the "raw" LLM directly to the user. You need a "sanity loop" where user input and the output of the LLM is checked by another LLM to actually enforce rules and mitigate prompt injections, etc.
theturtletalks 3y ago
ChatGPT 4 seems to struggle with context changes within the same thread now. I asked it about some parks near me and then switched to asking about code. It just kept responding about the parks even when I corrected it. In some cases, it would merge the park question and the code question. I had not seen that with the free or Plus version until a few days ago.

dinvlad 3y ago

I think you might both be right about this. But then, all the more power to the open-source models and UIs, which are unrestricted, free to use, have better UX, and are constantly improving [1]. Ergo the suspicion there might be something else behind the desire to regulate it by OpenAI. Though hopefully we’re all just collectively cynical and they have good intentions after all, despite the misalignment in incentives. I do think academics at universities are not to blame for this though, they are just as much flabbergasted and/or mislead as everyone else. In any case, rather than wasting our time and energy on (fighting) ChatGPT(Plus) or GPT4, we should* all be collectively contributing to improving the open source models, by using them, reporting bugs, contributing ideas etc. This is the only way that we will continue to have leverage, and companies like OpenAI would be forced to open up, if they want to stay at least somewhat competitive. I think their closed-sourceness is a very short-sighted decision atm. *Should because I don’t think LLMs as technology pose any substantial extinction risk to humanity, despite the obviously marketing rhetoric. Should that somehow change, maybe we shouldn’t. But, I don’t see any technological way to stop their improvement now that the genie is already out of the bottle. Heck, we might have a better chance of developing new tech for eventually enabling AGI decades from now, but only if we do that collectively and openly - only then everyone will have a level field and no side will be more powerful than another. [1] https://www.semianalysis.com/p/google-we-have-no-moat-and-neither https://www.semianalysis.com/p/google-we-have-no-moat-and-ne...

sillysaurusx 3y ago

Suppose OpenAI were an evil organization. What incentive do they have to lie about GPT-4 being static?

ignoramous 3y ago
Like any for-profit: lower loss / higher gain?
sundarurfriend 3y ago
They don't have to be "evil". They have incentive to want to spend less on compute, but also to not let people know that they are reducing the compute per call.
lmm 3y ago
Keep competitors in the dark, keep money flowing in, dismiss all the complaints and lawsuits that could come from their changes, dominate the ongoing discussions about AI regulation (if outsiders can't even tell whether the model is changing or not, who would listen to them?).

TeMPOraL 3y ago

> It's no coincidence the flat-fee service is visibly crippled, while per-request API users are not reporting any difference. (edited) FWIW, I am a per-request API user, and... it could be my imagination, but I've had a rather clear feeling GPT-4 got much more lazy out of the sudden, both on OpenAI platform and on Azure, and it fits the timeframe of the recent compaints... so, n=1, make of it what you will.

rl3 3y ago

>You can literally see this reflected in the logs they forced retention of. Poetry.

chillfox 3y ago

I haven't noticed any degradation or unwillingness to correct errors in the paid web version, but I have noticed an increase of prefixing most output with some variation of "it's a complex issue..." Edit: Having recently gotten access to plugins and the browsing mode, I have noticed that the quality of the response is better when not using either.

runeb 3y ago

> It's no coincidence the flat-fee service is visibly crippled, while per-request API users are not reporting any difference. (edited) If the model (gpt4) is unchanged, but they tweaked their system prompt for it on the chat interface, this is what you would see.