5 ms·
His point is that it literally didn't change, that includes the safeguards (which are a part of the model).
by H8crilA 3y ago
His point is that it literally didn't change, that includes the safeguards (which are a part of the model).
- jeremyjh 3y agoHe said the API hasn't changed. But what about the Chat website?
- jacquesm 3y agoExactly my question... I wonder if the pre and post processing of the chat interaction isn't what is driving the perceived differences.
- danielbln 3y agoI could not figure out what everyone was talking about yesterday as my experience with gpt4 has not degraded at all, but I'm using a third party client via API.
- flir 3y agoWhich 3rd party client, if you don't mind me asking? I'm looking to move, and the space isn't mature enough yet for there to be a clear leader.
- danielbln 3y agoWe've been using https://github.com/cogentapps/chat-with-gpt https://github.com/cogentapps/chat-with-gpt for a while, and overall happy with it, but its development is mostly dead.
- throwuwu 3y agoI tried comparing the API response against the chat on the same question a few times. There isn’t a huge difference but I’d pick the responses from the API over the ones from chat. Hard to say though, could be RNG.
- IanCal 3y agoCould be related to the system prompt.
- ethbr0 3y agoAren't there archived transcripts with prompts now? Seems like we need "model transparency" and log implementers to flag drift, a la RFC 9162 / Certificate Transparency.
- throwuwu 3y agoI guess we could go back through our histories and resubmit the same prompts a few times. Wouldn’t be a fair comparison since we only have 1 sample from the old version.
- aliston 3y agoIs it true that the safeguards are considered part of the model? I had assumed that the "safeguards" that limit certain types of responses in ChatGPT were separate from the actual language model.
- helpfulclippy 3y agoIt seems to me that there’s lots of room to change stuff that profoundly affects the range of responses without altering the base model. The prompt template alone seems like a place outside the model where we’ve seen safeguards get implemented, and other stuff that affects the usefulness of a model’s responses.
- z3c0 3y agoI'm of the same mind, and I believe they're keeping their ear to the ground to address any jailbreaks and loopholes. I have tricked GPT-4 into spitting out text it shouldn't have, only to have it dance around the same prompts less than 24 hours later. This "the model is the same" response seems like a deliberate deflection meant to mask the mechanical turk that ChatGPT is becoming.
- kmod 3y agoMy understanding is that most of the safeguards are inside the model, but there are some safeguards that are outside the model. In particular if you ask the API to generate copyrighted data it will, but then the connection will mysteriously break after the first few words which I assume is a separate system watching the responses.
- weird-eye-issue 3y agoChatGPT is not the same thing as the underlying model. ChatGPT is just a UI over the model. The tweet was about the model.
- danpalmer 3y agoChatGPT is also the fine tuning and prompting of the model. It’s a distinct set of weights from “raw” GPT-4/etc, it’s just not a foundational model.
- throwuwu 3y agoThe moderation endpoint is separate for the API so I’d imagine it’s the same for chat. The model could be the same while they change: Moderation Temperature / top-p System prompt Some other internal system we aren’t aware of
- aeternum 3y agoI wouldn't read too far into the tweet. They do tell us when it changed, and the bottom of the page clearly states : ChatGPT May 24 Version The release notes are producty and not very technical so it is difficult to tell what actually changed.
- onlyrealcuzzo 3y agoMy immediate gut reaction at the original post about GPT-4 get SUBSTANTIALLY worse was that... It didn't. People were just noticing LLMs still have a long way to go after using them more. I was shocked going through the thread that all of the popular comments were confirmations that it did, in fact, get MUCH worse. It's nice to see from OpenAI that it didn't...
- anticensor 3y agoThe inhibitive ("moderation") model is separate from the generative ("chatbot") model. They work in tandem.
- greenknight 3y agoMy understanding, is that ChatGPT... has a prompt that goes before your question (and previous answers) that set up the stage for how it should respond. It could be that the safeguards they have put in place, sit in this prefix prompt state... rather than GPT4... and that you would get the normal answers via the api rather than via ChatGPT.
- jiggawatts 3y agoThe employee is saying the API version didn't change, but the Chat web UI did definitely change.
- MuffinFlavored 3y agoThere's no way. I've been using it basically every day since it launched and it was only the first time ever I recently got a "that's too complex for me to go into detail, here's an overview" response.