4 ms·

Reading between the lines of the original article: 1. there are plenty of - consumer - GPUs, but it wouldn't be advantageous for OpenAI to fine tune the models

by tagyro 3y ago

Reading between the lines of the original article: 1. there are plenty of - consumer - GPUs, but it wouldn't be advantageous for OpenAI to fine tune the models to run on them, hence the artificial limitation to only use GPUs with 80+ GB. We already see plenty of projects working on adapting these models for 16-24 GB GPUs or even running on CPUs - that is obviously a problem for OpenAI 2. GPT-4 has been nerfed recently - there are plenty of discussions on this and I've also noticed it in my extended usage of both ChatGPT Plus and the OpenAI API. Also, as a developer (disclaimer) of a project on top of the OpenAI API, I've received multiple messages from users mentioning the issue. Even in the OpenAI discord there are multiple threads of developers complaining about the change to GPT-4. 3. Plugins do have PMF - in the OpenAI discord there are multiple threads about accessing plugins via the API. There are also many projects working specifically on this (AgentGPT, repository loader etc). My personal experience: as a developer, you can develop plugins but you can't charge for usage. At the same time, plugins are available only to paying users of ChatGPT Plus ... 4. OpenAI will wait and see what 3rd party projects get traction and once they see public interest in a project, will build that project. I'm willing to bet the ChatGPT iOS app was built in a couple of weeks (at most) and was a direct answer to the multitude of apps on the App Store. 5. OpenAI needs the cover of regulation. They developed their models without it but now regulation could help to prevent others from reaching their levels. At the same time, any open source projects that gain enough public interest will be copied/absorbed. This reads like a KGB press release.

justdep 3y ago
Can you elaborate on the changes to GPT-4 you've noticed specifically?
- tagyro 3y ago
  In general, the quality of the replies seems to have been affected. More specifically: - the context window seems to have been reduced: I only have access to the 4k model, but even when the prompts and replies are well under this limit, the model seems to loose context, as in, the replies have no connection to the prompt; - hallucinations: while this has been a general issue with LLM's and even taking into account a reduction in the context window, GPT-4 seems to hallucinate a lot more now; - not answering: prompts for which I have received answers before get a "my cut-off date ..." answer now. I used GPT-4 mostly via the API and I saved the threads (history) of all prompts I've been sending, so it's fairly easy for me to "test" the before/after changes. Some tests that are easy to run: - the "apple" test: write 10 sentences that end with the word "apple" - I used to get 8/10 most of the time, sometimes even 10/10. Now it's more like 6-7/10. - a friend suggested the "jwst" prompt: when was the jwst launched? Depending on the answer (sometimes it says it is scheduled to launch, sometimes it answers correctly), subsequently ask "what hour and minute" - use a 1-2k prompt and analyse the answer - this shouldn't be an issue with a 4k limit, but sometimes it will hallucinate or respond with something totally unrelated temperature: 0.2, top_p: 1, max_tokens: 4000, streaming: true edit: I've noticed that speed has been improved, but that doesn't really help when the quality suffers; also, data point of 1, but when using it from Germany (Europe?), the quality suffers, when prompting via vpn from US, it seems to be slower but the quality is better (but again, take that with a big grain of salt)

by tagyro 3y ago

justdep 3y ago

Can you elaborate on the changes to GPT-4 you've noticed specifically?

tagyro 3y ago
In general, the quality of the replies seems to have been affected. More specifically: - the context window seems to have been reduced: I only have access to the 4k model, but even when the prompts and replies are well under this limit, the model seems to loose context, as in, the replies have no connection to the prompt; - hallucinations: while this has been a general issue with LLM's and even taking into account a reduction in the context window, GPT-4 seems to hallucinate a lot more now; - not answering: prompts for which I have received answers before get a "my cut-off date ..." answer now. I used GPT-4 mostly via the API and I saved the threads (history) of all prompts I've been sending, so it's fairly easy for me to "test" the before/after changes. Some tests that are easy to run: - the "apple" test: write 10 sentences that end with the word "apple" - I used to get 8/10 most of the time, sometimes even 10/10. Now it's more like 6-7/10. - a friend suggested the "jwst" prompt: when was the jwst launched? Depending on the answer (sometimes it says it is scheduled to launch, sometimes it answers correctly), subsequently ask "what hour and minute" - use a 1-2k prompt and analyse the answer - this shouldn't be an issue with a 4k limit, but sometimes it will hallucinate or respond with something totally unrelated temperature: 0.2, top_p: 1, max_tokens: 4000, streaming: true edit: I've noticed that speed has been improved, but that doesn't really help when the quality suffers; also, data point of 1, but when using it from Germany (Europe?), the quality suffers, when prompting via vpn from US, it seems to be slower but the quality is better (but again, take that with a big grain of salt)