3 ms·
I've noticed the exact opposite, in my experience GPT3.5 is more likely to make code that doesn't work correctly, mess up URLs and other strings, or forget abou
by thewataccount 3y ago
I've noticed the exact opposite, in my experience GPT3.5 is more likely to make code that doesn't work correctly, mess up URLs and other strings, or forget about certain variables and not include them. It's also more likely to make up non-existent libraries and other code.
GPT4 in my experience has been much more consistent with it's results. It also doesn't seem to lose the context as quickly. And it can deal with more niche libraries a lot better.
Due to the lack of consistently I've found it difficult to use 3.5
I haven't tried it with LangChain type stuff so maybe it's different there?
As a side note - I've been noticing their performance change a lot in between their "updates" and they mostly seem to be getting worse at following instructions :/
- halfjoking 3y agoYeah GPT4 seems worse lately. I had it build a complex tree view and make incremental feature updates which really impressed me. But last week or so it's giving me junk - messing up variables or rearranging code for no reason even though the file I'm copying/pasting as reference hasn't changed in size.
- gwern 3y agoLikewise. I find that I regularly hit roadblocks with GPT3.5 and then switching the convo to 4 (in the Playground, so I can just change the toggle on the side), 4 often gets it right. Increasingly, I just start with 4.