3 ms·

ChatGPT 3.5 has been improving in its creative output pretty much every other week since I started looking at it. The improvements don't get announced, things j

ChatGPT 3.5 has been improving in its creative output pretty much every other week since I started looking at it. The improvements don't get announced, things just sort of change. Breaks prompts all the time though! For my usage, sometime around 3 weeks ago was probably peak, a few days ago its started getting really wordy, so much so that it runs into the 4k token limit before it can complete stories now!

TeMPOraL 3y ago
Obligatory follow-up question: I'm guessing you have a reason to only mentioning and using GPT-3.5 in this project, but in terms of more general evaluation of complex prompt handling, did you have a chance to test GPT-4, and if yes, how does it fare in comparison? I predict that, if you tested it, the answer will be "GPT-4 is much better than GPT-3.5 at this" - but the part I'm interested in is hearing your opinion on how much and how exactly is GPT-4 better at handling these kinds of prompts/interactions. (If my prediction is wrong and GPT-4 isn't much better than GPT-3.5 here, that would be even more interesting to hear about.)
- refulgentis 3y ago
  Significantly better, a more comprehensive answer than I could ever provide is the Sparks of AGI paper.
  - andrevoget 3y ago
    https://arxiv.org/abs/2303.12712 https://arxiv.org/abs/2303.12712
- com2kid 3y ago
  I'm using GPT3.5 because it is a public facing site running off of my personal credit card. :) Back when I first started out, GPT-4 was generating more creative stories for sure, and it obeyed instructions better. I have had to do a lot of prompt engineering to make sure GPT3.5 keeps to the format I have specified. If you take a look at the source I linked to, I basically taught GPT3.5 a syntax that uses emojis, which is super cool because in addition to the emojis I gave examples of, GPT3.5 has come up with its own use cases. For example when a princess got exiled, it used a door emoji! Sadly I have had to turn the temperature down to keep things on track, at higher temperatures GPT3.5 tends to forget the syntax. GPT4 is easier to direct, it follows instructions much better, and the stories are possibly better, but I haven't used it nearly as much for this particular project.

TeMPOraL 3y ago

Obligatory follow-up question: I'm guessing you have a reason to only mentioning and using GPT-3.5 in this project, but in terms of more general evaluation of complex prompt handling, did you have a chance to test GPT-4, and if yes, how does it fare in comparison? I predict that, if you tested it, the answer will be "GPT-4 is much better than GPT-3.5 at this" - but the part I'm interested in is hearing your opinion on how much and how exactly is GPT-4 better at handling these kinds of prompts/interactions. (If my prediction is wrong and GPT-4 isn't much better than GPT-3.5 here, that would be even more interesting to hear about.)

refulgentis 3y ago
Significantly better, a more comprehensive answer than I could ever provide is the Sparks of AGI paper.
- andrevoget 3y ago
  https://arxiv.org/abs/2303.12712 https://arxiv.org/abs/2303.12712
com2kid 3y ago
I'm using GPT3.5 because it is a public facing site running off of my personal credit card. :) Back when I first started out, GPT-4 was generating more creative stories for sure, and it obeyed instructions better. I have had to do a lot of prompt engineering to make sure GPT3.5 keeps to the format I have specified. If you take a look at the source I linked to, I basically taught GPT3.5 a syntax that uses emojis, which is super cool because in addition to the emojis I gave examples of, GPT3.5 has come up with its own use cases. For example when a princess got exiled, it used a door emoji! Sadly I have had to turn the temperature down to keep things on track, at higher temperatures GPT3.5 tends to forget the syntax. GPT4 is easier to direct, it follows instructions much better, and the stories are possibly better, but I haven't used it nearly as much for this particular project.