I am not sure that GPT-4 is not better in a fundamental way than GPT-3.5. To me they seem like night and day. If GPT-5 is a similar jump, it will be impossible

I am not sure that GPT-4 is not better in a fundamental way than GPT-3.5. To me they seem like night and day. If GPT-5 is a similar jump, it will be impossible to compete without using it (or using a related / similar model). Yes they are both GPT models and trained as simple autoregressive LM, but there is a dramatic change you can experience at a personal level once GPT-4 can synthesize information correctly to address your specific requests in so many different contexts that GPT-3.5 was simply parroting like a toddler. All of LLM is just probabilistic inference on large bodies of text, however I do buy the idea that with enough compute and data a sufficiently large model will build the architecture it optimally needs to understand the data in the best possible way during training. And once the data becomes multimodal, the benefit to these probabilistic models can theoretically be multiplicative not just additive as each new modality will clarify and eliminate any previously wrong representations of the world. Yes, we will all laugh at how good GPT-10 trained with text, image, video, audio, and taste sensors will be, and yet GPT-4 was a major step forward, much bigger than any step taken by humanity so far.