2 ms·
You worded your comment much better than I did, and you touch on the key point in all of this which is the assumption that chatGPT etc will continue to get noti
by therobots927 2y ago
You worded your comment much better than I did, and you touch on the key point in all of this which is the assumption that chatGPT etc will continue to get noticeably better at regular intervals. It’s entirely possible that openAI has reached a local maximum with the data, compute, and model structure they have available.
The data is tapped out for the most part. They already probably acquired 80% of the available text training data.
Compute is now heavily cost constrained. This is obvious because even GPTs responses have gotten worse over the last 6 months, especially with image generation tasks. Likely because they’re throttling compute.
Model structure: OpenAI has admitted that they cannot understand why chatGPT has generated a given output. If you can’t explain your model you can’t improve it either. And more compute won’t help you that much here if I had to guess.