Without having the foundational model to compare to, there's really no way to evaluate whether OpenAI's continuing fine tuning is absolutely making it better or

Without having the foundational model to compare to, there's really no way to evaluate whether OpenAI's continuing fine tuning is absolutely making it better or worse. At the same time, I'm yet again struck by the tendency for analysis of AI to fall into unnecessary binarisms. It's most probable that continued fine tuning is going to result in increased performance in the ways the NN is generally being used while decreasing performance in its broadest set of capabilities. So things like getting better at prompt gotchas but worse at its variety of vocabulary or style. So no, GPT-4 probably is getting worse over time. Just as it is also getting better over time. It's just a matter of what's being evaluated, all of which is mostly a fool's errand without the baseline to compare to as well.