It's more a comment on the capabilities of smaller models, the quality of output outside of benchmarks is always subjective and you'd need something like Chatbo

It's more a comment on the capabilities of smaller models, the quality of output outside of benchmarks is always subjective and you'd need something like Chatbot Arena (https://chat.lmsys.org/ https://chat.lmsys.org/) to evaluate it more quantitatively. Even after filtering out the common cheat techniques like merges, there are still 7B and 13B near the top, but yes it's still possible to train models on the evaluation datasets without decontamination. If you look at the Chatbot Arena leaderboards there are still decently-high ELOs for 7B models.