The problem is going from results on curated datasets - usually with at least a little implicit overfitting, since even if you reserve test data, it's typically

The problem is going from results on curated datasets - usually with at least a little implicit overfitting, since even if you reserve test data, it's typically chosen in the dataset creation process - to data in the wild. BabyTalk had that cool result in 2013, and there have been great papers following up in that area, but there's still not, e.g., a website where you can upload a photo and reliably get an accurate description. It's not because nobody's tried, it's because it's hard - neural network vision methods are still generally quite brittle to unexpected variations in data, and "accurate description" is actually a very loaded term that typically needs some sort of in-the-world contextualization to unpack.