For my part (phd in computational linguistics, first classical „AI“, then machine learning, then deep learning since 2014, not working on NLP for the past 5 yea

For my part (phd in computational linguistics, first classical „AI“, then machine learning, then deep learning since 2014, not working on NLP for the past 5 years) - I thought I would know a thing or two about what‘s possible, and I am still in absolute awe what these models spit out. People might not be aware that just a few years ago formulating a grammatically correct sentence was a hard problem, let alone all the other things that once had names (anaphora resolution, semantics, pragmatics etc. etc.) and conference tracks and workshops dedicated to them. All that swept away by data and scalable transformer models. It is such a f*cking miracle. I would be curious what the state of understanding these models is. Haven‘t done any literature review on this but just reverse engineering whatever structure the model has come about could be enlightning (perhaps).