I would call understanding what's the underlying mechanism behind LLMs and other deep models such a problem. I.e. what's the internal representation? How it eme

I would call understanding what's the underlying mechanism behind LLMs and other deep models such a problem. I.e. what's the internal representation? How it emerges over training? What its failure modes. There're two reasons: - In order to substantially improve these models we need to understand what they do. Something similar happened with steam engines when thermodynamics helped to explain what they do and improve them substantially. - We need to understand their failure modes in order to understand their safety profile. I.e. it's AI safety problem.