Googler, but opinion are my own. More than that, I believe we will hit a ceiling when the impossibility of these models to incorporate causality becomes eviden

Googler, but opinion are my own. More than that, I believe we will hit a ceiling when the impossibility of these models to incorporate causality becomes evident. Right now, LLMs are trained by predicting the next word given a context (the prompt). This approach, IMHO, gives a resemblance of cause/effect because the training data is made by humans and obviously we are able to express ourselves and reason in those terms. So we have a poor proxy for that which, also IMHO, partially explains why LLMs performance degrades when asked to solve novel problems (there was an entry few days ago about this).