Well no of course not. Not only are they still only predicting the next best word ("best" now working better thanks to attention), but anybody who's role played

Well no of course not. Not only are they still only predicting the next best word ("best" now working better thanks to attention), but anybody who's role played with an LLM learns many caveats of the architecture. I think especially regarding spatial awareness/object permanence etc LLMs lack any ability to work with these concepts when it comes to predicting the next token. I feel like we need to add more specialised heads (which I think some training regimen for LLMs already courages) for things like these concepts. It's easy for them to lose track of where an item is and you have to catch its errors early and correct it, otherwise it'll take the error and run with it.