> why the models are so bad with simple thinking outside the box solutions There is no outside the box in latent space. You want something a plain LLM can’t do

> why the models are so bad with simple thinking outside the box solutions There is no outside the box in latent space. You want something a plain LLM can’t do by design - but it isn’t out of question that it can step outside of its universe by random chance during the inference process and thanks to in-context learning.