I think we have to see LLMs as their own weird thing, whose capabilities are only just being seen now. For a lot of the time, ChatGPT does actually act like it

I think we have to see LLMs as their own weird thing, whose capabilities are only just being seen now. For a lot of the time, ChatGPT does actually act like it can reason. Going through a bag of data and answering a question you hadn't heard before is reasoning. For instance right now, I've been asking it how to move a postgres database from one machine to another, and it gave a coherent answer that works. Of course it's true that this information was on the internet in various forms already, but if you gave this task to a junior dev and asked him to figure it out, you wouldn't say the kid couldn't reason, would you? Even if it was slightly wrong, it wouldn't cross your mind that he hadn't substantially understood the task and made progress on it. OTOH, there are cases when the LLM just doesn't get it. Most commonly with images, eg the famous hands problem. Somehow even after looking at countless images with hands in them and having access to countless anatomy books, it doesn't know what shape a hand can take and what shapes it can't take. It seems to not have a model of _why_ hands can be these different shapes but not those different shapes. Perhaps this is do with LLMs being particularly good at text, I wouldn't know. It does seem to me like I've never seen it give a grammatically incorrect answer to anything, ever. Even when it answers something gibberish, it answers it in correct English.