3 ms·
Simpler and, I'd wager, wrong. It seems to me that this stems from fundamental limitations of current machine learning tech. As far as I'm aware, not a single n
by ookdatnog 4y ago
Simpler and, I'd wager, wrong. It seems to me that this stems from fundamental limitations of current machine learning tech. As far as I'm aware, not a single neural-net based AI has displayed the ability to do "System 2" style thinking well, regardless of their training set.
We have AIs that are good at System 2 thinking, that is, symbolic AI. But they don't do System 1 thinking at all. We haven't managed to integrate the two meaningfully and I don't think anyone has a clue how to do it. It's not enough to just have the ML-based system make calls into a symbolic AI when it needs to do reasoning, that's like you have a child who doesn't know how to multiply, so you give them a calculator and declare "this child now knows how to multiply".
- yunyu 4y agoThe claim that current LLMs are not that good at deep thinking might be true in general, but it is definitely not the best or correct explanation for the rot13 task being described in the article (or other simple character manipulation tasks that can’t be represented at the subword token level, including base64 or arithmetic)