Humans have two modes of thinking. What is one plus one? Two! No real thinking involved, answering that question got hard wired into your brain. This is what we

Humans have two modes of thinking. What is one plus one? Two! No real thinking involved, answering that question got hard wired into your brain. This is what we are currently teaching large language models, a hard wired function from inputs to outputs. What is 51,307 minus 17,469? Now you have to actually start thinking, you have memorized a procedure to do this and you know how to follow this procedure the arrive at the answer. This is somewhat like chain of thought where intermediate results - which a human would either keep in memory or write down if it gets too much - get dumped out with the output in order to be consumable when later tokens are produced. This could also span several levels, where you first derive the procedure to follow from another procedure you have memorized to than answer the actual question. And how you solve problems can change over time, when you first learn a thing, then you usually learn a procedure to follow. When you first learn to write letters, you consciously draw a sequence of lines and arcs. After enough practice this becomes hard wired and you just write the letter. When you do not do something for a long time, then it may go the other direction, you might still remember the procedure and be able to do it step by step, but you can no longer just do it. So what is the point of this comment? While you might in principle be able to learn any function from inputs to outputs, that is create a large language model that can produce the correct answer for every question without really thinking about it, I do not think that this is practicable. Every time a human follows a step by step procedure you will essentially have to learn the completely unrolled procedure in order to produce the answer in one pass. Feed forward neural networks have no loops unless you externally feed the output back into the input. That also means you have to necessarily mix the intermediate internal states with the output of an autoregressiv feed forward system, i.e. it will never be able to behave like a human as it will constantly have to output its internal thought process. If you want to mimic a human response, you will have to hide part of the autoregression internally.