Arguing LLMs can't solve AGI by themselves is like arguing an ALU unit will never make a general purpose computer. Yes it's true, it takes more than an ALU to m

Arguing LLMs can't solve AGI by themselves is like arguing an ALU unit will never make a general purpose computer. Yes it's true, it takes more than an ALU to make a CPU and it takes more than an LLM to make _anything_. To gain an intuitive understanding what LLMs can be used for one has to understand how they work. The easiest way I think is the following. Imagine A LLM is a black box and a dictionary. A dictionary contains let's say 65000 words, each word has its number. The input called the prompt is a sequence of numbers. The model spends some time and as its output it gives us not a single number as answer, no, it gives us 65000 numbers. Each number is a probability assignment for each word in the dictionary. The most trivial approach to use this output is to find the word with the highest "probability" value and consider it the output. Then we add that word to the input and we run again until we get the number that means "this is the end". This is how most trivial LLM implementations work. However, one can be a lot more creative with that output. For example recently there was a paper describing a technique to improve source code generation quality by LLM where the words in the output are compared with various known variables or objects defined elsewhere in the code essentially making it more probable for the model to generate correctly named code items. Another method can be allowing the model to take multiple paths. For example pick 3 most probable output words, run the model 3 times with each word added to the output, and so on. In the end you have many versions of the output. Feed them back into the model one after another (or together if short enough) and ask it to score them. Choose the best one. To me this is the closest thing we have to "ponder on this for a while" for LLMs. Edit: I envy people that have access to huge amounts of hardware that can be used to run LLMs. There are so many interesting experiments one can run just by exploring various ways to process model output. It is such an obvious thing I very much doubt it's not being done as we speak by Microsoft, meta, Google. Why are there no papers about it? (other than that paper I mentioned?) Because it's considered "the secret sauce" that gives them a competitive advantage.