> the narrative itself is a circular proposition I only use the simile to make the text more approachable to leymen. I don't ever conclude that because embeddi

> the narrative itself is a circular proposition I only use the simile to make the text more approachable to leymen. I don't ever conclude that because embeddings are like thoughts, thoughts are like embeddings---or LLMs are intelligent because they have thoughts in the form of embeddings. That is not a rational inference to make IMO. I don't think we should avoid similes and metaphors because of the risk of this misunderstanding. A is like B does not mean A is the same as B. > I disagree. I think the problem word is "next". The structure of an LLM's model is not one dimensional, and the path of a continuation function is not linear. The LLM is also not alone in this story: there's also the original human-written text it was made from, and the human-written text it is prompted with. Those had to be given to the LLM. You can do many things with LLMs but when they are run during inference (like the commonly known case of ChatGPT), they are predicting the next token, so calling them next-token predictors is fine. The LLM is predicting the next token after the human-written text it is prompted with. Nowhere is it implied that that is not a part of the story. > The prompt gets divided into words, and each word gets assigned to a human. The article states that the API should be as human-like as possible. They would thus certainly not give each person a single word. They would give each prompt to a single person so they had the entire context. The divvying is amongst different prompts, not words. > The humans aren't thinking about what your prompt means. The point of the hypothetical is that to make this LLM-like API, you have to employ humans, and you have to have them think about the output. To make it as human-like as possible, you can just pay them to think about it. > Except that you did just explicitly answer that question. A person using the API is not able to tell if there is a human behind it, or an LLM. That is the point. > All we are saying is that real intelligence is present, and that the LLM extends that intelligence. There is no LLM in the hypothetical you are referring to. It happens in the year 2000. > What we absolutely failed to prove is that the LLM is intelligent or that it does intelligence. The segment you are referring to is just making the argument that a next-token predictor can be intelligent. Not that all next-token predictors are intelligent, or are the same. If you read this segment: https://www.arnaldur.be/writing/about/large-language-model-reasoning#our-reasoning-does-not-occur-in-a-vacuum https://www.arnaldur.be/writing/about/large-language-model-r... You will see the arguments for LLMs being able to reason, why they in some cases seem unable to, and how it should be expected in some cases.