3 ms·

> This process is extremely well-understood, and we know necessarily that it is just approximating a historical distribution. Prove that this precludes formati

> This process is extremely well-understood, and we know necessarily that it is just approximating a historical distribution. Prove that this precludes formation of mind. Prove that human minds do not work this way. Until such premises are established, your argument is not sound. This leaves open the possibility what we're seeing is in fact a degree of reasoning ability in LLMs.

mjburgess 2y ago
As I said in my other comment. Statistical models generate output with constant time/energy. So for any input y, and any output x, the computation is constant time. eg., generating any 100 tokens of output takes the same time as any other. If you reflect on that carefully, you'll note the only way this is possible is if the algorithm does not use any semantic features of the input. Indeed, if it has no relevant capacities at all. Consider e.g., the prompt, "imagine a world where..., and then infer..., and then what would be... ?" for various "..." we can choose, the output ought take arbitrarily longer to compute; but its constant. For any word: imagine, reason, suppose, simulate, believe, remember... denoting any alleged mental operation, we can trivially construct cases where actually performing this operation would take more or less time. "Given premises A, B, C, D,...; infer conclusion..." has a prompt reply which is constant in time regardless of how many premises we input. "Recall memories A, B,C,.." likewise. The only way that answering "yes" to a question, say, regardless of what that question is, taking the same time/energy/etc. to answer.. would be if no semantic feature of that question was being used in its answering. This is an LLM, it's all statistical AI: the number of operations performed per prediction is constant for all predictions. Almost no alleged capacities are consistent with this fact.
- FeepingCreature 2y ago
  The human brain computes precisely the same amount of information every millisecond. That is, the computational power of a single neuron is necessarily capped by its switching speed. This proves humans cannot think.
  - mjburgess 2y ago
    Whether that is true or not, and it obviously isn't -- the energy used by the brain is highly variable -- that isnt the point. The point is that if I ask you to do some complex task that requires a one-word answer, you will take a long time to think about it. If I ask an LLM, it will always take the same amount of time, regardless of the task. It follows from P =/= NP that this should necessarily not happen if the LLM is actually evaluating the semantics of the question. However, it also follows from just the properties of what capacities are being claimed Reasoning, as an algorithm, is at least, O(NumberOfPremises) etc. The number of operations performed, in every case, for all statistical AI algs, is the same. It's not O(ANY PROPERTY OF THE PROBLEM) -- this is a catastrophe for anyone who would claim that a statistical algorithm considers the problem at all. Statistics here is just a very naive short-cut. Rather than solve the problem, you're just sampling from previous answers. This, of course, should be obvious to any one with the bare minimum of knowledge of applied statistics and the formalism of ML. Neverthless, of course, it isnt. One has to imagine that overwhelming amounts of corporate propaganda and scifi wishfulfilment are at work.
    - FeepingCreature 2y ago
      The energy used by the human brain is highly variable because the brain has been optimized to reduce power use. This was a going concern in the ancestral environment. It has not been a great concern in the developmental environment of LLMs, so they are not optimized towards this. There are in fact many papers on how to optimize energy use of LLMs, and some amount of them may have already been deployed. You note this is offtopic, but I disagree. This is part of your fundamental misunderstanding of how to make good use of LLMs - and how LLMs make good use of context space. > The point is that if I ask you to do some complex task that requires a one-word answer, you will take a long time to think about it. If I ask an LLM, it will always take the same amount of time, regardless of the task. This is simply not the case. I will take a long time to think about it, because my brain is generating tokens in the conscious workspace. I will take roughly the same time for every token, same as a LLM. (I believe that's what brain rhythms are about.) However, as opposed to a LLM, I can do computation without echoing it to stdout. If you stuck a probe in my conscious workspace, I believe you would find it looking suspiciously similar to the output stream of a multimodal LLM! And there are several papers to the extent of - hey, LLMs do in fact give better answers if you let them insert filler words, or even dots or spaces or invisible tokens - because they learn to use the additional time you give them to -- think. Cough just like us cough. That's why all these tasks that tell the LLM to "not output any extraneous information but only give the answer" are so dumb and misleading. They're literally telling the LLM to blurt out the first thing that comes to mind. In other words, you think the context window is speech, which is why you are confused. The context window is the LLM's axis of time. That their time is conflated with speech is the primary reason why LLMs have a reputation for being glib, surface speakers - and why they continue to underperform. They literally cannot stop to think - not because of any fundamental inadequacy of the technology, but just because that's how we've set them up. I suspect the amount of boilerplate text given before an answer correlates directly with quality of the answer. That's why the energy is misleading - the computational work isn't just the token, but every token between the point where the answer could be computed and where the LLM was forced to commit to a final answer. And as one could expect from this, if you give the LLM time to think without speaking, their performance shoots up significantly. The moment somebody at OpenAI reads the QuietSTaR paper (GPT-2 scale network with GPT-3 quality!) and understands what it means, the timer to AGI begins.

mjburgess 2y ago

As I said in my other comment. Statistical models generate output with constant time/energy. So for any input y, and any output x, the computation is constant time. eg., generating any 100 tokens of output takes the same time as any other. If you reflect on that carefully, you'll note the only way this is possible is if the algorithm does not use any semantic features of the input. Indeed, if it has no relevant capacities at all. Consider e.g., the prompt, "imagine a world where..., and then infer..., and then what would be... ?" for various "..." we can choose, the output ought take arbitrarily longer to compute; but its constant. For any word: imagine, reason, suppose, simulate, believe, remember... denoting any alleged mental operation, we can trivially construct cases where actually performing this operation would take more or less time. "Given premises A, B, C, D,...; infer conclusion..." has a prompt reply which is constant in time regardless of how many premises we input. "Recall memories A, B,C,.." likewise. The only way that answering "yes" to a question, say, regardless of what that question is, taking the same time/energy/etc. to answer.. would be if no semantic feature of that question was being used in its answering. This is an LLM, it's all statistical AI: the number of operations performed per prediction is constant for all predictions. Almost no alleged capacities are consistent with this fact.

FeepingCreature 2y ago
The human brain computes precisely the same amount of information every millisecond. That is, the computational power of a single neuron is necessarily capped by its switching speed. This proves humans cannot think.
- mjburgess 2y ago
  Whether that is true or not, and it obviously isn't -- the energy used by the brain is highly variable -- that isnt the point. The point is that if I ask you to do some complex task that requires a one-word answer, you will take a long time to think about it. If I ask an LLM, it will always take the same amount of time, regardless of the task. It follows from P =/= NP that this should necessarily not happen if the LLM is actually evaluating the semantics of the question. However, it also follows from just the properties of what capacities are being claimed Reasoning, as an algorithm, is at least, O(NumberOfPremises) etc. The number of operations performed, in every case, for all statistical AI algs, is the same. It's not O(ANY PROPERTY OF THE PROBLEM) -- this is a catastrophe for anyone who would claim that a statistical algorithm considers the problem at all. Statistics here is just a very naive short-cut. Rather than solve the problem, you're just sampling from previous answers. This, of course, should be obvious to any one with the bare minimum of knowledge of applied statistics and the formalism of ML. Neverthless, of course, it isnt. One has to imagine that overwhelming amounts of corporate propaganda and scifi wishfulfilment are at work.
  - FeepingCreature 2y ago
    The energy used by the human brain is highly variable because the brain has been optimized to reduce power use. This was a going concern in the ancestral environment. It has not been a great concern in the developmental environment of LLMs, so they are not optimized towards this. There are in fact many papers on how to optimize energy use of LLMs, and some amount of them may have already been deployed. You note this is offtopic, but I disagree. This is part of your fundamental misunderstanding of how to make good use of LLMs - and how LLMs make good use of context space. > The point is that if I ask you to do some complex task that requires a one-word answer, you will take a long time to think about it. If I ask an LLM, it will always take the same amount of time, regardless of the task. This is simply not the case. I will take a long time to think about it, because my brain is generating tokens in the conscious workspace. I will take roughly the same time for every token, same as a LLM. (I believe that's what brain rhythms are about.) However, as opposed to a LLM, I can do computation without echoing it to stdout. If you stuck a probe in my conscious workspace, I believe you would find it looking suspiciously similar to the output stream of a multimodal LLM! And there are several papers to the extent of - hey, LLMs do in fact give better answers if you let them insert filler words, or even dots or spaces or invisible tokens - because they learn to use the additional time you give them to -- think. Cough just like us cough. That's why all these tasks that tell the LLM to "not output any extraneous information but only give the answer" are so dumb and misleading. They're literally telling the LLM to blurt out the first thing that comes to mind. In other words, you think the context window is speech, which is why you are confused. The context window is the LLM's axis of time. That their time is conflated with speech is the primary reason why LLMs have a reputation for being glib, surface speakers - and why they continue to underperform. They literally cannot stop to think - not because of any fundamental inadequacy of the technology, but just because that's how we've set them up. I suspect the amount of boilerplate text given before an answer correlates directly with quality of the answer. That's why the energy is misleading - the computational work isn't just the token, but every token between the point where the answer could be computed and where the LLM was forced to commit to a final answer. And as one could expect from this, if you give the LLM time to think without speaking, their performance shoots up significantly. The moment somebody at OpenAI reads the QuietSTaR paper (GPT-2 scale network with GPT-3 quality!) and understands what it means, the timer to AGI begins.