3 ms·
> Finding edge cases is not a great way to evaluate probabilistic processes. These aren't "edge cases". They are proofs by contradiction: Claim 1: LLMs ha
by mjburgess 2y ago
> Finding edge cases is not a great way to evaluate probabilistic processes.
These aren't "edge cases". They are proofs by contradiction:
Claim 1: LLMs have a fixed computation budget per token
Claim 2: LLMs engage in reasoning
Claim 3: Reasoning is an unbounded process whose time is a function of the complexity of the problem
Claim 4 Therefore: LLMs compute proportional to the complexity of the problem
Problem: Contradiction in Claims 1, 4.
There are an infinite number of "edge" cases to illustrate this point; but the point has nothing to do with LLMs failing on some algorithms. It's not an argument about LLM accuracy.
> Can you prove humans aren't just sampling from training data + preserving context
Yes. Above, Claim 1': Humans do not have a fixed computation budget per token. So there's no contradiction.
It will take you an appropriately long time to say "Yes" or "No" as the task complexity increases. Hence you are actually engaged in reasoning.
Though I don't find your style of argument good-faith, but wish fulfilment and arguments from ignorance. "I do not know anything about human beings, and I will tell you that you do not, so that I can preserve what I wish to be true" -- this is the position of almost all defenders of AI i've encountered, and I find it disagreeably religious.
- viraptor 2y agoThis falls apart at claim 1, because it's for a token, not the answer. You can start a reasoning process, but I don't know of anyone seriously claiming it happens in a single step. LLMs have a fixed budget per token, but the result you normally want is multiple tokens long. Engaging in chain-of-thought answer means a variable number of tokens, with the length depending on the complexity of the task. In this thread I mentioned the prompt + generated subtasks, so it's not like we're talking about single-token outputs. Or in a simple example, the answer to "using step by step reasoning, what's the colour of a white sheet" is significantly shorter than "using step by step reasoning, why is Chewbacca defence a fallacy". The more complex task automatically got more steps and more tokens dedicated to it. > Humans do not have a fixed computation budget per token. That's not even wrong. Humans don't operate on tokens. That just doesn't make sense.
- mjburgess 2y agoAt least two strategies of reply: 1) choose problems P1..n such that they have yes/no answers; 2) rephrase my argument to consider max-tokens given any prompt. But I think you could see these yourself, so I'm at a loss as to what the purpose of this game is. What is the motivation to insist that animal cognition consists of some process similar to compressing 1PB of books into frequency associations and computing a conditional probability? Is it some desire to believe one hasnt been duped by the technology? "No no, I am no fool, it really is real!" etc. ? Or wishfullfilment of some other sort? Or just the cynical view that if one is nihilistic about people one is most likely right? (Each, of course, a pseudoscience of its own sort).