4 ms·

4. We don't actually have a coherent concept of what "reasoning" means in general. Same for "hallucination"...

ninetyninenine 1y ago
We do. Reasoning is clearly defined. Sentience or consciousness is not. Reasoning is basically the same as using logic to arrive at a conclusion when given a set of rules and axioms. Hallucination is producing a conclusion by not following rules or axioms.
- staunton 1y ago
  You could go further and say that anything a computation could ever possibly do is a matter of using logic to arrive at a conclusion given just the right rules and axioms. That would mean anything going beyond that (potentially "sentience" or "consciousness", possibly even -- as you seemed to suggest -- "hallucination", though apparently a computer program can do it) would have to be "magic" that's impossible to even describe using logic-based science, let alone simulate by a program. (Could we have science that transcends logic?) My original claim refers to the mundane situations where people get an LLM output which isn't what they wanted and go "see, it isn't reasoning! That's the problem!". For each such instance (or at least most of them), the "rules and axioms" that would have to be used to arrive at the desired conclusion would be different, sometimes incompatible, and in any case the people demanding "that it reason" wouldn't be able to spell those rules and axioms out, often even given years to do it, at least without allowing them to be obviously contradictory or trivially close to the specific conclusion or output they want. (Incidentally, that's why LLMs are interesting in the first place!) So sure, you can trick an LLM into failing on a syllogism problem, and that does mean "it can't reason" in a certain narrow sense, but I don't think that's what we're actually talking about when someone is contrasting "reasoning" and "hallucinating", which is what I was originally objecting to. That's what I meant when I said that we don't have a coherent concept of what "reasoning" means in general.

We do. Reasoning is clearly defined. Sentience or consciousness is not. Reasoning is basically the same as using logic to arrive at a conclusion when given a set of rules and axioms. Hallucination is producing a conclusion by not following rules or axioms.

staunton 1y ago
You could go further and say that anything a computation could ever possibly do is a matter of using logic to arrive at a conclusion given just the right rules and axioms. That would mean anything going beyond that (potentially "sentience" or "consciousness", possibly even -- as you seemed to suggest -- "hallucination", though apparently a computer program can do it) would have to be "magic" that's impossible to even describe using logic-based science, let alone simulate by a program. (Could we have science that transcends logic?) My original claim refers to the mundane situations where people get an LLM output which isn't what they wanted and go "see, it isn't reasoning! That's the problem!". For each such instance (or at least most of them), the "rules and axioms" that would have to be used to arrive at the desired conclusion would be different, sometimes incompatible, and in any case the people demanding "that it reason" wouldn't be able to spell those rules and axioms out, often even given years to do it, at least without allowing them to be obviously contradictory or trivially close to the specific conclusion or output they want. (Incidentally, that's why LLMs are interesting in the first place!) So sure, you can trick an LLM into failing on a syllogism problem, and that does mean "it can't reason" in a certain narrow sense, but I don't think that's what we're actually talking about when someone is contrasting "reasoning" and "hallucinating", which is what I was originally objecting to. That's what I meant when I said that we don't have a coherent concept of what "reasoning" means in general.