3 ms·
But it does not seem to know when it is right or not does it? It doesn't seem to be conscious of its own reasoning or is it? And if you ask it about its reasoni
by galaxyLogic 4y ago
But it does not seem to know when it is right or not does it? It doesn't seem to be conscious of its own reasoning or is it? And if you ask it about its reasoning it could give an answer that might or might not be true right? So what confidence can we have that it is actually doing reasoning in one form or another?
When we see a movie on the screen we see what looks like people speaking and doing things but really it's just a recording. The images on the screen or celluloid possess no intelligence even though on the screen they seem to behave intelligently.
The difference between movies and the language models is that the so called AI-bot is like a movie which has a big tree of possible plot-choices it can take depending on how the audience cheers or boos. That doesn't mean there is any process we could call "reasoning" behind the bot's answers. You know like logical deduction.
A person can explain how they reasoned some conclusion. And so can SYMBOLIC AI. But these trained language models they just mimic the observed speech, not the reasoning processes that go on inside the brains of people who are speaking. If they were we should be able to produce a trace of such reasoning, and the language models should also be able be able to describe it to us, like humans can.
- astrange 4y agoYou can ask it questions noone has ever asked before, that it can't possibly have memorized the answers to, and simply say "explain the answer step by step". It's not guaranteed that it will answer correctly, but it often does answer correctly and with a chain of reasons. The cases where it answers incorrectly are sometimes reasoning failures, sometimes because it doesn't know its sources (because the model training doesn't "cite its sources" when dumping knowledge in there), and sometimes because of fixed-function parts of the model (like how it doesn't see individual numbers/letters, but rather tokens using compressed "byte pair encoding".)
- Terretta 4y agoYou're tricked by probabilities. In the joke explainer example posted here a couple weeks back, the "chain of reasons" why each joke was funny was dead wrong though probably right for most jokes. It has no reason, only likelihoods. Your belief it has a chain of reasons is usually confirmed because the dumb probability-selected list is, by design, probably plausible.
- galaxyLogic 4y ago> It's not guaranteed that it will answer correctly, Why? Because its reasoning-algorithm is not correct, because it has none. All it has is statistics, likelihoods, which makes it look like as if there was some reasoning behind it.
- astrange 4y agoStatistics are the only thing anything has. You don’t have an algorithm either. Or rather, you (or a CPU or an ML model) have a “statistical” base layer and then other abstractions on top of that which are capable of being (relatively) deterministic on top of the unreliable execution machinery.
- galaxyLogic 4y ago> You don’t have an algorithm either. Are you saying algorithms don't exist, can't be followed and reasoned about, and proven correct, or not?