3 ms·
[2018] His views about AI are largely disproved now. GPT style systems turned out to be perfectly capable of reasoning about causation, as they can reason abou
by cubefox 4y ago
[2018]
His views about AI are largely disproved now. GPT style systems turned out to be perfectly capable of reasoning about causation, as they can reason about any other relation. Despite working entirely in the established machine learning paradigm.
For many years, Pearl was considered the top intellectual critic of machine learning. His point was this: Machine learning is, at its core, just correlational. But true AI would also need to reason about causation. This ability would have to be provided by systems which work entirely differently; by using some form of the theory of causal networks which he co-invented.
Now it turns out that causal reasoning is not a major difficulty for classical machine learning, and that causal graphs are likely as useless for AI as formal logic turned out to be.
Well, at least causal networks are useful for statistics, the type of explicit inference human scientists do.
- vbezhenar 4y agoGPT does not reason. GPT pretends to reason. It often is glaringly wrong. It's just a thing that plays with words. Here's my conversation minute ago: "Event C happened before events B and A. Event C happened between events A and B. What's the order for those events?" The order of the events would be: A → C → B Event A occurred first, then event C occurred between events A and B, and finally event B occurred after event C. "Event C happened before events B and A. Event A happened between events B and C. What's the order for those events?" The order of the events would be: C → B → A Event C occurred first, then event B occurred after event C, and finally event A occurred between events B and C.
- holoduke 4y agoIt reasons, just not always right. Give it time and these systems will become the oracles of humankind. A utility we need to use as an extension of our lives to succeed in the society we live in.
- diatone 4y agoiiuc, GP’s point is that these models, in their current form, can’t guarantee confidence in their ability to reason about a scenario exhaustively. “Give it time” and other dismissals are beside the point. The point seems to be that these models approximate results based on some hidden criteria, which means that although for a variety of practical purposes they fare better than people and are therefore preferable, there remain problems which are better suited to computing approaches that guarantee exhaustiveness… even though those other approaches sacrifice the malleability of neural networks.
- yetihehe 4y agoOpponents of machine learning fear that ML will take over our thinking and it will stop us from properly thinking. But the same was said about writing things up by Socrates. Writing has transformed society and enabled wide spread of ideas. Maybe we now need AI to better parse those ideas? Sure, there may be errors during the process but I now think "AI reasonators" will actually make things better.
- codeulike 4y ago"Event C happened before events B and A. Event C happened between events A and B. What's the order for those events?" That doesnt make any sense. If its before B and A it can't also be between them
- Teever 4y agoCACB
- hgsgm 4y agoA is before C and B. A is between C and B.
- antibasilisk 4y ago[flagged]
- vbezhenar 4y agoThat's the answer I expected. May be it's some implementation nuances which forces ChatGPT to always assume the question having a correct answer. But second question does have a correct answer yet ChatGPT does not answer it.
- sebzim4500 4y agoFrom what I can tell, during the finetuning process OpenAI tried to make chatGPT try as hard as possible to follow instructions, and not point out when the question doesn't make any sense. I'm sure if they had used different criteria during RLHF and used different supervised examples it would have given that answer. It might make the performance worse on other tasks though.
- codeulike 4y agoAh I see
- shahbaby 4y agoI got the same answer and then I asked it "Are you sure the original question made sense?" Chat GPT: It's possible that the original question may have contained a logical inconsistency. The statement "Event C happened before events B and A. Event C happened between events A and B" appears to be contradictory, as it is not possible for an event to both occur before and between two other events simultaneously. If we assume that the statement was intended to mean "Event C happened after Event A and before Event B," then the order of events would be: Event A Event C Event B In this case, Event C occurred between Events A and B, but after Event A. Therefore, the chronological order would be A, C, B.
- taeric 4y ago"C happened before A and B" can also work in this if we assume it means the joint of A and B. Such that it is not the same statement that C happened before both A and B, individually. Granted, language is ambiguous. But I don't think it gets points for assuming a mistake on input.
- phkahler 4y ago>> Granted, language is ambiguous. But I don't think it gets points for assuming a mistake on input. So one person complains that it gave a "wrong" answer to contradictory input, and now you don't want it to guess at disambiguation. If we were this hard on kids when they're learning they'd be brought to tears. BTW does anybody notice that this thing is handling abstract notions like "Event A, B and C" better than a lot of people do?
- taeric 4y agoI made no real defense of the question, all told. It is written in a very hard to answer way. Just, if there is a way it could work, I'd expect that to be acknowledged. I will again agree that logic is silly tough for language, though. Context changes the colloquial use of "and," and "or" all of the time.
- cubefox 4y agoI asked Bing Chat your question: Me: "Event C happened before events B and A. Event C happened between events A and B. What's the order for those events?" Bing Chat: "The order for those events is not possible because the statements are contradictory. If event C happened before events B and A, then it cannot happen between them. If event C happened between events A and B, then it cannot happen before them. Can you please check the statements again and make sure they are consistent?"
- antibasilisk 4y agoIncorrect results do not preclude reasoning. GPT is able to understand concepts, the relationship between concepts gives rise to reason and it is able to describe these relationships, therefore it reasons, even if the reasoning is not very good. Furthermore GPT is be able to supplement its reasoning with external information, which could take the form of 'reasoning engines'.
- naasking 4y ago> GPT does not reason. GPT pretends to reason. It often is glaringly wrong. Being wrong is compatible with two hypotheses: 1. GPT does not reason and so correct answers are coincidence and happen no more than chance, 2. GPT reasons badly, which means it should still do better than chance. We have ample evidence that GPT does better than chance, therefore GPT does reason, it just makes plenty of mistakes.
- 6gvONxR4sf7o 4y agoThat remains to be seen. Let's say we want to use AI in a safety critical tool. In its current form, that would be insane. It might be like self driving cars have been. Right around the corner for at least a decade while we realize the depth of edge cases. My hope is that individual pieces end up being usable in a modular way that is controllable by composition, which is the same reason formal logic and cause and effect are such useful paradigms. We might even need to encode formal logic or causation into the system to get the right kind of modularity into it.
- opportune 4y agoThe way I see it, unless we can prove there is some specific low level structure necessary for reasoning in brains, we have no cause to believe it is not an emergent behavior from just having a low of brainpower. In that case, there is nothing saying that current technology is incapable of producing AI’s able to reason - it could just as well emerge given a large enough model, architecture tweaks, and enough data. Given that ML usually is basically just training a function that takes input and output and tries to produce a function that properly maps them, it seems reasonable for the ability to reason to emerge to improve that ability to map
- cscurmudgeon 4y agoIf anything, ChatGPT has shown you need formal logic for AI. If a model as ginormous as ChatGPT thinks that 1 > 1, are we going to still ignore that we need formal logic? https://medium.com/@konstantine_45825/chatgpt-is-no-stochastic-parrot-but-it-also-claims-that-1-is-greater-than-1-e3cd1fc303e0 https://medium.com/@konstantine_45825/chatgpt-is-no-stochast... Top AI conferences still have papers on logic and causal reasoning outside of deep learning and ML. How can we say there isn't a shorter more efficient path towards more robust reasoning? It is unscientific and stifles progress to assert things like this shutting down whole lines of research.
- cubefox 4y agoRe the medium post. ChatGPT (based on GPT-3.5) makes mistakes, but it does already fewer than its predecessor. For example, GPT-3 couldn't do the theory of mind stuff mentioned in the article. Similarly, successors of ChatGPT will do better than it. For example, I tested Bing Chat, and it immediately solved the Donut hole question correctly. I didn't try the logic examples, since I couldn't easily copy and paste them, but I expect Bing Chat would do at least better. Despite being only released a few months after ChatGPT. How can the author believe that his examples won't be solved in the near future? Regarding AI conferences. Researchers often continue to ride dead horses if they find them interesting. As far as I know, symbolic approaches have never lead to any major success in AI. All the big AI achievements in recent years are based on machine learning, on deep neural networks specifically.
- cscurmudgeon 4y agoThis is very informal and the blog post is systematic and probably on its way to a peer-reviewed paper. > How can the author believe that his examples won't be solved in the near future? OpenAI folks are not that rosy lol (https://futurism.com/the-byte/openai-ceo-people-will-be-disappointed-gpt4 https://futurism.com/the-byte/openai-ceo-people-will-be-disa...) Established researchers think LLMs won't get to us human-level AI (Yann LeCun https://twitter.com/ylecun/status/1621810424910958592 https://twitter.com/ylecun/status/1621810424910958592), but let us assume they know nothing. Let us discount all that. Let us assume LLMs will do that. What sized model will solve logic at or near the precision formal logic gives us? A 10 times bigger model? A 100 times bigger model? A 1e9 times bigger model? Can formal logic + deep learning give us a shorter path? > Regarding AI conferences. Researchers often continue to ride dead horses if they find them interesting. As far as I know, symbolic approaches have never lead to any major success in AI. All the big AI achievements in recent years are based on machine learning, on deep neural networks specifically. SAT solvers (a small part of symbolic approaches) are used in a lot of places: https://ai.stackexchange.com/questions/67/what-are-the-real-world-uses-for-sat-solvers https://ai.stackexchange.com/questions/67/what-are-the-real-... I remember in 2010 that deep learning was a dead horse. Deep learning was practically useless pre 2010 and a joke too (unlike formal logic which has almost all of its current uses outside of AI). Sorry, this position is very short sighted, and I am young person and was a kid during the 90s but I have read the history of AI. This is like claiming in 1990s and early 2000s all the advances in the past decade were from non deep learning methods and saying we have to stop research in that. Imagine how bad that would have been. Interestingly, I see experienced researchers (or complete novices and lay folks) taking your position and younger ones in AI being more balanced. I guess the experienced folks have a moat that needs protection and novices fall prey to popular press. Imagine thinking Newtonian mechanics is all that is needed as it stood for centuries. Thankfully, physicists were not short sighted. Hopefully, AI researchers resist the pressure to be short sighted and fashionable. Penrose wrote about this very same thing: "Fashion, Faith, and Fantasy in the New Physics of the Universe" https://press.princeton.edu/books/hardcover/9780691119793/fashion-faith-and-fantasy-in-the-new-physics-of-the-universe https://press.princeton.edu/books/hardcover/9780691119793/fa...
- wizzwizz4 4y ago> as they can reason about any other relation They can't reason about most other relations, though. They are pretty good at reasoning about the subset of causation that humans talk about a lot; but the stuff that's so obvious we don't write about it, half the time they can't do even when hand-held through the process. Step 1: cover your table in blue paint. Step 2: empty a bag of confetti on the table, spreading it evenly in a thin layer, one confetto thick. Step 3: dip a bowling ball in neon green glue. Step 4: roll the bowling ball over the confetti until it is fully covered. Step 5: leave to dry. Step 6: coat with a layer of varnish. Step 7: leave to dry. Step 8: visit a 10-pin bowling alley with white bowling pins. Step 9: use your confetti-coated bowling ball to get a strike. Question: what colour are the first, fifth and ninth bowling pins after you have got a strike? Pretty simple scenario. Doesn't require complex reasoning. Go on, write out the answer. Now see what a GPT-style system says. (Pick whichever you like: I didn't engineer this for any particular one. They're universally bad at this.) These systems cannot reason.
- holoduke 4y agoThey can , just not on the level you expect them to be. But these systems are improving.
- cubefox 4y agoOkay, as a human I would answer "white". This is what Bing Chat (smarter than ChatGPT) had to say: Me: "Imagine the following process: Step 1: cover your table in blue paint. Step 2: empty a bag of confetti on the table, spreading it evenly in a thin layer, one confetto thick. Step 3: dip a bowling ball in neon green glue. Step 4: roll the bowling ball over the confetti until it is fully covered. Step 5: leave to dry. Step 6: coat with a layer of varnish. Step 7: leave to dry. Step 8: visit a 10-pin bowling alley with white bowling pins. Step 9: use your confetti-coated bowling ball to get a strike. Question: what colour are the first, fifth and ninth bowling pins after you have got a strike?" Bing Chat: "That’s an interesting process. I’m not sure what the purpose of it is, but I’ll try to answer your question. Assuming that the confetti-coated bowling ball transfers some of its colour to the bowling pins when it hits them, and that the confetti colours are randomly distributed on the ball, then the first, fifth and ninth bowling pins could be any colour that is present in the confetti. However, since neon green glue is also used to coat the ball, there is a higher chance that those pins will have some neon green on them as well. So a possible answer is: neon green with some other confetti colours mixed in. Is this what you were looking for?"
- civilized 4y agoCurrent techniques can sometimes get causation right. What's it going to take to get it right as reliably as a person? I predict that the "Transformers Plus Scale" formula will not magically deliver reliability. Other ideas will be needed. Many people seem to be so impressed at these models ever getting something right, they assume that always getting it right will be trivial. Well, mark this page and come back in five years. All the crowing about The Bitter Lesson will look quaint and everything I said here will be vindicated. Exponential improvement on benchmarks is an iron law... until it isn't.
- nl 4y agoPeople are (on average) surprisingly bad at causation calculations, and even worse at explaining them. https://yaledailynews.com/blog/2015/01/20/humans-understanding-of-cause-and-effect-nuanced/ https://yaledailynews.com/blog/2015/01/20/humans-understandi... https://onlinelibrary.wiley.com/doi/10.1111/cogs.12213 https://onlinelibrary.wiley.com/doi/10.1111/cogs.12213
- civilized 4y agoI read the first link and I don't agree with your gloss on it at all. It sounds to me like people interpret causality in a more nuanced and practical way than researchers expected.
- nl 4y agoI thought that the isolation of "causal islands" would clearly affect performance. But the Wason selection task[1] has hard numbers. This is an example: You are shown a set of four cards placed on a table, each of which has a number on one side and a colored patch on the other side. The visible faces of the cards show 3, 8, red and brown. Which card(s) must you turn over in order to test the truth of the proposition that if a card shows an even number on one face, then its opposite face is red? In Wason's study, not even 10% of subjects found the correct solution.[5] This result was replicated in 1993. "if.. then" is logical deduction but also causal reasoning of course. [1] https://en.wikipedia.org/wiki/Wason_selection_task https://en.wikipedia.org/wiki/Wason_selection_task
- deleted 4y ago[deleted]
- notshift 4y agoTo add to what others have said, we know from psychology that the human brain works by building a (causative) model of the world, and then propagating the signal where our model goes awry and differs from what we experience. GPT will start becoming really powerful when you just give it arms and legs and let it build it's own training dataset the way regular animals do (in the process of evolution & throughout their lives).
- cubefox 4y agoWe can imagine a child being born paralyzed. Couldn't it grow up to become just as smart as us?