18 ms·

Artificial Intelligence Is Stupid and Causal Reasoning Will Not Fix It

104 points by trenchgun 4y ago

otikik 4y ago
Could still be a problem of scale. A single neuron is quite stupid. But put enough of them together and they become … still stupid, but it’s more difficult to tell.
- nightski 4y ago
  AI will consume half the electricity on earth and it will still be pretty dumb at this point lol. We need to find a better solution than scale up.
  - gambiting 4y ago
    Is that based on anything, or just a gut feeling?
    - nightski 4y ago
      It's based on the power consumption of existing leading edge models such as ChatGPT, Stable Diffusion, etc... These are very impressive models, but come at great cost. If these are just a starting point in scaling up our models we are going to have a large problem (I mean, we already do but that is besides the point).
      - gambiting 4y ago
        I mean, 30 years ago you needed a supercomputer and to burn MWh of power to render anything remotely realistic, and now a phone can do it, using a fraction of a kWh. I'm not sure why we don't expect a similar leap here again?
      - peaslock 4y ago
        Geoffrey Hinton has recently been talking about how analog and "imperfect" computing with specialized hardware/circuitry may yield much cheaper neural nets, that could easily be as large as human brains, but would only cost a few dollars and would be extremely cheap to run. Not a new idea, but it is a fairly promising outlook, I think. https://www.zdnet.com/article/we-will-see-a-completely-new-type-of-computer-says-ai-pioneer-geoff-hinton-mortal-computation/ https://www.zdnet.com/article/we-will-see-a-completely-new-t...
      - karmakaze 4y ago
        I'm not so sure how inefficient this is. Compare this to how humans have to eat and learn one by one. We don't have a way to take a trained model and run it on other instances. Instead the person who knows has to teach it to other individuals. Writing or media can help but is very slow and compared to loading a model.
- damascus 4y ago
  > they become … still stupid, but it’s more difficult to tell. I feel like this describes my entire formal education path.
jqpabc123 4y ago
In other words, "probabilistic associations” are a poor substitute for logic and reason. They are not inherently reliable nor consistent. Bottom line: Real, reliable, "full self driving" automobiles are unlikely to be achieved using current approaches. About the best that can be expected is a marketing gimmick that is sure to cause lots of accidents.
- eternityforest 4y ago
  I don't see why self driving cars need true reasoning. If you have high resolution sensors, you can use deterministic algorithms to avoid hitting anything or being hit by anything. Especially if you start putting some kind of easily detected nonlinear reflector sticker on all cars and backpacks. You might get a few false trips if a trash bag blows in the road and the rear sensor says nothing will hit you if you stop, but who cares? If we really prioritize safety, we'll just accept that, and find other ways to save time. A self driving car would essentially give a person extra hours in the day, and allow people with disabilities to go outside of major cities unassisted.
  - rasterdog 4y ago
    thanks. let's refashion our cityscape and our clothing to accommodate self driving cars. don't care about pedestrians. </sarcasm>
    - valine 4y ago
      Not hitting pedestrians is already solved. Waymo couldn’t do driverless taxis rides if it wasn’t. I also have tesla’s fsd beta on my car, and it does a marvelous job not hitting people. If anything it’s way too cautious around pedestrians.
      - jqpabc123 4y ago
        So if you're traveling at interstate speed and someone steps out too close to the roadway is it going to swerve into the next lane or slam on the brakes to avoid them? I recently had a situation where a large truck off to the right in front of me had a blowout and a large chunk of tire flew over into my lane. There were other cars traveling fast on both sides and behind me. What to do? I chose to maintain my speed and lane and just hit the tire. It caused some damage to the front of my car but in my split second judgment, this was the least bad option. What would your AI do?
curiousgal 4y ago
I mean one could argue that most humans are stupid so yeah the bar might be lower than we think.
- jqpabc123 4y ago
  The point of the article is that even stupid humans can do lots of tricks that AI can't.
  - alpaca128 4y ago
    Sure, and even GPT-2 can do tricks that Siri can't. It's not a binary thing. It's still going to take quite some time, but imho AI is slowly getting into a range where it starts being actually useful and it doesn't look like improvements are slowing down.
anankaie 4y ago
It seems to me like the Chinese Room Argument does not address the notion that in carrying out the algorithm, a possibility that the original author is instantiating a computational mind whose contents are inaccessible to the substrate hosting the mind. In other words, this argument falls because it does not resolve the hard problem of consciousness. The pan-psychism argument is an interesting parallel to the Free Will Theorem. I am a bit wary of assuming of which side of the possibilities the outcome must fall.
abetusk 4y ago
I've only skimmed but it seems like the general idea is sound, that neural networks are good at pattern matching and that shallow pattern matching is only one aspect of "deep thought", logical reasoning being the other one. I'm confused though, they give examples of Go and Chess but, as far as I understand, this is exactly what the author is saying doesn't exist. AlphaGo, Deep Blue, etc, all use neural networks to weight how to traverse the minimax tree [0]. The minimax algorithm/tree is precisely the "logical reasoning". The neural network is "just" prioritizing the path to take. Anyway, like I said, the reasoning is sound. I just read a paper specifically about this breakdown, called "On the Paradox of Learning to Reason from Data" by Zhang, et all [1] where they construct a synthetic data set based on some simple logical rules and then try to train a neural network to recapture the underlying logic (spoiler alter, it fails). From their paper: """ Our study provides an explanation for this paradox: instead of learning to emulate the correct reasoning function, BERT has, in fact, learned statistical features that inherently exist in logical reasoning problems. We also show that it is infeasible to jointly remove statistical features from data, illustrating the difficulty of learning to reason in general. """ [0] https://en.wikipedia.org/wiki/Minimax https://en.wikipedia.org/wiki/Minimax [1] https://arxiv.org/abs/2205.11502 https://arxiv.org/abs/2205.11502
- abetusk 4y ago
  Nope, I take it back. The article is not coming from a place of (in my opinion) reason. This is, in fancy language and snowed under by enumeration of historical research, math and science, an argument that "computers can never have a soul". From the article: """ No matter how sophisticated the computation is ... there remains an unbridgeable gap (a “humanity gap”) between the ... ability of machine and the general problem solving ability of man. """ This is just standard dualist nonsense under the guise of logical objectivity. Note that there is a deeper issue about the shortcomings of statistical pattern matching to do logic, which is what the paper I reference above is trying to get at, but the parent article's fundamental assumption/conclusion is that computers will never be able to reason.
  - c43verge 4y ago
    Thanks a lot for summarizing it. I was curious to see whether it had a new idea, apparently not.
dsign 4y ago
It's an article on a psychology journal, those folk tend to use different ontologies than computer scientists. That said, it's a very substantial review and I don't necessarily disagree with it. No, causal reasoning won't fix it. But it is an ingredient more. I can't tell which ones are all the others, but the kind of thing that LLM do is already very close to the mindless repetition I did at kindergarten to get positive feedback. Scientists have also noted that animal intelligence, and by extension ours, tends to raise as a response to a challenging environment. Then there is the illusion of self-consciousness, the clearing-house of environmental and internal information that engenders our thoughts. It's like we have a stack of papers and we are searching a way to bind them together to make a book. I think we will eventually find a way and that will be AGI. What we will get won't be like us, but will pass all the tests. Now, is it just me, or is creating AGI stupidly irresponsibly?
- RosanaAnaDana 4y ago
  > or is creating AGI stupidly irresponsibly Stupidly irresponsible and yet inevitable. It's just a question of if it's a democratic accessible model of AGI or a nation state that wields it like the atomic bomb shortly after WWII. That might be our best corollary for an utterly game changing technology with profound leverage.
  - astrange 4y ago
    Nothing new about creating general intelligence. WWI and II were fought using general intelligences called soldiers which were mass-produced by defense technologies called mothers. (The difference between this and AGI is that you're imagining AGI has superpowers, but since it's your imagination just don't do that and now it's safe. Well, as safe as an army is, so not very.)
- gitfan86 4y ago
  The minute we created agriculture this day was inevitable. It is stupid that we created agriculture? That is a matter of opinion,but I would say no.
  - 4y ago
allemagne 4y ago
>“There are times when it seems autonomy is around the corner and the vehicle can go for a day without a human driver intervening … other days reality sets in because the edge cases are endless …” Self-driving cars (and AGI in general) might not be as close as we thought, but edge cases trip up humans too.
- ozim 4y ago
  I also don't expect AGI to appear. Current state of AI is still interesting - we can leave edge cases to people and let computation deal with everything in its function range. I might still have to drive through the city because it is too much to deal with - but I would be glad if we have highways restricted for autonomous cars, where I could travel between cities reading a book like I would be in a train but with all convenience of traveling on my time schedule instead of adjusting my schedule with train schedule.
- CrimsonRain 4y ago
  difference is a human can usually understand they are dealing with an edge case and can try to work-around it or solve it. "AI" just outputs garbage.
Arjuna144 4y ago
I can highly recommend nobel laureate Sir Roger Penroses' take on AI. He argues that AI does in no way 'undestand' anything. He goes even further and uses Goedels Theorem to show that 'understanding' itself is non-computeable. I am not sure about this, but after playing with chatGPT I can clearly see what he means by lack of 'understanding'. There is absolutely ZERO real understanding.... only correlations and probabilities of words....
- jcims 4y ago
  User idiotsecant posted this excerpt from ChatGPT a couple of weeks ago. It kind of blows my mind (the last paragraph). It feels a bit beyond just basic probabilities of words. https://news.ycombinator.com/item?id=33833178 https://news.ycombinator.com/item?id=33833178 We're just at the beginning.
  - bratwurst3000 4y ago
    My opinion is as long the dont solve hard problems they are just probability . For example one of those top ten mathematical problems. Humans solved a few of those, lets challenge AI
    - gnomewrecker 4y ago
      By this metric the vast majority of humans are “just probability” too. I think it’s not a very good metric.
      - bratwurst3000 4y ago
        Hmm is see it this way. Humans are a network of information sharing that could solve this problems. Computers are the same and should also Solve hard problems. And yes, sadly, many humans are just probalility in my eyes.
  - notahacker 4y ago
    Didn't blow mine, even though it involved a howitzer! It's obviously operating on a higher level than "what was the previous word" but doesn't exactly feel like high level reasoning: when prompted for similarities it elides "inflated" with "blew up" to seed a paragraph (rather than to crack a corny joke, which is the only context I can imagine a human in possession of knowledge of what a howitzer and a beach ball doing likewise!), adds some sentences consistent with sentences in its corpus with "blowing up" and "loading" as the respective verbs and appends a sentence which states that it's tenuous, because howitzer and beach balls and inflating and loading verbs aren't often found in close proximity in its corpus (and caveating stuff clearly was well rewarded in ChatGPT's training; it's almost comically quick to do so in every single non-trivial answer) Impressive that it does so in coherent English, but I'm not convinced it's in any danger of understanding firearms or fun beyond realising sentence forms those words would be a good fit and poor fit for. I posted an excerpt from ChatGPT a couple of days ago where it stated - in an identical format to its previous two correct answers - that 355 was a prime number because it wasn't divisible 5, and that's the sort of problem that Turing machines are known to be able to completely and accurately model
29athrowaway 4y ago
"AI"... which AI? each AI solution is different.
kmeisthax 4y ago
>In other words, unless we are content to “bite the bullet” of panpsychism, then no machine, however complex, can ever realize phenomenal consciousness purely in virtue of the execution of a particular computer program. Ok, but why are biological neurons exempted from this argument? I mean, aside from the obvious conflict of interest that the person who wrote this article is made out of them, and they would much rather not be ontologically dissolved into a soulless zombie. I'm not sure you actually have to bite the panpsychism bullet either (or I'm misunderstanding what that word means). Adding a Putnam mapping to an FSA or giving you a book that maps Chinese characters to other Chinese characters doesn't necessarily make the FSA conscious or you a literate Chinese reader. It just makes the sum total of those things conscious. Likewise, each individual neuron in your head does not need to be totally conscious. Consciousness is an emergent property of your whole body, not just something attributable to parts of it.
- Arjuna144 4y ago
  > Consciousness is an emergent property of your whole body, not just something attributable to parts of it. What an arrogant statement to make! All the wisdom traditions of all the ages and of all the places have always taught that consciousness is the Epiphenomenon. The first cause that has no cause. This is even a scientific statement, because there is an experiment that can be reproduced (and has been reproduced since thousands of years) If you meditate very deeply for hours upon hours (20.000 to 30.000 hours, is a rough estimate given by the tradition of kriya yoga) you will make the experience yourself. I highly recommend to look at ORCH-OR (a theory on consciousness by nobel laureat Sir Roger Penrose and Prof. Stuard Hameroff. That substructures (microtubeli) inside biological neurons are taping into the state of the universe by going into superposition and collapsing... It is too much to further elaborate but worth checking out! Better even, increase you meditations ;))
  - cogwheel 4y ago
    Appeals to tradition and authority are not arguments. Look in the mirror for arrogance.
  - aatd86 4y ago
bryan0 4y ago
This article made more sense pre-GPT. Now I’m not so sure. I think if a transformer based architecture was able to live update its weights and was trained on the proper abstraction of data, then it could possibly pass the Turing test. Edit: actually I take it back. This article would not have made any sense pre-GPT either.
- HarHarVeryFunny 4y ago
  The Turing test is a meaningless benchmark. So what if you can fool people? It's like saying if a magician can fool people then they really have supernatural powers. If you believe the magician really has powers to make stuff materialize out of thin air, then you'd be disappointed they couldn't just materialize you a million bucks, and if you believe some chatbot that passed the Turing test really is intelligent, then you'd be disappointed it couldn't help you make a million bucks in the stock market, or do anything that actually required real intelligence vs the faked appearance of intelligence. Anyway, there's not even anything today that would come remotely close to passing the Turing test. I guess ChatGPT might fool some people into thinking it's an idiot who was locked in a box with a printout of WikiPedia last year and has no access to the outside world, but not that it's a normal person with access to, and able to interact with, the outside world.
  - bryan0 4y ago
    You say: “The Turing test is a meaningless benchmark” and then in the next sentence you say: “there’s not anything today that would come remotely close to passing the Turing test” So which is it? I think you’re confused because there’s a difference between fooling a random person and passing a Turing test as described by Kurzweil [0] The magician analogy is kind of like James Randi’s Paranormal test [1]. Again a proper test should determine whether the magic is real or a trick. [0] https://www.kurzweilai.net/a-wager-on-the-turing-test-the-rules https://www.kurzweilai.net/a-wager-on-the-turing-test-the-ru... [1] https://en.m.wikipedia.org/wiki/One_Million_Dollar_Paranormal_Challenge https://en.m.wikipedia.org/wiki/One_Million_Dollar_Paranorma...
    - 4y ago
nootropicat 4y ago
All those arguments reduce to "humans have a soul, computers don't, checkmate atheists". Ironically, soon AI will be able to generate articles arguing that AI can't ever think, automating yet another job. Any model capable of executing a turing machine (given infinite resources, so in practice bounded of course) can emulate another program, even if inefficiently. GPT empirically can execute a simple vm. At best the 'wrong model' argument would have to point out that specific design collapses once the program to execute becomes too big - no matter how much compute/memory is available, but then, it's only a technical claim about one specific design. This specific article has another problem - it handwaves the definition of consciousness. It has to do it, because any attempt to actually define necessarily either defeats the whole argument (it's some property of a computable system), or reduces it to a magical claim about an external soul.
esjeon 4y ago
We simply can't mechanically derive the underlying model strictly from facts. No amount of computation that transforms facts into other facts can reach the deeper layer of the truth. There's simply no logical nor mathematical connection b/w the phenomenon on the surface and models that can describe it concisely. The modern AI models can successfully fit to various phenomenon, including even some chaotic ones, but they never can describe the phenomenon in truthful and concise ways. The modern AI strategy will need billions of parameters to unreliably predict a chaotic system that can be cleanly simulated w/ thousands of variables. Reducing the complexity from billions to thousands is a huge leap that can't be derived logically nor mathematically only from the observed facts.
friend_and_foe 4y ago
I'm really enjoying this discussion that's being had about these things, I think that while these developments will be able to do things, in the end we won't create AGI, but it will help us better understand what consciousness or cognition actually are. I personally believe, for reasons I haven't fully fleshed out enough to clearly articulate, that an intelligence cannot be created by emulation, that the computation or process or whatever it is has to occur on the "bare metal" of the universe. This is alluded to in this article, that a machine cannot experience raw sensation. I also think it is possible for some reason that the process has to emerge naturally the way we did, although whether "emerge naturally" includes being created by something that emerged naturally I am not sure of, and at that point it becomes a meaningless distinction.
kbrkbr 4y ago
> In the following, I briefly review three a priori arguments that purport to demonstrate that “computation” alone can never realize human-like understanding, and, a fortiori, no computational AI system will ever fully “grasp” human meaning. So on the one side we have science and engineering, backed by lots and lots of working applications, saying „let’s try and find out“, „let’s see how far we come“. And on the other side, we have philosophy, with a priori arguments which say this and that must or must not be so. Clearly I’m on the side of a priori arguments, they have a history of working well. Remember when Kant proved that empirical space must be euclidean [1]? Or Hegel that there can only be seven planets [2]? /s [1] http://jur.byu.edu/?p=9200 http://jur.byu.edu/?p=9200 [2] https://scienceblogs.com/stoat/2009/01/17/hegel-does-maths https://scienceblogs.com/stoat/2009/01/17/hegel-does-maths
walnutclosefarm 4y ago
Trying to prove a priori that AI can neither think nor understand is an endlessly alluring fools errand. But in the end, the arguments all seem to me to assume their conclusion. AI is entirely computational, and human understanding clearly requires understanding that can't be computed, therefore AI can't achieve human-like understanding or thought. But none of the efforts to describe what happens in human cognition that is not computable are remotely convincing. In this particular article, after reciting a litani of such unconvincing arguments, Bishop ends with nothing more profound than "current AI doesn't demonstrate understanding, which is because understanding is non-computable (why, not answered), hence no AI can understand. Just show me what the magic that is can't be computed that is going in in the wet lab of our brain, but nowhere else is, and I might by their conclusion.
- jvanderbot 4y ago
  I thoroughly believe we have overestimated human intelligence to the point we are perpetually moving the goalpost for GAI. Read this: https://3cents.substack.com/p/culture-is-smarter-than-you-are https://3cents.substack.com/p/culture-is-smarter-than-you-ar... We have memories, processes, and learned responses and aversions, that's all stuff existing AI can handle, if we fill just a bit more that doesn't exist in training data yet.
b1c1jones 4y ago
great article, however it focuses on the limitations of using Euclidean space and neglects quaternion application to neural networks.
karmakaze 4y ago
These parts raise issues for me: > the underlying reason for such mistakes is that “... all the impressive achievements of deep learning amount to just curve fitting.” The key, as Pearl suggests, is to replace “reasoning by association” with “causal reasoning” —the ability to infer causes from observed phenomena. This seems hypocritical in that for tens of thousands of years and for a large part of the population is still what humans do. Only a subset generally apply true logical reasoning. It's also funny that we're now saying logical circuits can't do logic. > “we need to stop building computer systems that merely get better and better at detecting statistical patterns [...]—and start building computer systems that from the moment of their assembly innately grasp three basic concepts: time, space, and causality. This seems like a mistake assuming something we ourselves haven't determined are fundamental concepts. I would prefer if the data produced the concepts and potential proofs or disproofs.

104 points by trenchgun 4y ago

Could still be a problem of scale. A single neuron is quite stupid. But put enough of them together and they become … still stupid, but it’s more difficult to tell.

nightski 4y ago
AI will consume half the electricity on earth and it will still be pretty dumb at this point lol. We need to find a better solution than scale up.
- gambiting 4y ago
  Is that based on anything, or just a gut feeling?
  - nightski 4y ago
    It's based on the power consumption of existing leading edge models such as ChatGPT, Stable Diffusion, etc... These are very impressive models, but come at great cost. If these are just a starting point in scaling up our models we are going to have a large problem (I mean, we already do but that is besides the point).
    - gambiting 4y ago
      I mean, 30 years ago you needed a supercomputer and to burn MWh of power to render anything remotely realistic, and now a phone can do it, using a fraction of a kWh. I'm not sure why we don't expect a similar leap here again?
    - peaslock 4y ago
      Geoffrey Hinton has recently been talking about how analog and "imperfect" computing with specialized hardware/circuitry may yield much cheaper neural nets, that could easily be as large as human brains, but would only cost a few dollars and would be extremely cheap to run. Not a new idea, but it is a fairly promising outlook, I think. https://www.zdnet.com/article/we-will-see-a-completely-new-type-of-computer-says-ai-pioneer-geoff-hinton-mortal-computation/ https://www.zdnet.com/article/we-will-see-a-completely-new-t...
    - karmakaze 4y ago
      I'm not so sure how inefficient this is. Compare this to how humans have to eat and learn one by one. We don't have a way to take a trained model and run it on other instances. Instead the person who knows has to teach it to other individuals. Writing or media can help but is very slow and compared to loading a model.
damascus 4y ago
> they become … still stupid, but it’s more difficult to tell. I feel like this describes my entire formal education path.

jqpabc123 4y ago

In other words, "probabilistic associations” are a poor substitute for logic and reason. They are not inherently reliable nor consistent. Bottom line: Real, reliable, "full self driving" automobiles are unlikely to be achieved using current approaches. About the best that can be expected is a marketing gimmick that is sure to cause lots of accidents.

eternityforest 4y ago
I don't see why self driving cars need true reasoning. If you have high resolution sensors, you can use deterministic algorithms to avoid hitting anything or being hit by anything. Especially if you start putting some kind of easily detected nonlinear reflector sticker on all cars and backpacks. You might get a few false trips if a trash bag blows in the road and the rear sensor says nothing will hit you if you stop, but who cares? If we really prioritize safety, we'll just accept that, and find other ways to save time. A self driving car would essentially give a person extra hours in the day, and allow people with disabilities to go outside of major cities unassisted.
- rasterdog 4y ago
  thanks. let's refashion our cityscape and our clothing to accommodate self driving cars. don't care about pedestrians. </sarcasm>
  - valine 4y ago
    Not hitting pedestrians is already solved. Waymo couldn’t do driverless taxis rides if it wasn’t. I also have tesla’s fsd beta on my car, and it does a marvelous job not hitting people. If anything it’s way too cautious around pedestrians.
    - jqpabc123 4y ago
      So if you're traveling at interstate speed and someone steps out too close to the roadway is it going to swerve into the next lane or slam on the brakes to avoid them? I recently had a situation where a large truck off to the right in front of me had a blowout and a large chunk of tire flew over into my lane. There were other cars traveling fast on both sides and behind me. What to do? I chose to maintain my speed and lane and just hit the tire. It caused some damage to the front of my car but in my split second judgment, this was the least bad option. What would your AI do?

curiousgal 4y ago

I mean one could argue that most humans are stupid so yeah the bar might be lower than we think.

jqpabc123 4y ago
The point of the article is that even stupid humans can do lots of tricks that AI can't.
- alpaca128 4y ago
  Sure, and even GPT-2 can do tricks that Siri can't. It's not a binary thing. It's still going to take quite some time, but imho AI is slowly getting into a range where it starts being actually useful and it doesn't look like improvements are slowing down.

anankaie 4y ago

It seems to me like the Chinese Room Argument does not address the notion that in carrying out the algorithm, a possibility that the original author is instantiating a computational mind whose contents are inaccessible to the substrate hosting the mind. In other words, this argument falls because it does not resolve the hard problem of consciousness. The pan-psychism argument is an interesting parallel to the Free Will Theorem. I am a bit wary of assuming of which side of the possibilities the outcome must fall.

abetusk 4y ago

I've only skimmed but it seems like the general idea is sound, that neural networks are good at pattern matching and that shallow pattern matching is only one aspect of "deep thought", logical reasoning being the other one. I'm confused though, they give examples of Go and Chess but, as far as I understand, this is exactly what the author is saying doesn't exist. AlphaGo, Deep Blue, etc, all use neural networks to weight how to traverse the minimax tree [0]. The minimax algorithm/tree is precisely the "logical reasoning". The neural network is "just" prioritizing the path to take. Anyway, like I said, the reasoning is sound. I just read a paper specifically about this breakdown, called "On the Paradox of Learning to Reason from Data" by Zhang, et all [1] where they construct a synthetic data set based on some simple logical rules and then try to train a neural network to recapture the underlying logic (spoiler alter, it fails). From their paper: """ Our study provides an explanation for this paradox: instead of learning to emulate the correct reasoning function, BERT has, in fact, learned statistical features that inherently exist in logical reasoning problems. We also show that it is infeasible to jointly remove statistical features from data, illustrating the difficulty of learning to reason in general. """ [0] https://en.wikipedia.org/wiki/Minimax https://en.wikipedia.org/wiki/Minimax [1] https://arxiv.org/abs/2205.11502 https://arxiv.org/abs/2205.11502

abetusk 4y ago
Nope, I take it back. The article is not coming from a place of (in my opinion) reason. This is, in fancy language and snowed under by enumeration of historical research, math and science, an argument that "computers can never have a soul". From the article: """ No matter how sophisticated the computation is ... there remains an unbridgeable gap (a “humanity gap”) between the ... ability of machine and the general problem solving ability of man. """ This is just standard dualist nonsense under the guise of logical objectivity. Note that there is a deeper issue about the shortcomings of statistical pattern matching to do logic, which is what the paper I reference above is trying to get at, but the parent article's fundamental assumption/conclusion is that computers will never be able to reason.
- c43verge 4y ago
  Thanks a lot for summarizing it. I was curious to see whether it had a new idea, apparently not.

dsign 4y ago

It's an article on a psychology journal, those folk tend to use different ontologies than computer scientists. That said, it's a very substantial review and I don't necessarily disagree with it. No, causal reasoning won't fix it. But it is an ingredient more. I can't tell which ones are all the others, but the kind of thing that LLM do is already very close to the mindless repetition I did at kindergarten to get positive feedback. Scientists have also noted that animal intelligence, and by extension ours, tends to raise as a response to a challenging environment. Then there is the illusion of self-consciousness, the clearing-house of environmental and internal information that engenders our thoughts. It's like we have a stack of papers and we are searching a way to bind them together to make a book. I think we will eventually find a way and that will be AGI. What we will get won't be like us, but will pass all the tests. Now, is it just me, or is creating AGI stupidly irresponsibly?

RosanaAnaDana 4y ago
> or is creating AGI stupidly irresponsibly Stupidly irresponsible and yet inevitable. It's just a question of if it's a democratic accessible model of AGI or a nation state that wields it like the atomic bomb shortly after WWII. That might be our best corollary for an utterly game changing technology with profound leverage.
- astrange 4y ago
  Nothing new about creating general intelligence. WWI and II were fought using general intelligences called soldiers which were mass-produced by defense technologies called mothers. (The difference between this and AGI is that you're imagining AGI has superpowers, but since it's your imagination just don't do that and now it's safe. Well, as safe as an army is, so not very.)
gitfan86 4y ago
The minute we created agriculture this day was inevitable. It is stupid that we created agriculture? That is a matter of opinion,but I would say no.
- 4y ago

allemagne 4y ago

>“There are times when it seems autonomy is around the corner and the vehicle can go for a day without a human driver intervening … other days reality sets in because the edge cases are endless …” Self-driving cars (and AGI in general) might not be as close as we thought, but edge cases trip up humans too.

ozim 4y ago
I also don't expect AGI to appear. Current state of AI is still interesting - we can leave edge cases to people and let computation deal with everything in its function range. I might still have to drive through the city because it is too much to deal with - but I would be glad if we have highways restricted for autonomous cars, where I could travel between cities reading a book like I would be in a train but with all convenience of traveling on my time schedule instead of adjusting my schedule with train schedule.
CrimsonRain 4y ago
difference is a human can usually understand they are dealing with an edge case and can try to work-around it or solve it. "AI" just outputs garbage.

Arjuna144 4y ago

I can highly recommend nobel laureate Sir Roger Penroses' take on AI. He argues that AI does in no way 'undestand' anything. He goes even further and uses Goedels Theorem to show that 'understanding' itself is non-computeable. I am not sure about this, but after playing with chatGPT I can clearly see what he means by lack of 'understanding'. There is absolutely ZERO real understanding.... only correlations and probabilities of words....

jcims 4y ago
User idiotsecant posted this excerpt from ChatGPT a couple of weeks ago. It kind of blows my mind (the last paragraph). It feels a bit beyond just basic probabilities of words. https://news.ycombinator.com/item?id=33833178 https://news.ycombinator.com/item?id=33833178 We're just at the beginning.
- bratwurst3000 4y ago
  My opinion is as long the dont solve hard problems they are just probability . For example one of those top ten mathematical problems. Humans solved a few of those, lets challenge AI
  - gnomewrecker 4y ago
    By this metric the vast majority of humans are “just probability” too. I think it’s not a very good metric.
    - bratwurst3000 4y ago
      Hmm is see it this way. Humans are a network of information sharing that could solve this problems. Computers are the same and should also Solve hard problems. And yes, sadly, many humans are just probalility in my eyes.
- notahacker 4y ago
  Didn't blow mine, even though it involved a howitzer! It's obviously operating on a higher level than "what was the previous word" but doesn't exactly feel like high level reasoning: when prompted for similarities it elides "inflated" with "blew up" to seed a paragraph (rather than to crack a corny joke, which is the only context I can imagine a human in possession of knowledge of what a howitzer and a beach ball doing likewise!), adds some sentences consistent with sentences in its corpus with "blowing up" and "loading" as the respective verbs and appends a sentence which states that it's tenuous, because howitzer and beach balls and inflating and loading verbs aren't often found in close proximity in its corpus (and caveating stuff clearly was well rewarded in ChatGPT's training; it's almost comically quick to do so in every single non-trivial answer) Impressive that it does so in coherent English, but I'm not convinced it's in any danger of understanding firearms or fun beyond realising sentence forms those words would be a good fit and poor fit for. I posted an excerpt from ChatGPT a couple of days ago where it stated - in an identical format to its previous two correct answers - that 355 was a prime number because it wasn't divisible 5, and that's the sort of problem that Turing machines are known to be able to completely and accurately model

29athrowaway 4y ago

"AI"... which AI? each AI solution is different.

kmeisthax 4y ago

>In other words, unless we are content to “bite the bullet” of panpsychism, then no machine, however complex, can ever realize phenomenal consciousness purely in virtue of the execution of a particular computer program. Ok, but why are biological neurons exempted from this argument? I mean, aside from the obvious conflict of interest that the person who wrote this article is made out of them, and they would much rather not be ontologically dissolved into a soulless zombie. I'm not sure you actually have to bite the panpsychism bullet either (or I'm misunderstanding what that word means). Adding a Putnam mapping to an FSA or giving you a book that maps Chinese characters to other Chinese characters doesn't necessarily make the FSA conscious or you a literate Chinese reader. It just makes the sum total of those things conscious. Likewise, each individual neuron in your head does not need to be totally conscious. Consciousness is an emergent property of your whole body, not just something attributable to parts of it.

Arjuna144 4y ago
> Consciousness is an emergent property of your whole body, not just something attributable to parts of it. What an arrogant statement to make! All the wisdom traditions of all the ages and of all the places have always taught that consciousness is the Epiphenomenon. The first cause that has no cause. This is even a scientific statement, because there is an experiment that can be reproduced (and has been reproduced since thousands of years) If you meditate very deeply for hours upon hours (20.000 to 30.000 hours, is a rough estimate given by the tradition of kriya yoga) you will make the experience yourself. I highly recommend to look at ORCH-OR (a theory on consciousness by nobel laureat Sir Roger Penrose and Prof. Stuard Hameroff. That substructures (microtubeli) inside biological neurons are taping into the state of the universe by going into superposition and collapsing... It is too much to further elaborate but worth checking out! Better even, increase you meditations ;))
- cogwheel 4y ago
  Appeals to tradition and authority are not arguments. Look in the mirror for arrogance.
- aatd86 4y ago

bryan0 4y ago

This article made more sense pre-GPT. Now I’m not so sure. I think if a transformer based architecture was able to live update its weights and was trained on the proper abstraction of data, then it could possibly pass the Turing test. Edit: actually I take it back. This article would not have made any sense pre-GPT either.

HarHarVeryFunny 4y ago
The Turing test is a meaningless benchmark. So what if you can fool people? It's like saying if a magician can fool people then they really have supernatural powers. If you believe the magician really has powers to make stuff materialize out of thin air, then you'd be disappointed they couldn't just materialize you a million bucks, and if you believe some chatbot that passed the Turing test really is intelligent, then you'd be disappointed it couldn't help you make a million bucks in the stock market, or do anything that actually required real intelligence vs the faked appearance of intelligence. Anyway, there's not even anything today that would come remotely close to passing the Turing test. I guess ChatGPT might fool some people into thinking it's an idiot who was locked in a box with a printout of WikiPedia last year and has no access to the outside world, but not that it's a normal person with access to, and able to interact with, the outside world.
- bryan0 4y ago
  You say: “The Turing test is a meaningless benchmark” and then in the next sentence you say: “there’s not anything today that would come remotely close to passing the Turing test” So which is it? I think you’re confused because there’s a difference between fooling a random person and passing a Turing test as described by Kurzweil [0] The magician analogy is kind of like James Randi’s Paranormal test [1]. Again a proper test should determine whether the magic is real or a trick. [0] https://www.kurzweilai.net/a-wager-on-the-turing-test-the-rules https://www.kurzweilai.net/a-wager-on-the-turing-test-the-ru... [1] https://en.m.wikipedia.org/wiki/One_Million_Dollar_Paranormal_Challenge https://en.m.wikipedia.org/wiki/One_Million_Dollar_Paranorma...
  - 4y ago

nootropicat 4y ago

All those arguments reduce to "humans have a soul, computers don't, checkmate atheists". Ironically, soon AI will be able to generate articles arguing that AI can't ever think, automating yet another job. Any model capable of executing a turing machine (given infinite resources, so in practice bounded of course) can emulate another program, even if inefficiently. GPT empirically can execute a simple vm. At best the 'wrong model' argument would have to point out that specific design collapses once the program to execute becomes too big - no matter how much compute/memory is available, but then, it's only a technical claim about one specific design. This specific article has another problem - it handwaves the definition of consciousness. It has to do it, because any attempt to actually define necessarily either defeats the whole argument (it's some property of a computable system), or reduces it to a magical claim about an external soul.

esjeon 4y ago

We simply can't mechanically derive the underlying model strictly from facts. No amount of computation that transforms facts into other facts can reach the deeper layer of the truth. There's simply no logical nor mathematical connection b/w the phenomenon on the surface and models that can describe it concisely. The modern AI models can successfully fit to various phenomenon, including even some chaotic ones, but they never can describe the phenomenon in truthful and concise ways. The modern AI strategy will need billions of parameters to unreliably predict a chaotic system that can be cleanly simulated w/ thousands of variables. Reducing the complexity from billions to thousands is a huge leap that can't be derived logically nor mathematically only from the observed facts.

friend_and_foe 4y ago

I'm really enjoying this discussion that's being had about these things, I think that while these developments will be able to do things, in the end we won't create AGI, but it will help us better understand what consciousness or cognition actually are. I personally believe, for reasons I haven't fully fleshed out enough to clearly articulate, that an intelligence cannot be created by emulation, that the computation or process or whatever it is has to occur on the "bare metal" of the universe. This is alluded to in this article, that a machine cannot experience raw sensation. I also think it is possible for some reason that the process has to emerge naturally the way we did, although whether "emerge naturally" includes being created by something that emerged naturally I am not sure of, and at that point it becomes a meaningless distinction.

kbrkbr 4y ago

> In the following, I briefly review three a priori arguments that purport to demonstrate that “computation” alone can never realize human-like understanding, and, a fortiori, no computational AI system will ever fully “grasp” human meaning. So on the one side we have science and engineering, backed by lots and lots of working applications, saying „let’s try and find out“, „let’s see how far we come“. And on the other side, we have philosophy, with a priori arguments which say this and that must or must not be so. Clearly I’m on the side of a priori arguments, they have a history of working well. Remember when Kant proved that empirical space must be euclidean [1]? Or Hegel that there can only be seven planets [2]? /s [1] http://jur.byu.edu/?p=9200 http://jur.byu.edu/?p=9200 [2] https://scienceblogs.com/stoat/2009/01/17/hegel-does-maths https://scienceblogs.com/stoat/2009/01/17/hegel-does-maths

walnutclosefarm 4y ago

Trying to prove a priori that AI can neither think nor understand is an endlessly alluring fools errand. But in the end, the arguments all seem to me to assume their conclusion. AI is entirely computational, and human understanding clearly requires understanding that can't be computed, therefore AI can't achieve human-like understanding or thought. But none of the efforts to describe what happens in human cognition that is not computable are remotely convincing. In this particular article, after reciting a litani of such unconvincing arguments, Bishop ends with nothing more profound than "current AI doesn't demonstrate understanding, which is because understanding is non-computable (why, not answered), hence no AI can understand. Just show me what the magic that is can't be computed that is going in in the wet lab of our brain, but nowhere else is, and I might by their conclusion.

jvanderbot 4y ago
I thoroughly believe we have overestimated human intelligence to the point we are perpetually moving the goalpost for GAI. Read this: https://3cents.substack.com/p/culture-is-smarter-than-you-are https://3cents.substack.com/p/culture-is-smarter-than-you-ar... We have memories, processes, and learned responses and aversions, that's all stuff existing AI can handle, if we fill just a bit more that doesn't exist in training data yet.

b1c1jones 4y ago

great article, however it focuses on the limitations of using Euclidean space and neglects quaternion application to neural networks.

karmakaze 4y ago

These parts raise issues for me: > the underlying reason for such mistakes is that “... all the impressive achievements of deep learning amount to just curve fitting.” The key, as Pearl suggests, is to replace “reasoning by association” with “causal reasoning” —the ability to infer causes from observed phenomena. This seems hypocritical in that for tens of thousands of years and for a large part of the population is still what humans do. Only a subset generally apply true logical reasoning. It's also funny that we're now saying logical circuits can't do logic. > “we need to stop building computer systems that merely get better and better at detecting statistical patterns [...]—and start building computer systems that from the moment of their assembly innately grasp three basic concepts: time, space, and causality. This seems like a mistake assuming something we ourselves haven't determined are fundamental concepts. I would prefer if the data produced the concepts and potential proofs or disproofs.