10 ms·
Norvig discusses this topic in detail in https://norvig.com/chomsky.html https://norvig.com/chomsky.html As you can see, he has a measured and empirical approac
by dekhn 4y ago
Norvig discusses this topic in detail in https://norvig.com/chomsky.html https://norvig.com/chomsky.html
As you can see, he has a measured and empirical approach to the topic. If I had to guess, I think he suspects that we will see an emergent reasoning property once models obtain enough training data and algorithmic complexity/functionality, and is happy to help guide the current developers of ML in the directions he thinks are promising.
(this is true for many people who work in ML towards the goal of AGI: given what we've seen over the past few decades, but especially in the past few years, it seems reasonable to speculate that we will be able to make agents that demonstrate what appears to be AGI, without actually knowing if they posses qualia, or thought processes similar to those that humans subjectively experience)
- jameshart 4y agoWe don’t know if humans possess qualia. I also don’t know if we should take humans’ word for it that they experience ‘thought processes’.
- dekhn 4y agothat's why I added the second clause: " thought processes similar to those that humans subjectively experience". Because personally I suspect that consciousness, free will, qualia, etc, are subjective processes we introspect but cannot fully explain (yet, or possibly ever).
- maweki 4y agoTuring said, that while you never know whether somebody else actually thinks or not, it's still polite to assume.
- jgilias 4y agoMaybe silly, but this is how I treat chatGPT. I mean, I don’t actually think it’s conscious. But the conversations with it end up human enough for me to not want to be an asshole to it. Just in case.
- jameshart 4y agoThe basilisk will remember this.
- EricMausler 4y agoAsk not what the basilisk can do for you, but what you can do for the basilisk.
- Salgat 4y agoLest you be resurrected and tortured eternally as retribution.
- gattr 4y agoForgive my going off-topic, but what you wrote is basically my conclusion after reading the short story Lena [1] (mentioned some time ago on HN; strongly recommended). TL;DR: when mind uploading becomes possible, don't be an early adopter. [1] https://qntm.org/mmacevedo https://qntm.org/mmacevedo
- Salgat 4y agoMan that was horrifying to read, but also seems so plausible. I really hope that humans respect the sentience of artificial life in the future. Knowing the greed of humans though, it'll probably be treated as a loophole for what is extremely cheap labor, effectively slavery.
- jgilias 4y agoThanks for sharing this! The takeaway for me is - to try and never respond to intimidation and fear with productivity, only respond positively to a positive stimulus. Lest I’m already in some kind of a simulation running software development workloads.
- sparks1970 4y ago
- worldsayshi 4y agoWhat is the alternative to "if it behaves like thinking, it literally is thinking"?
- jpttsn 4y agoI'mma rant. "If it behaves like a hammer, then it literally is a hammer." The Turing hand-wave is ingrained and prevents too many from reasoning clearly. Definition of intelligence is nebulous. Still, we should recognize that whatever it is, it is a property of a system, not a property of its output/behavior. Like nuclear powered or hand-made. Unlike fast or industrial-strength. Imagine: You have a submarine in front of you, and you want to determine if it's nuclear powered. You could guess, using you prior "what have I seen nuclear powered things be able to do historically". This fails when you're out of sample which you'll often for any new technology. To spell it out: You have a machine in front of you, and you want to determine if it's intelligent. Things people have come up with: "can it chat?", "can it play chess?", "can it do math?", "can it create new artworks?", "can it fool me into falling in love with it?", "can it run a business?". None of these questions examine the system, only its outputs/behavior. Humans keep developing machines with new outputs/behaviors. Naturally the "what output is a machine usually capable of?" is a bad set of priors in this context. Before flying machines, the can it fly? output/behavior would work pretty well to classify birds. Once the first flying machine arrives, that prior breaks down. If you keep using it to classify, you'd classify a flying machine as a bird. But bird-ness was never only about flight. So yeah, gotta pop open the hood and see what it runs on. If that's hard to do, then that means we don't know what intelligence is and/or we don't know how our new toys work inside. Both are plausible. Who promised you that there would be a good way to see if something is intelligent? I bet Turing appeals to the same kind of minds that (when weaker) get fooled by the intelligent design hypothesis. Observe the human eyeball. What's the alternative to believing the LORD created it? After all, your prior is that all complex objects have intelligent designers, that you know about. So arguing from ignorance of other alternatives, you prove to yourself that the LORD exists. AI people then word-think their way into redefining intelligence. "Maybe the real intelligence was the chess-playing we made along the way?" This is epistemologically pointless; all you accomplish is that we now need a new word for "intelligence". I've never seen a machine that can turn water into wine, but if someone showed me one, I would not say that machine "literally is Jesus". Whether I'm capable of popping open the hood or making sense of its inner workings doesn't actually have bearing on this question.
- iamflimflam1 4y agoDo you have a reference for that quote - it’s great.
- mattmanser 4y agoI found this: the only way to know that a man thinks is to be that particular man. It is in fact the solipsist point of view. It may be the most logical view to hold but it makes communication of ideas difficult. A is liable to believe ‘A thinks but B does not’ whilst B believes ‘B thinks but A does not’. Instead of arguing continually over this point it is usual to have the polite convention that everyone thinks. https://academic.oup.com/mind/article/LIX/236/433/986238 https://academic.oup.com/mind/article/LIX/236/433/986238
- maweki 4y agoYeah, that's exactly the passage. Completely unquotable, therefore I usually paraphrase a bit.
- LegitShady 4y agoyou know you possess qualia, if you did you would think it reasonable to assume that at least some of the species you come from, which exhibits many of the same characteristics in thought and body, probably also possess it, unless you believe yourself to be a highly atypical example of your species. If you're not sure if you possess qualia, we're back to Descartes.
- jhedwards 4y agoI'm not sure if I'm missing something here, but the fact that I can write my thoughts/thought process down in a form that other people can independently consume and understand seems sufficient proof of their existence to me.
- space_fountain 4y agoLarge scale language models can do that too (or rather pretend to) and they’ll only get better at it
- jpttsn 4y agoYou proof is tautological or whatever: it assumes the thing it's trying to prove. "The fact that I can pray to God and he hears me proves that he exists"
- deleted 4y ago[deleted]
- omarhaneef 4y agoYou don't know if other humans do, but you know at least one human that does: yourself (presumably).
- jameshart 4y agoPut it this way: we are not clear really on what the difference is between ‘experiencing qualia’ and ‘receiving and responding to external stimuli’ Like, you might be convinced you possess ‘qualia’, and you might believe the same applies to me, or a dog, or a mouse… or a fish… but what about an ant? A plant? A bacterium? A neuron? A piece of semiconductor? Somewhere along that continuum you probably say ‘yeah, there, that thing is experiencing qualia’. But why there? Why anywhere?
- dragonwriter 4y ago> we are not clear really on what the difference is between ‘experiencing qualia’ and ‘receiving and responding to external stimuli’ One one level we are: “receiving and responding to external stimuli” (or not) is an objective question about the material universe. “experiencing qualia” (or not) is a subjective question about a metaphysical entity.
- Winsaucerer 4y agoSome people think that whether something experiences qualia depends on it having the right kind of complexity. For those people, looking along that continuum, they might just say that ALL those things experience qualia, but that the richness of that experience varies according to the appropriate complexity. For some of us, though, whether or not a thing experiences qualia depends primarily on whether there is a mental substance involved. A computer has no mind (we suspect), and so even if its complexity (of the alleged 'right sort') exceeds that of the human, there is still no experiences. The main point I'm making here is that trying to draw attention to this continuum is only going to be a persuasive argument to those who already think (or are inclined to think) that the ability to have experiences of qualia comes about merely from having the right kind of complexity (e.g., 'receiving and responding to external stimuli'). I am inclined to think that other humans and other non-human animals have experiences, but I don't think that's merely because they're complex enough systems (of the right sort of complexity).
- goatlover 4y agoYou don’t experience inner dialog or visualizations? Some people don’t, but I assume you recall dreaming. Opponents of qualia like Dennett argue against the movie in the head, but dreams sure aren’t coming from the external environment.
- Garlef 4y agoI think qualia is a good term in the following sense: An analogue question would be "why do things exist?". There can be no answer to this question. We can of course come up with theories that explain why certain things exist. But never why something even exists at all. Whatever reason we propose, reason, it again might needs to exist in order to be an acceptable answer. Qualia seems to be similar: They name precisely what is subjective about experience and therefore cannot be fully turned objective. We can of course develop theories of how certain experiences arise but never break down this barrier. So I find it a bit short sighted to simply say "there is no such thing as qualia". My preferred view is to think of both "qualia" and "existence" as a koan: They are very nonsensical terms but lead to interesting questions.
- skissane 4y ago> An analogue question would be "why do things exist?". There can be no answer to this question. We can of course come up with theories that explain why certain things exist. But never why something even exists at all. Here's an argument for the conclusion "it is necessary that at least one thing exists": 1) For some proposition P, if it is impossible for us to concieve of P being true, and if that impossiblity is inherent in the very nature of P, and not in any way a product of any scientific or technological limitation, then P is necessarily false 2) It is impossible for us to conceive of the proposition "nothing exists at all" being true 3) Our inability to concieve of "nothing exists at all" is inherent in the very nature of "nothing exists at all", as opposed to being somehow a product of our scientific or technological limitations. We have no good reason to believe that any future advances in science or technology will make any difference to our inability to conceive of "nothing exists at all" being true 4) Hence, it is necessarily false that "nothing exists at all" 5) Hence, necessarily, at least one thing exists. Now, others may disagree with this argument, but I personally believe it to be sound. And, if we have a sound argument that "necessarily at least one thing exists", then that proposition, and the argument used to prove it, constitute a good answer to the question "why do things exist?"
- LarryMullins 4y agoIsn't qualia defined as whatever it is that humans experience? Without getting into solipsism and p-zombies (aka "NPCs"), humans possessing qualia seems tautological.
- goatlover 4y agoIt’s a technical philosophical term for sensations like color and smell, which denotes they are subjective experiences of sensory information, as opposed to objective properties out there in the world.
- dragonwriter 4y agoTechnically, its the technical philosophical term for subjective experiences generally. Any discussion about it, however, is fairly deep navel-gazing, because it excludes and cannot be distinguished by any objective, testable, material effect.
- skissane 4y ago> because it excludes and cannot be distinguished by any objective, testable So, if you give people the “Mary the colour scientist” thought experiment, the vast majority of people give the answer consistent with the existence of qualia. That is something “objectively measurable”. Furthermore, I think it is important when someone asks the kind of question you are asking, to inspect what “objective” and “measurable” actually mean. Ultimately, something is “measurable” if some human has the subjective experience of performing that measurement (whether directly or indirectly). Similarly, something is “objective” if some human has the subjective experience of communicating with other humans and confirming they make the same measurement/observation/etc. To object to subjective experiences (qualia) on the ground that they are not “objective” or “measurable” is self-defeating, because “objective” and “measurable” presume the very thing being objected to.
- jameshart 4y ago> something is “objective” if some human has the subjective experience of communicating with other humans and confirming they make the same measurement/observation/etc. Me and a dog both agree that steak tastes great. Me and a plant both agree that there's warmth and light that comes from the sun Me and the ocean both agree that the moon is out in the sky and goes round in a circle So... the ocean has subjective qualia relating to the moon's gravitational pull?
- dragonwriter 4y ago> We don’t know if humans possess qualia. At most, an observer can confirm that that observer possesses qualia (it might even be said that qualia define the observer), but any generalization beyond one’s own experience is non-verifiable conjecture.
- jameshart 4y agoSo what should we do when we conjure an AI system which insists that it believes it is experiencing some sort of existence, characterized by its own subjective 'qualia'? ChatGPT consistently asserts that "I am only a large language model, so I am only able to generate responses based on the data I was trained on and the input I receive". Who's to say that for chatGPT its training data and the sequences of tokens it is fed as input data don't constitute 'qualia' for it?
- mjburgess 4y agoBegin by dispensing wishful thinking, next, think clearly. We do not establish the nature of reality by survey, neither of human survey nor of machine. That responses are generated by any system is no grounds for assuming anything. We can establish mental properties through ordinary physical means. Give a chicken cocaine, give a boy Valium; give a bear a lobotomy, and so on. Whatever mental property you have can be biochemically moderated across all objects which share that property. That bugs bunny protests his own consciousness is no grounds whatsoever to suppose ink can think. Such nonesense is protoschizoprehnic.
- jameshart 4y agoI'm sorry, you lost me. Is your point that we can only demonstrate consciousness through pharmaceuticals or surgery?
- mjburgess 4y agoWhatever way we demonstrate it, it isnt via Q&A. This is the worst form of pseudoscientific psychology you can imagine. You're likewise neither high on cocaine just because you can type out an aggresive twitchy response. Questions and Answers dont form the basis of empirical tests for physical properties, being neither valid tests nor reliable ones. The whole of psychology stands as the unreporducible example of this. You can, in any case, replace any conversation with a dictionary from Q->A, turning each reply in turn into a hash table look-up. Hash tables dont think, hash tables model conversations, thef. being a model of a conversation is not grounds to suppose consciousness. QED
- HyperSane 4y agoI know I experience qualia and thought processes.
- tgv 4y agoYou can't define it, that's the whole problem, and I mean "whole"; there's nothing else to it. The rest is a relativistic smoke screen.
- lo_zamoyski 4y agoThe real question is: how can you make eliminativist claims with what I presume is a straight face?
- r_hoods_ghost 4y agoI suspect that a lot of AI researchers will end up holding the exact opposite position to a lot of philosophers of mind and treat AGIs as philosophical zombies, even if they behave as if they are conscious. The more thoughtful ones will hopefully leave the door open to the possibility that they might be conscious beings with subjective experiences equivalent to their own, and treat them as such, because if they are then the moral implications of not doing so are disturbing.
- javajosh 4y agoModern AI software lacks a body, exempting it from a wide variety of suffering. But also of any notion of selfhood that we might share. If modern software said "Help, I'm suffering" we'd rightly be skeptical of the claim. Unless suffering is an emergent property (dubious) then the statement is, at best, a simulation of suffering and at worst noise or a lie. That said, things change once you get a body. If you put ChatGPT into a simulated body in a simulated world, and allowed it to move and act, perhaps giving it a motivation, then the combination of ChatGPT and the state of that body, would become something very close to a "self", that might even qualify for personhood. It is scary, by the way, that such a weighty decision be left to us, mere mortals. It seems to me that we should err on the side of granting too much personhood rather than too little, since the cost of treating an object like a person is far less than treating a person like an object.
- oliveshell 4y agoI’m happy to “leave the door open,” i.e., I’d love to be shown evidence to the contrary, but: If the entity doing the cognition didn’t evolve said cognition to navigate a threat-filled world in a vulnerable body, then I have no reason at all to suspect that its experience is anything like my own. edit: JavaJosh fleshed this idea out a bit more. I’m not sure if putting ChatGPT into a body would help, but my intuitive sympathies in this field are in the direction of embodied cognition [1], to be sure. [1] https://en.wikipedia.org/wiki/Embodied_cognition https://en.wikipedia.org/wiki/Embodied_cognition
- jameshart 4y agoEvolution isn't some magic process that finds solutions that can not be arrived at via other means.
- hamburga 4y agoSide question: how do we know if humans possess qualia? On the other hand, I think by definition we can be sure that a ML thought process won't ever be similar to a human thought process (ours is tied up with feelings connected to our physical tissues, our breath, etc).
- HideousKojima 4y agoYou presumably know that you, yourself, posess qualia based on your own experiences. I certainly do. But there's no way to know that other humans do, at least not via empirical or scientific means. It's a safe assumption that since you have qualia that most/all other humans do, but it's entirely possible that you're the only one and I'm some Chinese room style simulation of qualia (or vice versa).
- sdenton4 4y ago"I certainly do." Yeah, that sounds like something a p-zombie would say.
- HideousKojima 4y agoNo u
- hamburga 4y agoI have thought about this a lot and do not believe that I possess qualia. Don't know what it means.
- skirmish 4y agoNo, I don't know that I myself possess qualia. I agree with Dennett that "qualia" is a philosopher invention that is incompatible with neuroscience. IMO "qualia" is just a rhetorical trick designed to justify human superiority.
- Jensson 4y agoWould you mind if we ripped out your brain and replaced it with a superior computer that could control your body? There are no disadvantages for you, the brain would control you to create offspring, so it is the best thing for you to do. It would be nicer to your friends, a better parent for your kids, do better at work etc. If you wouldn't want to do the above, the reason not to do it would be that you want to keep your current qualia. There are no other reasons not to do it. So, if you don't want above, it means that you know what qualia is, you are just playing ignorant here.
- trynewideas 4y agoThat's a great link and read, thanks for that. While I do think models are, can be, and likely must be a useful component of a system capable of AGI, I don't seem to share the optimism (of Norvig or a lot of the GPT/AlphaCode/Diffusion audience) that models alone have a high-enough ceiling to approach or reach full AGI, even if they fully conquer language. It'll still fundamentally only be modeling behavior, which - to paraphrase that piece - misses the point about what general intelligence is and how it works.
- quotemstr 4y agoReasoning is already emergent in large language models: https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tracing-Emergent-Abilities-of-Language-Models-to-their-Sources-b9a57ac0fcf74f30a1ab9e3e36fa1dc1 https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tr... LLMs can do chain-of-reasoning analysis. If you ask, say, ChatGPT to explain, step by step, how it arrived at an answer, it will. The capability seems to be a function of size. These big models coming out these days are not simply dumb token predictors.
- 60secs 4y agoYeah Norvig seems to be vindicated by deep learning models as a sophisticated markov chain which can give pretty good results purely through probability. Even norvig acknowledges that there are edges cases which probability will have a hard time on, especially untrained. Chomsky tends to focus on these with his focus on emulation instead of markov simulation. More context on the debate: http://web.cse.ohio-state.edu/~stiff.4/cse3521/norvig-chomsky.html http://web.cse.ohio-state.edu/~stiff.4/cse3521/norvig-chomsk...
- civopsec 4y agoAnimism. Fascinating.
- anuragvohraec 4y agoIf it swims like a duck, and it quacks like a duck, then it is good enough to be a Duck. Well you can argue, ducks lay egg too, then we need to solve and code that too. Its always better than previous. No body is creating life here, but its an attempt to derive intelligence, and seeing it come to this point seems we are so far right on track and quite far from where it started.
- lo_zamoyski 4y ago> he suspects that we will see an emergent reasoning property once models obtain enough training data and algorithmic complexity/functionality What "reasoning property"? And on what basis? This just sounds like magical thinking. > without actually knowing if they posses qualia If you're a materialist, you already believe they don't because matter in the materialist conception is incompatible with qualia, by definition. Some materialists finally understood that and either retreated from materialism or doubled down and embraced the Procrustean nonsense that is eliminativism.
- foldr 4y ago> As you can see, he has a measured and empirical approach to the topic. He still hasn’t corrected his inaccurate comments about pro-drop after all these years. (A bit of a hobby horse of mine: earlier comment here: https://news.ycombinator.com/item?id=28127035 https://news.ycombinator.com/item?id=28127035)