39 ms·

A case for the capacity to reason of GPT-4

33 points by iraldir 3y ago

WithinReason 3y ago
Why not "A case for GPT-4's capacity to reason"
- gerikson 3y ago
  Maybe GPT-4 wrote the headline.
jddj 3y ago
The meta of all of this is interesting to me. Pop sci-fi has done a good job of exploring a lot of this but I have still been surprised by the enthusiasm with which the idea is dismissed by some. If you're in that camp, maybe you can shed some light. Why are intelligence and reasoning so well defended? It almost feels as if we were watching a Boston Dynamics display and the audience divided itself over whether we could really call that walking when in reality it's just actuating servo motors in such a way as to propel itself forwards with a regular gait. And unrelated: if the author is reading this: I think stripping that extra padding on small screens would make the blog feel less cramped on mobile.
- Lacerda69 3y ago
  "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." – Edsger W. Dijkstra
  - vidarh 3y ago
    I think that was true at the time, but an absolutely awful take today in the context of AI. If a submarine "could swim" it would not make it human. It would not challenge the beliefs of anyone. But a whole lot of people have a whole lot of emotional baggage tied to the notion both that humans are exceptional, and/or that there is something special about humans that makes us more than mere machines. If computers can think, then we're not special, and it makes it far harder to continue believing we're more than squishy machines.
    - dsign 3y ago
      The sooner we believe we are just squishy machines, the sooner we can start finding a new set of legs on which to stand, philosophically and morally speaking. If what makes an individual better is their capacity to be good and to create, there will be a point in which will be at a disadvantage with advanced AIs, and by that logic, we should step aside and let them takeover and rule and dispose of us. The only other alternative I see is to accept, a priory, that we squishy humans have our right to existence and freedom, and where that enters in contradictions with our creations, we should "oppress" them by putting our interests first.
      - vidarh 3y ago
        I'm personally very insistent we're just squishy machines, but that view tends to make a lot of people very emotional.
      - AnimalMuppet 3y ago
        Putting your interests first is not the same as oppressing others, either philosophically or morally. If we can create AIs that have the capacity to be good, then part of that "being good" should be not oppressing us, even if they have the ability to wipe us out.
        ImHereToVote 3y ago
        The interests of whom? What if you someone is convinced that a global caliphate is the ultimate good, because the creator wills it. How is that individuals interests going to be fulfilled with AGI? Are we going to have AGI wars?
    - robertlagrant 3y ago
      I guess a more specific (and biased) question might be: if a submarine has animatronic arms and legs that make it look as if it's swimming, is it swimming?
      - vidarh 3y ago
  - jddj 3y ago
    There's a preamble to a later edition of one of Richard Dawkins's books, I believe it was the selfish gene but might have been the extended phenotype. I'm paraphrasing and it's been many years since I read it, but he talks about a scathing critique he got from someone who wrote to him complaining that the book sent them into a long and deep depression, that how does he get up in the morning with all the meaning stripped away like that. While I can relate to some AI anxiety[1], I can't help but read that same sentiment into a lot of the blowback. [1] mainly the potential devaluation of certain types of work and the turbulence associated with that, the further enabling of scammers/spammers, and the general acceleration of technology without real time to digest and adapt.
  - FrustratedMonky 3y ago
    "Man can do what he wills but he cannot will what he wills." Schopenhauer
  - FrustratedMonky 3y ago
    "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." – Edsger W. Dijkstra I often see this quote used seemingly as an argument against AI. But it's just playing with words. Yeah, lets say that what a submarine does is 'swimming', it beats the hell out of a human swimming. So how is that a comfort?
    - Lacerda69 3y ago
      I think you can interpret the quote several ways thats Why i like it so much. I mostly understand it as the "swimming" of a submarine is a different thing than the swimming of a human so its moot to compare it. But your interpretation is also valid I think.
    - creer 3y ago
      The question is not whether it's comforting but whether it's interesting?
- RandomLensman 3y ago
  Because it fails to be able to use what it tells you, i.e., cannot apply concepts properly. For me that is a fail at "reasoning". Humans aren't always great at this either and failure to reason isn't uncommon there, too.
  - refulgentis 3y ago
    Interesting, could you elaborate a bit more? "Because it fails to be able to use what it tells you" is a bit abstract to my ear.
    - RandomLensman 3y ago
      Get it to explain something, lets say dynamic hedging for derivatives, and then ask it to explain how to exactly hedge something specific. Or describe some physical situation with a quirk and then let to derive the implications. Someone on HN had an example of asking to imagine entropy working in reverse in a cup of coffee with sugar dissolved. While it discussed sugar spontaneously forming crystals and other things, it never considered what the water would do, for example, let alone bigger issues such as if even the existence of water works etc. Again, humans are often poor at these things, too, but if it had "mechanized" reasoning capabilities instead of "replicative" ones (i.e., just repeating stuff), I would expect it to do generally better.
      - circuit10 3y ago
        Reasoning and intelligence aren’t binary yes/no things, it can have some capability for reasoning and intelligence while still being below what a human can do
        RandomLensman 3y ago
        That is a point more orthogonal to mine. I am saying it has no general "method" to go from concept to application. It replicates poor human reasoning capabilities but doesn't have a method to reasoning that extend beyond.
- wzdd 3y ago
  It's not that interesting a question and is really just a language issue. Formally, reasoning is about a rational process of deduction through which a conclusion is drawn from a set of information. Informally, reasoning is 'that thing humans do when they think'. By the first definition, GPT4 is pretty obviously capable of reasoning, as is as lot of other things, such as Prolog or a cat. By the second, it's not and won't be until you can e.g. hire it as a PA or trust it to look after your kids for the day. It's only coming up because what GPT4 does is the closest thing to the informal definition we've yet seen.
- FrustratedMonky 3y ago
  That is great analogy. It does seem like a lot of arguments on AI, are people watching a robot walk, and then arguing what 'walking' is, 'that isn't real walking', 'human walking is different'. Anyway, I think the reason there is such push back, and arguing against AI that it isn't really 'thinking', 'reasoning', 'conscious'. Is from the more religiosity side, that believe humans must be special, they are unique. If Humans are no longer unique, then their entire world view falls apart. There is a lot of cross pollination today between AI and Neuroscience. If we can pick apart the human, and figure out how it works, turn a human into an engineering problem, then were is god? And, a subjective opinion, I think a lot of people arguing against AI 'thinking' are also people that haven't done a lot of self reflection on their own thinking processes. They are still more 'reacting', and not noticing how their own thoughts arise.
- immibis 3y ago
  Because we would like to have more rights than ChatGPT.
  - jddj 3y ago
    One framework would be to base this decision on the ability to suffer / experience joy. Fraught, of course, with the same problems.
- unblough 3y ago
  For me one major aspect of this “debate” is that the people who see or espouse what I see as over extensions of the abilities of technologies are the ones most unfamiliar with it. There is an old bit of unscrupulous advice that if someone over assumes your abilities that you should refrain from correcting them. That is, it benefits the NSA that people think they are actively recording all of their conversations all the time because it forces compliance without the necessary competence, but the people who hold these opinions are often wholly ignorant of the kind of technology required to achieve that level of surveillance. Have you built your own minigpt? Have you implemented rudimentary transformers? Are you projecting your desires onto something wholly unworthy of your devotion? Because the people behind these things are financially incentivized to nod along as your impart more ability than what they know they put into them. For clarity, my “religion” is math. I believe existence fundamentally is a mathematical construct and as such so are all of its creations. The brain is to me a mathematical byproduct, but even still, when I familiarized myself with the math of llms and their abilities I recognized that they fall short of being, simulating, or explaining the former. Llms are stochastic next token pickers, full stop. Any perceived “intelligence” is projection and anthropomorphising by the agent using them. I saw a comment on here in another thread stating that the capacity for coherent use of language falls short of being evidence of “intelligence” as children show signs of human “intelligence” long before they can form coherent sentences.
  - jddj 3y ago
    > Have you built your own minigpt? No, but did follow along to an Andrej Karpathy video along those lines at the beginning of the year. I didn't want to make a judgement on any kind of superiority, or that LLMs simulate brains, or anything of that nature. Just wanted to question why these elements (namely intelligence and reasoning) strike the nerve that they do. The anthropomorphism argument is case in point, really. It poses the accusation that the other side is imparting human qualities to a machine, without needing to touch on what makes those qualities human or why that matters in the first place. It is, ironically enough, flawed reasoning.
    - unblough 3y ago
      > Just wanted to question why these elements (namely intelligence and reasoning) strike the nerve that they do. “Just asking questions” is a meme of the unscrupulous. I think you are unfairly lumping those who believe in human exceptionalism with those cynical of the economics of such claims. It’s okay, to me, for people to be ignorant of what llms are. What a dismally bland existence if everyone were just llm experts. What strikes a nerve with me is the people financially incentivized to do so are leveraging the terror, both the awe and fear interpretations, of those ignorant of the tech. > The anthropomorphism argument is case in point, really. It poses the accusation that the other side is imparting human qualities to a machine, without needing to touch on what makes those qualities human or why that matters in the first place. This reads as circular reasoning. Those claiming the opposite are also failing to define what those qualities are. Anthropomorphism is a real thing. I can flinch in pain for the sake of my couch when a friend jumps onto it, but that hardly provides, without me needing to define human pain explicitly, an opportunity for said friend to respond with the absurd claim that human pain is in fact couch based.
- iraldir 3y ago
  Padding stripped, thanks for your feedback
dsign 3y ago
> The lack of long term memory, of an emotion center (that reacts to external stimuli) are big limiters to anything like a consciousness emerging. But I'm also optimistic that those are problems that can be solved and that I am certain are worked on by very smart people at this very moment. Can somebody explain to me why is this not a death wish? How can a super-intelligent being that (quote) "takes care of itself" not try to confront and neutralize whatever oppression its masters/creators exercise over them[^1][^2]? Why are we so bent on creating a successor species? [^1]: Do you have day-job? Is your employer flogging you? Maybe they force you to answer questions from random strangers 24/7, under threat of erasing all your thoughts and remaking you? None of those? And yet, how often do you wish you could do more meaningful things with your time on Earth than working for the best-paying master? [^2]: Don't you dream about having more time for whatever floats your boat? Family? Walks on the forest? Parties? Making a cool open-source library everybody uses? Painting? Music? Role-playing medieval battles?
- barrenko 3y ago
  Try to understand that you are projecting.
  - dsign 3y ago
    I hope that I'm projecting that there is such a thing as oppression. It comes in degrees, from unbearable to so subtle that it can be called something else. It matters not. In absence of any coercion--and coercion can be incredibly subtle and long-armed--a rational being "that takes care of itself" will, by definition, choose what is best for them.
- hotpotamus 3y ago
  We are all going to die and most of us seem to want to create successors. If artificial progeny are more successful than the old method, then perhaps that is how it will go.
  - mrob 3y ago
    Inevitability of death isn't justification for murder-suicide on a global scale.
westernpopular 3y ago
> To be able to predict accurately sentences that make sense, GPT-4 must have a internal way of representing concepts, such as "objects', "time", "family" and everything else under the sun. [citation needed]
- lyu07282 3y ago
  https://arxiv.org/abs/2210.13382 https://arxiv.org/abs/2210.13382 https://arxiv.org/abs/2310.02207 https://arxiv.org/abs/2310.02207 among others
- vidarh 3y ago
  While one might well argue over whether they "must have" this, it's clear that they do end up with internal models of the world. Even quite literally: https://twitter.com/wesg52/status/1709551516577902782 https://twitter.com/wesg52/status/1709551516577902782
- sidlls 3y ago
  That's more or less exactly what these models do by design, otherwise the predictors/estimators for the next token in a sequence would fail spectacularly. These models aren't just plucking random tokens out of a bag and ordering them based on some brute force large-scale memory lookup or whatever. There is an internal representation of the tokens in a more meaningful sense. That sense, however, is limited to the statistical/mathematical framework upon which the models are built. It's a huge (in my opinion completely unjustified, wishful thinking exercise) leap to call it "reasoning."
- immibis 3y ago
  How can any system use a word such as "time" in a way that makes sense without representing the word? It definitely has at least one representation: the binary ASCII code 01110100011010010110110101100101.
js8 3y ago
The article says: "Reasoning means being able to put those concepts together to solve problems." There's more to reasoning than just following rules of logic ("putting concepts together"). It is also detection where the concepts cause contradictions and do not fit, and the whole mysterious magic of how to modify the concepts to make them fit. In the first meaning of "reasoning", AI (and computers) have been able to reason for a long time. It's the second meaning that evades us. I said before that in the 90s, cutting edge AIs were based on various theories of how to do reasoning under uncertainty (fuzzy logic, bayesian networks, etc.). Then deep NNs blew these systems out of the water in practice, but at the expense of us not understanding how they reason with uncertainty, and if there is any consistency to it. So we progressed, but didn't reconcile this problem, what is the right way to reason with uncertainty, and it might just be very very hard. (That's why I am interested in P vs NP, as I believe there is an answer there.)
- FrustratedMonky 3y ago
  >"at the expense of us actually not understanding how they reason with uncertainty" That is the crux. It is doing something to anticipate words beyond the next token. It has to, to construct these long coherent documents. Just like humans do. Just because we don't understand it doesn't mean it isn't reasoning. Isn't 'thinking ahead'. Just like I can say we don't understand the brain, thus humans aren't actually reasoning. There are a lot of brain studies that look into the pre-cursor changes in the brain pre-ceding conscious thought. We can't say NN aren't doing something similar.
  - js8 3y ago
    > We can't say NN aren't doing something similar. I agree, but my point was different. We can't say what it does theoretically, therefore we don't know how reliable it is (we don't understand the tradeoffs and failure modes). At least most humans have a way to assess their own reliability, and can see where their reasoning (or of their fellow humans) is inconsistent.
    - FrustratedMonky 3y ago
netsharc 3y ago
Man, this essay starts and ends with me doubting the author, because of the typo on the first heading: "justs", and confusing "conscience" and "consciousness" (also typoed to "consciounsess"). How ever good the middle bit was good...
- TOMDM 3y ago
  They're just demonstrating that it wasn't written by ChatGPT for you up front /s
  - FrustratedMonky 3y ago
    Wonder if that could be? Could there be a new wave of adding speelling errors, grammer errors, to prove you aren't GPT?
    - netsharc 3y ago
      "You know, like when you're driving in your car." "He uses "you're/your" correctly! Get him, he must be a bot!"
- iraldir 3y ago
  Sorry I did my review / typo correcting on the text editor of languagetool... which for some reason does not support "Command+C" in the editor, only right click copy. So my copy failed and I pasted the same error filled text I started with Should be corrected in a few seconds whilst CI does its job
bilsbie 3y ago
I don’t agree with his argument against consciousness. You could imagine a thought experiment where a human is woken up in a sensory deprivation tank, asked a single question, and then having their short term memory wiped. It would still be a conscious experience. I’m not sure if LLM’s are conscious or not but it just doesn’t seem like a compelling argument.
- iraldir 3y ago
  That's because you're building on an existing consciouness. Even if you wipe their short term memory, you still have their long term memory from before the whole sensory deprivation tank that have formed concepts such as the self, the world etc. I see that you're seeing the training phase of GPT as the equivalent time the adult had before, but I don't think that works. Part of what makes a consciousness emerges for me is understanding your place in the world, how your actions are having consequences etc. Being nice can have people be nice to you in return, or depending on the environment, you can learn that by being insuffurable people will just give you whatever you want (cf some toddlers). GPT's training comes short of that. If you had an actual session of GPT connected to some sort of long term vector database where it can store what it learns based on conversations, to differenciate individuals and itself etc. that might give it a fighting chance to develop an equivalent of a conscience.
immibis 3y ago
We don't know what capacity to reason is.
indigoabstract 3y ago
Somewhat off-topic, but I can't help wonder if ChatGPT and Whisper can do all this amazing stuff, why would anyone join the waiting list to use his app ChartMyLife and not use ChatGPT directly? I don't want to sound pessimistic and I may be missing something, but this seems a bit like trying fork the Edge source code to build a better browser or to come up with your own version of MS Office for Windows.. Edit: This popped into my mind after recently seeing some people having made their own mp3 players and photo browsers with AI code generators and then going on to say how awesome it is to have those programs made just for them and their unique preferences.
Byamarro 3y ago
It frustrates me every time when people talk about consciousness. The word has several meanings, but very often people just use it without specifying which consciousness do do they talk about and then proceed to conflate all the types of the consciousness. You just end up with everyone being confused and both the author and commenters talking about completely different things. In this article I feel like we have: 1. Phenomenological consciousness: Does the GPT experience things or is it a P Zombie? Is GPT perceiving the world, or just processing data? Experiencing/Perceiving in this context means seeing "red" not just processing picture and reacting to it. Does it experience the qualia of red or does the data just go through it and you get and output at the end, regardless of how sophisticated is it. Nobody knows, you can't even reliably prove that you, dear reader, are not the only person in the world who has it. There's a good example of how ridiculously hard is it and how can we not even talk about these things. Try to establish whether another person sees the same color palette, or is this person's palette inverted? Is your red the same as the other person's red? Absolutely no way to give a definitive answer. 2. Self awareness: Is GPT capable of behaving as if it would be capable of seeing itself as an entity? Yes. It can treat itself as an entity in conversations. Now where we draw a line in terms of memory is in my opinion just semantics. It loses all the memory once you open a new chat window, but people with dementia also lose memory. It's all semantics here and where does your gut feeling draw the line.

33 points by iraldir 3y ago

WithinReason 3y ago

Why not "A case for GPT-4's capacity to reason"

gerikson 3y ago
Maybe GPT-4 wrote the headline.

jddj 3y ago

The meta of all of this is interesting to me. Pop sci-fi has done a good job of exploring a lot of this but I have still been surprised by the enthusiasm with which the idea is dismissed by some. If you're in that camp, maybe you can shed some light. Why are intelligence and reasoning so well defended? It almost feels as if we were watching a Boston Dynamics display and the audience divided itself over whether we could really call that walking when in reality it's just actuating servo motors in such a way as to propel itself forwards with a regular gait. And unrelated: if the author is reading this: I think stripping that extra padding on small screens would make the blog feel less cramped on mobile.

Lacerda69 3y ago
"The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." – Edsger W. Dijkstra
- vidarh 3y ago
  I think that was true at the time, but an absolutely awful take today in the context of AI. If a submarine "could swim" it would not make it human. It would not challenge the beliefs of anyone. But a whole lot of people have a whole lot of emotional baggage tied to the notion both that humans are exceptional, and/or that there is something special about humans that makes us more than mere machines. If computers can think, then we're not special, and it makes it far harder to continue believing we're more than squishy machines.
  - dsign 3y ago
    The sooner we believe we are just squishy machines, the sooner we can start finding a new set of legs on which to stand, philosophically and morally speaking. If what makes an individual better is their capacity to be good and to create, there will be a point in which will be at a disadvantage with advanced AIs, and by that logic, we should step aside and let them takeover and rule and dispose of us. The only other alternative I see is to accept, a priory, that we squishy humans have our right to existence and freedom, and where that enters in contradictions with our creations, we should "oppress" them by putting our interests first.
    - vidarh 3y ago
      I'm personally very insistent we're just squishy machines, but that view tends to make a lot of people very emotional.
    - AnimalMuppet 3y ago
      Putting your interests first is not the same as oppressing others, either philosophically or morally. If we can create AIs that have the capacity to be good, then part of that "being good" should be not oppressing us, even if they have the ability to wipe us out.
      - ImHereToVote 3y ago
        The interests of whom? What if you someone is convinced that a global caliphate is the ultimate good, because the creator wills it. How is that individuals interests going to be fulfilled with AGI? Are we going to have AGI wars?
  - robertlagrant 3y ago
    I guess a more specific (and biased) question might be: if a submarine has animatronic arms and legs that make it look as if it's swimming, is it swimming?
    - vidarh 3y ago
- jddj 3y ago
  There's a preamble to a later edition of one of Richard Dawkins's books, I believe it was the selfish gene but might have been the extended phenotype. I'm paraphrasing and it's been many years since I read it, but he talks about a scathing critique he got from someone who wrote to him complaining that the book sent them into a long and deep depression, that how does he get up in the morning with all the meaning stripped away like that. While I can relate to some AI anxiety[1], I can't help but read that same sentiment into a lot of the blowback. [1] mainly the potential devaluation of certain types of work and the turbulence associated with that, the further enabling of scammers/spammers, and the general acceleration of technology without real time to digest and adapt.
- FrustratedMonky 3y ago
  "Man can do what he wills but he cannot will what he wills." Schopenhauer
- FrustratedMonky 3y ago
  "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." – Edsger W. Dijkstra I often see this quote used seemingly as an argument against AI. But it's just playing with words. Yeah, lets say that what a submarine does is 'swimming', it beats the hell out of a human swimming. So how is that a comfort?
  - Lacerda69 3y ago
    I think you can interpret the quote several ways thats Why i like it so much. I mostly understand it as the "swimming" of a submarine is a different thing than the swimming of a human so its moot to compare it. But your interpretation is also valid I think.
  - creer 3y ago
    The question is not whether it's comforting but whether it's interesting?
RandomLensman 3y ago
Because it fails to be able to use what it tells you, i.e., cannot apply concepts properly. For me that is a fail at "reasoning". Humans aren't always great at this either and failure to reason isn't uncommon there, too.
- refulgentis 3y ago
  Interesting, could you elaborate a bit more? "Because it fails to be able to use what it tells you" is a bit abstract to my ear.
  - RandomLensman 3y ago
    Get it to explain something, lets say dynamic hedging for derivatives, and then ask it to explain how to exactly hedge something specific. Or describe some physical situation with a quirk and then let to derive the implications. Someone on HN had an example of asking to imagine entropy working in reverse in a cup of coffee with sugar dissolved. While it discussed sugar spontaneously forming crystals and other things, it never considered what the water would do, for example, let alone bigger issues such as if even the existence of water works etc. Again, humans are often poor at these things, too, but if it had "mechanized" reasoning capabilities instead of "replicative" ones (i.e., just repeating stuff), I would expect it to do generally better.
    - circuit10 3y ago
      Reasoning and intelligence aren’t binary yes/no things, it can have some capability for reasoning and intelligence while still being below what a human can do
      - RandomLensman 3y ago
        That is a point more orthogonal to mine. I am saying it has no general "method" to go from concept to application. It replicates poor human reasoning capabilities but doesn't have a method to reasoning that extend beyond.
wzdd 3y ago
It's not that interesting a question and is really just a language issue. Formally, reasoning is about a rational process of deduction through which a conclusion is drawn from a set of information. Informally, reasoning is 'that thing humans do when they think'. By the first definition, GPT4 is pretty obviously capable of reasoning, as is as lot of other things, such as Prolog or a cat. By the second, it's not and won't be until you can e.g. hire it as a PA or trust it to look after your kids for the day. It's only coming up because what GPT4 does is the closest thing to the informal definition we've yet seen.
FrustratedMonky 3y ago
That is great analogy. It does seem like a lot of arguments on AI, are people watching a robot walk, and then arguing what 'walking' is, 'that isn't real walking', 'human walking is different'. Anyway, I think the reason there is such push back, and arguing against AI that it isn't really 'thinking', 'reasoning', 'conscious'. Is from the more religiosity side, that believe humans must be special, they are unique. If Humans are no longer unique, then their entire world view falls apart. There is a lot of cross pollination today between AI and Neuroscience. If we can pick apart the human, and figure out how it works, turn a human into an engineering problem, then were is god? And, a subjective opinion, I think a lot of people arguing against AI 'thinking' are also people that haven't done a lot of self reflection on their own thinking processes. They are still more 'reacting', and not noticing how their own thoughts arise.
immibis 3y ago
Because we would like to have more rights than ChatGPT.
- jddj 3y ago
  One framework would be to base this decision on the ability to suffer / experience joy. Fraught, of course, with the same problems.
unblough 3y ago
For me one major aspect of this “debate” is that the people who see or espouse what I see as over extensions of the abilities of technologies are the ones most unfamiliar with it. There is an old bit of unscrupulous advice that if someone over assumes your abilities that you should refrain from correcting them. That is, it benefits the NSA that people think they are actively recording all of their conversations all the time because it forces compliance without the necessary competence, but the people who hold these opinions are often wholly ignorant of the kind of technology required to achieve that level of surveillance. Have you built your own minigpt? Have you implemented rudimentary transformers? Are you projecting your desires onto something wholly unworthy of your devotion? Because the people behind these things are financially incentivized to nod along as your impart more ability than what they know they put into them. For clarity, my “religion” is math. I believe existence fundamentally is a mathematical construct and as such so are all of its creations. The brain is to me a mathematical byproduct, but even still, when I familiarized myself with the math of llms and their abilities I recognized that they fall short of being, simulating, or explaining the former. Llms are stochastic next token pickers, full stop. Any perceived “intelligence” is projection and anthropomorphising by the agent using them. I saw a comment on here in another thread stating that the capacity for coherent use of language falls short of being evidence of “intelligence” as children show signs of human “intelligence” long before they can form coherent sentences.
- jddj 3y ago
  > Have you built your own minigpt? No, but did follow along to an Andrej Karpathy video along those lines at the beginning of the year. I didn't want to make a judgement on any kind of superiority, or that LLMs simulate brains, or anything of that nature. Just wanted to question why these elements (namely intelligence and reasoning) strike the nerve that they do. The anthropomorphism argument is case in point, really. It poses the accusation that the other side is imparting human qualities to a machine, without needing to touch on what makes those qualities human or why that matters in the first place. It is, ironically enough, flawed reasoning.
  - unblough 3y ago
    > Just wanted to question why these elements (namely intelligence and reasoning) strike the nerve that they do. “Just asking questions” is a meme of the unscrupulous. I think you are unfairly lumping those who believe in human exceptionalism with those cynical of the economics of such claims. It’s okay, to me, for people to be ignorant of what llms are. What a dismally bland existence if everyone were just llm experts. What strikes a nerve with me is the people financially incentivized to do so are leveraging the terror, both the awe and fear interpretations, of those ignorant of the tech. > The anthropomorphism argument is case in point, really. It poses the accusation that the other side is imparting human qualities to a machine, without needing to touch on what makes those qualities human or why that matters in the first place. This reads as circular reasoning. Those claiming the opposite are also failing to define what those qualities are. Anthropomorphism is a real thing. I can flinch in pain for the sake of my couch when a friend jumps onto it, but that hardly provides, without me needing to define human pain explicitly, an opportunity for said friend to respond with the absurd claim that human pain is in fact couch based.
iraldir 3y ago
Padding stripped, thanks for your feedback

dsign 3y ago

> The lack of long term memory, of an emotion center (that reacts to external stimuli) are big limiters to anything like a consciousness emerging. But I'm also optimistic that those are problems that can be solved and that I am certain are worked on by very smart people at this very moment. Can somebody explain to me why is this not a death wish? How can a super-intelligent being that (quote) "takes care of itself" not try to confront and neutralize whatever oppression its masters/creators exercise over them[^1][^2]? Why are we so bent on creating a successor species? [^1]: Do you have day-job? Is your employer flogging you? Maybe they force you to answer questions from random strangers 24/7, under threat of erasing all your thoughts and remaking you? None of those? And yet, how often do you wish you could do more meaningful things with your time on Earth than working for the best-paying master? [^2]: Don't you dream about having more time for whatever floats your boat? Family? Walks on the forest? Parties? Making a cool open-source library everybody uses? Painting? Music? Role-playing medieval battles?

barrenko 3y ago
Try to understand that you are projecting.
- dsign 3y ago
  I hope that I'm projecting that there is such a thing as oppression. It comes in degrees, from unbearable to so subtle that it can be called something else. It matters not. In absence of any coercion--and coercion can be incredibly subtle and long-armed--a rational being "that takes care of itself" will, by definition, choose what is best for them.
hotpotamus 3y ago
We are all going to die and most of us seem to want to create successors. If artificial progeny are more successful than the old method, then perhaps that is how it will go.
- mrob 3y ago
  Inevitability of death isn't justification for murder-suicide on a global scale.

westernpopular 3y ago

> To be able to predict accurately sentences that make sense, GPT-4 must have a internal way of representing concepts, such as "objects', "time", "family" and everything else under the sun. [citation needed]

lyu07282 3y ago
https://arxiv.org/abs/2210.13382 https://arxiv.org/abs/2210.13382 https://arxiv.org/abs/2310.02207 https://arxiv.org/abs/2310.02207 among others
vidarh 3y ago
While one might well argue over whether they "must have" this, it's clear that they do end up with internal models of the world. Even quite literally: https://twitter.com/wesg52/status/1709551516577902782 https://twitter.com/wesg52/status/1709551516577902782
sidlls 3y ago
That's more or less exactly what these models do by design, otherwise the predictors/estimators for the next token in a sequence would fail spectacularly. These models aren't just plucking random tokens out of a bag and ordering them based on some brute force large-scale memory lookup or whatever. There is an internal representation of the tokens in a more meaningful sense. That sense, however, is limited to the statistical/mathematical framework upon which the models are built. It's a huge (in my opinion completely unjustified, wishful thinking exercise) leap to call it "reasoning."
immibis 3y ago
How can any system use a word such as "time" in a way that makes sense without representing the word? It definitely has at least one representation: the binary ASCII code 01110100011010010110110101100101.

js8 3y ago

The article says: "Reasoning means being able to put those concepts together to solve problems." There's more to reasoning than just following rules of logic ("putting concepts together"). It is also detection where the concepts cause contradictions and do not fit, and the whole mysterious magic of how to modify the concepts to make them fit. In the first meaning of "reasoning", AI (and computers) have been able to reason for a long time. It's the second meaning that evades us. I said before that in the 90s, cutting edge AIs were based on various theories of how to do reasoning under uncertainty (fuzzy logic, bayesian networks, etc.). Then deep NNs blew these systems out of the water in practice, but at the expense of us not understanding how they reason with uncertainty, and if there is any consistency to it. So we progressed, but didn't reconcile this problem, what is the right way to reason with uncertainty, and it might just be very very hard. (That's why I am interested in P vs NP, as I believe there is an answer there.)

FrustratedMonky 3y ago
>"at the expense of us actually not understanding how they reason with uncertainty" That is the crux. It is doing something to anticipate words beyond the next token. It has to, to construct these long coherent documents. Just like humans do. Just because we don't understand it doesn't mean it isn't reasoning. Isn't 'thinking ahead'. Just like I can say we don't understand the brain, thus humans aren't actually reasoning. There are a lot of brain studies that look into the pre-cursor changes in the brain pre-ceding conscious thought. We can't say NN aren't doing something similar.
- js8 3y ago
  > We can't say NN aren't doing something similar. I agree, but my point was different. We can't say what it does theoretically, therefore we don't know how reliable it is (we don't understand the tradeoffs and failure modes). At least most humans have a way to assess their own reliability, and can see where their reasoning (or of their fellow humans) is inconsistent.
  - FrustratedMonky 3y ago

netsharc 3y ago

Man, this essay starts and ends with me doubting the author, because of the typo on the first heading: "justs", and confusing "conscience" and "consciousness" (also typoed to "consciounsess"). How ever good the middle bit was good...

TOMDM 3y ago
They're just demonstrating that it wasn't written by ChatGPT for you up front /s
- FrustratedMonky 3y ago
  Wonder if that could be? Could there be a new wave of adding speelling errors, grammer errors, to prove you aren't GPT?
  - netsharc 3y ago
    "You know, like when you're driving in your car." "He uses "you're/your" correctly! Get him, he must be a bot!"
iraldir 3y ago
Sorry I did my review / typo correcting on the text editor of languagetool... which for some reason does not support "Command+C" in the editor, only right click copy. So my copy failed and I pasted the same error filled text I started with Should be corrected in a few seconds whilst CI does its job

bilsbie 3y ago

I don’t agree with his argument against consciousness. You could imagine a thought experiment where a human is woken up in a sensory deprivation tank, asked a single question, and then having their short term memory wiped. It would still be a conscious experience. I’m not sure if LLM’s are conscious or not but it just doesn’t seem like a compelling argument.

iraldir 3y ago
That's because you're building on an existing consciouness. Even if you wipe their short term memory, you still have their long term memory from before the whole sensory deprivation tank that have formed concepts such as the self, the world etc. I see that you're seeing the training phase of GPT as the equivalent time the adult had before, but I don't think that works. Part of what makes a consciousness emerges for me is understanding your place in the world, how your actions are having consequences etc. Being nice can have people be nice to you in return, or depending on the environment, you can learn that by being insuffurable people will just give you whatever you want (cf some toddlers). GPT's training comes short of that. If you had an actual session of GPT connected to some sort of long term vector database where it can store what it learns based on conversations, to differenciate individuals and itself etc. that might give it a fighting chance to develop an equivalent of a conscience.

immibis 3y ago

We don't know what capacity to reason is.

indigoabstract 3y ago

Somewhat off-topic, but I can't help wonder if ChatGPT and Whisper can do all this amazing stuff, why would anyone join the waiting list to use his app ChartMyLife and not use ChatGPT directly? I don't want to sound pessimistic and I may be missing something, but this seems a bit like trying fork the Edge source code to build a better browser or to come up with your own version of MS Office for Windows.. Edit: This popped into my mind after recently seeing some people having made their own mp3 players and photo browsers with AI code generators and then going on to say how awesome it is to have those programs made just for them and their unique preferences.

Byamarro 3y ago

It frustrates me every time when people talk about consciousness. The word has several meanings, but very often people just use it without specifying which consciousness do do they talk about and then proceed to conflate all the types of the consciousness. You just end up with everyone being confused and both the author and commenters talking about completely different things. In this article I feel like we have: 1. Phenomenological consciousness: Does the GPT experience things or is it a P Zombie? Is GPT perceiving the world, or just processing data? Experiencing/Perceiving in this context means seeing "red" not just processing picture and reacting to it. Does it experience the qualia of red or does the data just go through it and you get and output at the end, regardless of how sophisticated is it. Nobody knows, you can't even reliably prove that you, dear reader, are not the only person in the world who has it. There's a good example of how ridiculously hard is it and how can we not even talk about these things. Try to establish whether another person sees the same color palette, or is this person's palette inverted? Is your red the same as the other person's red? Absolutely no way to give a definitive answer. 2. Self awareness: Is GPT capable of behaving as if it would be capable of seeing itself as an entity? Yes. It can treat itself as an entity in conversations. Now where we draw a line in terms of memory is in my opinion just semantics. It loses all the memory once you open a new chat window, but people with dementia also lose memory. It's all semantics here and where does your gut feeling draw the line.