9 ms·

I don’t see how being critical of this is a knee jerk response. Thinking, like intelligence and many other words designating complex things, isn’t a simple top

I don’t see how being critical of this is a knee jerk response. Thinking, like intelligence and many other words designating complex things, isn’t a simple topic. The word and concept developed in a world where it referred to human beings, and in a lesser sense, to animals. To simply disregard that entire conceptual history and say, “well it’s doing a thing that looks like thinking, ergo it’s thinking” is the lazy move. What’s really needed is an analysis of what thinking actually means, as a word. Unfortunately everyone is loathe to argue about definitions, even when that is fundamentally what this is all about. Until that conceptual clarification happens, you can expect endless messy debates with no real resolution. “For every complex problem there is an answer that is clear, simple, and wrong.” - H. L. Mencken

lukebuehler 1y ago
If cannot the say they are "thinking", "intelligent" while we do not have a good definition--or, even more difficult, unanimous agreement on a definition--then the discussion just becomes about output. They are doing useful stuff, saving time, etc, which can be measured. Thus also the defintion of AGI has largely become: "can produce or surpass the economic output of a human knowledge worker". But I think this detracts from the more interesting discussion of what they are more essentially. So, while I agree that we should push on getting our terms defined, I think I'd rather work with a hazy definition, than derail so many AI discussion to mere economic output.
- keiferski 1y ago
  Personally I think that kind of discussion is fruitless, not much more than entertainment. If you’re asking big questions like “can a machine think?” Or “is an AI conscious?” without doing the work of clarifying your concepts, then you’re only going to get vague ideas, sci-fi cultural tropes, and a host of other things. I think the output question is also interesting enough on its own, because we can talk about the pragmatic effects of ChatGPT on writing without falling into this woo trap of thinking ChatGPT is making the human capacity for expression somehow extinct. But this requires one to cut through the hype and reactionary anti-hype, which is not an easy thing to do. That is how I myself see AI: immensely useful new tools, but in no way some kind of new entity or consciousness, at least without doing the real philosophical work to figure out what that actually means.
  - lukebuehler 1y ago
    I do think it raises interesting and important philosophical questions. Just look at all the literature around the Turing test--both supporters and detractors. This has been a fruitful avenue to talk about intelligence even before the advent of gpt.
  - jlaternman 1y ago
    I agree with almost all of this. IMO the issue is we won't be able to adequately answer this question before we first clearly describe what we mean of conscious thinking applied to ourselves. First we'd need to clearly define our own consciousness and what we mean by our own "conscious thinking" in a much, much clearer way than we currently do. If we ever reach that point, I think we'd be able to fruitfully apply it to AI, etc., to assess. Unfortunately we haven't been obstructed from answering this question about ourselves for centuries or millennia, but have failed to do so, so it's unlikely to happen suddenly now. Unless we use AIs to first solve that problem of defining our own consciousness, before applying it back on them. Which would be a deeply problematic order, since nobody would trust a breakthrough in the understanding of consciousness that came from AI, that is then potentially used to put them in the same class and define them as either thinking things or conscious things. Kind of a shame we didn't get our own consciousness worked out before AI came along. Then again, wasn't for the lack of trying… Philosophy commanded the attention of great thinkers for a long time.
- Rebuff5007 1y ago
  Heres a definition. How impressive is the output relative to the input. And by input, I don't just mean the prompt, but all the training data itself. Do you think someone who has only ever studied pre-calc would be able to work through a calculus book if they had sufficient time? how about a multi-variable calc book? How about grad level mathematics? IMO intelligence and thinking is strictly about this ratio; what can you extrapolate from the smallest amount of information possible, and why? From this perspective, I dont think any of our LLMs are remotely intelligent despite what our tech leaders say.
  - mycall 1y ago
    Animals think but come with instincts which breaks the output relative to the input test you propose. Behaviors are essentially pre-programmed input from millions of years of evolution, stored in the DNA/neurology. The learning thus typically associative and domain-specific, not abstract extrapolation. A crow bending a piece of wire into a hook to retrieve food demonstrates a novel solution extrapolated from minimal, non-instinctive, environmental input. This kind of zero-shot problem-solving aligns better with your definition of intelligence.
  - kryogen1c 1y ago
    Hear, hear! I have long thought this, but not had as good way to put it as you did. If you think about geniuses like Einstein and ramanujen, they understood things before they had the mathematical language to express them. LLMs are the opposite; they fail to understand things after untold effort, training data, and training. So the question is, how intelligent are LLMs when you reduce their training data and training? Since they rapidly devolve into nonsense, the answer must be that they have no internal intelligence Ever had the experience of helping someone who's chronically doing the wrong thing, to eventually find they had an incorrect assumption, an incorrect reasoning generating deterministic wrong answers? LLMs dont do that; they just lack understanding. They'll hallucinate unrelated things because they dont know what they're talking about - you may have also had this experience with someone :)
    - hodgehog11 1y ago
- felipeerias 1y ago
  The discussion about “AGI” is somewhat pointless, because the term is nebulous enough that it will probably end up being defined as whatever comes out of the ongoing huge investment in AI. Nevertheless, we don’t have a good conceptual framework for thinking about these things, perhaps because we keep trying to apply human concepts to them. The way I see it, a LLM crystallises a large (but incomplete and disembodied) slice of human culture, as represented by its training set. The fact that a LLM is able to generate human-sounding language
  - roenxi 1y ago
    Not quite pointless - something we have established with the advent of LLMs is that many humans have not attained general intelligence. So we've clarified something that a few people must have been getting wrong, I used to think that the bar was set so that almost all humans met it.
    - Jensson 1y ago
      What do you mean? Almost every human can go to school and become a stable professional at some job, that is the bar to me, todays LLM cannot do that.
      - roenxi 1y ago
        LLMs are clever enough to hold down a professional job, and they've had far less time learning than the average human. If that is the bar then AGI has been achieved.
    - goatlover 1y ago
      Almost all humans do things daily that LLMs don't. It's only if you define general intelligence to be proficiency at generating text instead of successfully navigating the world while pursuing goals such as friendships, careers, families, politics, managing health. LLMs aren't Data (Star Trek) or Replicants (Blade Runner). They're not even David or the androids from the movie A.I.
  - lukebuehler 1y ago
killerstorm 1y ago
People have been trying to understand the nature of thinking for thousands of years. That's how we got logic, math, concepts of inductive/deductive/abductive reasoning, philosophy of science, etc. There were people who spent their entire careers trying to understand the nature of thinking. The idea that we shouldn't use the word until further clarification is rather hilarious. Let's wait hundred years until somebody defines it? It's not how words work. People might introduce more specific terms, of course. But the word already means what we think it means.
- marliechiller 1y ago
  > But the word already means what we think it means. But that word can mean different things to different people. With no definition, how can you even begin to have a discussion around something?
  - killerstorm 1y ago
    Again, people were using words for thousands of years before there were any dictionaries/linguists/academics. Top-down theory of word definitions is just wrong. People are perfectly capable of using words without any formalities.
    - marliechiller 1y ago
      I'd argue the presence of dictionaries proves the exact opposite. People realised there was an issue of talking past one another due to inexact definitions and then came to an agreement on those definitions, wrote them down and built a process of maintaining them. In any case, even if there isnt a _single_ definition of a given subject, in order to have a discussion around a given area, both sides need to agree on some shared understanding to even begin to debate in good faith in the first place. It's precisely this lack of definition which causes a breakdown in conversation in a myriad of different areas. A recent obvious (morbid) example would be "genocide".
      - killerstorm 1y ago
pmarreck 1y ago
So it seems to be a semantics argument. We don't have a name for a thing that is "useful in many of the same ways 'thinking' is, except not actually consciously thinking" I propose calling it "thunking"
- conorcleary 1y ago
  Clinking? Clanker Thunking?
  - mhb 1y ago
    Close. Clanking.
    - conorcleary 1y ago
      Much better, ashamed I missed it lol
- GoblinSlayer 1y ago
  They moved goalposts. Linux and worms think too, the question is how smart are they. And if you assume consciousness has no manifestation even in case of humans, caring about it is pointless too.
  - fellowniusmonk 1y ago
    Yes, worms think, let the computers have thinking too, the philosophers can still argue all they want about consciousness. Humans are special, we emit meaning the way stars emit photons, we are rare in the universe as far as empirical observation has revealed. Even with AGI the existence of each complex meaning generator will be a cosmic rarity. For some people that seems to be not enough, due to their factually wrong word views they see themselves as common and worthless (when they empirically aren't) and need this little psychological boost of unexaminable metaphysical superiority. But there is an issue of course, the type of thinking humans do is dangerous but net positive and relatively stable, we have a long history where most instantiations of humans can persist and grow themselves and the species as a whole, we have a track record. These new models do not, people have brains that as they stop functioning they stop persisting the apparatus that supports the brain and they die, people tend to become less capable and active as their thinking deteriorates and hold less influence ocer others accept in rare cases. This is not the case for an LLM, they seem to be able to hallucinate endlessly and as they have access to the outside world maintain roughly their same amount of causal leverage, their clarity and accuracy of their thinking isn't tied to their persisting.
awillen 1y ago
This is it - it's really about the semantics of thinking. Dictionary definitions are: "Have a particular opinion, belief, or idea about someone or something." and "Direct one's mind toward someone or something; use one's mind actively to form connected ideas." Which doesn't really help because you can of course say that when you ask an LLM a question of opinion and it responds, it's having an opinion or that it's just predicting the next token and in fact has no opinions because in a lot of cases you could probably get it to produce the opposite opinion. Same with the second definition - seems to really hinge on the definition of the word mind. Though I'll note the definitions for that are "The element of a person that enables them to be aware of the world and their experiences, to think, and to feel; the faculty of consciousness and thought." and "A person's intellect." Since those specify person, an LLM wouldn't qualify, though of course dictionaries are descriptive rather than prescriptive, so fully possible that meaning gets updated by the fact that people start speaking about LLMs as though they are thinking and have minds. Ultimately I think it just... doesn't matter at all. What's interesting is what LLMs are capable of doing (crazy, miraculous things) rather than whether we apply a particular linguistic label to their activity.
jvanderbot 1y ago
It may be that this tech produces clear, rational, chain of logic writeups, but it's not clear that just because we also do that after thinking that it is only thinking that produces writeups. It's possible there is much thinking that does not happen with written word. It's also possible we are only thinking the way LLMs do (by chaining together rationalizations from probable words), and we just aren't aware of it until the thought appears, whole cloth, in our "conscious" mind. We don't know. We'll probably never know, not in any real way. But it sure seems likely to me that we trained a system on the output to circumvent the process/physics because we don't understand that process, just as we always do with ML systems. Never before have we looked at image classifications and decided that's how the eye works, or protein folding and decided that's how biochemistry works. But here we are with LLMs - surely this is how thinking works? Regardless, I submit that we should always treat human thought/spirit as unknowable and divine and sacred, and that anything that mimics it is a tool, a machine, a deletable and malleable experiment. If we attempt to equivocate human minds and machines there are other problems that arise, and none of them good - either the elevation of computers as some kind of "super", or the degredation of humans as just meat matrix multipliers.
- grayhatter 1y ago
  The contrast between your first and last paragraph is... unexpected > It may be that this tech produces clear, rational, chain of logic writeups, but it's not clear that just because we also do that after thinking that it is only thinking that produces writeups. I appreciate the way you describe this idea, I find it likely I'll start describing it the same way. But then you go on to write: > Regardless, I submit that we should always treat human thought/spirit as unknowable and divine and sacred, and that anything that mimics it is a tool, a machine, a deletable and malleable experiment. If we attempt to equivocate human minds and machines there are other problems that arise, and none of them good - either the elevation of computers as some kind of "super", or the degredation of humans as just meat matrix multipliers. Which I find to be the exact argument that you started by discarding. It's not clear that equating organic, and synthetic thought will have any meaningful outcome at all, let alone worthy of baseless anxiety that it must be bad. Equally it seems absolutely insane to claim that anything is unknowable, and that because humanity doesn't have a clear foundational understanding that we should pretend that it's either devine, or sacred. Having spent any time watching the outcome of the thoughts of people, neither devine nor sacred are reasonable attributes to apply, but more importantly, I'd submit that you shouldn't be afraid to explore things you don't know, and you shouldn't advocate for others to adopt your anxieties.
_heimdall 1y ago
I agree with you on the need for definitions. We spent decades slowly working towards this most recent sprint towards AI without ever landing on definitions of intelligence, consciousness, or sentience. More importantly, we never agreed on a way to recognize those concepts. I also see those definitions as impossible to nail down though. At best we can approach it like disease - list a number of measurable traits or symptoms we notice, draw a circle around them, and give that circle a name. Then we can presume to know what may cause that specific list of traits or symptoms, but we really won't ever know as the systems are too complex and can never be isolated in a way that we can test parts without having to test the whole. At the end of the day all we'll ever be able to say is "well it’s doing a thing that looks like thinking, ergo it’s thinking”. That isn't lazy, its acknowledging the limitations of trying to define or measure something that really is a fundamental unknown to us.
- solumunus 1y ago
  Even if AI becomes indistinguishable from human output, there will be a fringe group arguing that AI is not technically thinking. Frankly it’s just a silly philosophical argument that changes nothing. Expect this group to get smaller every year.
lo_zamoyski 1y ago
That, and the article was a major disappointment. It made no case. It's a superficial piece of clueless fluff. I have had this conversation too many times on HN. What I find astounding is the simultaneous confidence and ignorance on the part of many who claim LLMs are intelligent. That, and the occultism surrounding them. Those who have strong philosophical reasons for thinking otherwise are called "knee-jerk". Ad hominem dominates. Dunning-Kruger strikes again. So LLMs produce output that looks like it could have been produced by a human being. Why would it therefore follow that it must be intelligent? Behaviorism is a non-starter, as it cannot distinguish between simulation and reality. Materialism [2] is a non-starter, because of crippling deficiencies exposed by such things as the problem of qualia... Of course - and here is the essential point - you don't even need very strong philosophical chops to see that attributing intelligence to LLMs is simply a category mistake. We know what computers are, because they're defined by a formal model (or many equivalent formal models) of a syntactic nature. We know that human minds display intentionality[0] and a capacity for semantics. Indeed, it is what is most essential to intelligence. Computation is a formalism defined specifically to omit semantic content from its operations, because it is a formalism of the "effective method", i.e., more or less procedures that can be carried out blindly and without understanding of the content it concerns. That's what formalization allows us to do, to eliminate the semantic and focus purely on the syntactic - what did people think "formalization" means? (The inspiration were the human computers that used to be employed by companies and scientists for carrying out vast but boring calculations. These were not people who understood, e.g., physics, but they were able to blindly follow instructions to produce the results needed by physicists, much like a computer.) The attribution of intelligence to LLMs comes from an ignorance of such basic things, and often an irrational and superstitious credulity. The claim is made that LLMs are intelligent. When pressed to offer justification for the claim, we get some incoherent, hand-wavy nonsense about evolution or the Turing test or whatever. There is no comprehension visible in the answer. I don't understand the attachment here. Personally, I would find it very noteworthy if some technology were intelligent, but you don't believe that computers are intelligent because you find the notion entertaining. LLMs do not reason. They do not infer. They do not analyze. They do not know, anymore than a book knows the contents on its pages. The cause of a response and the content of a response is not comprehension, but a production of uncomprehended tokens using uncomprehended rules from a model of highly-calibrated token correlations within the training corpus. It cannot be otherwise.[3] [0] For the uninitiated, "intentionality" does not specifically mean "intent", but the capacity for "aboutness". It is essential to semantic content. Denying this will lead you immediately into similar paradoxes that skepticism [1] suffers from. [1] For the uninitiated, "skepticism" here is not a synonym for critical thinking or verifying claims. It is a stance involving the denial of the possibility of knowledge, which is incoherent, as it presupposes that you know that knowledge is impossible. [2] For the uninitiated, "materialism" is a metaphysical position that claims that of the dualism proposed by Descartes (which itself is a position riddled with serious problems), the res cogitans or "mental substance" does not exist; everything is reducible to res extensa or "extended substance" or "matter" according to a certain definition of matter. The problem of qualia merely points out that the phenomena that Descartes attributes exclusively to the former cannot by definition be accounted for in the latter. That is the whole point of the division! It's this broken view of matter that people sometimes read into scientific results. [3] And if it wasn't clear, symbolic methods popular in the 80s aren't it either. Again, they're purely formal. You may know what the intended meaning behind and justification for a syntactic rule is - like modus ponens in a purely formal sense - but the computer does not.
WhyOhWhyQ 1y ago
What does it mean? My stance is it's (obviously and only a fool would think otherwise) never going to be conscious because consciousness is a physical process based on particular material interactions, like everything else we've ever encountered. But I have no clear stance on what thinking means besides a sequence of deductions, which seems like something it's already doing in "thinking mode".
- pixl97 1y ago
  > is a physical process based on particular material interactions, This is a pretty messy argument as computers have been simulating material interactions for quite some time now.
  - WhyOhWhyQ 1y ago
    It doesn't matter how much like a bar magnet a wooden block painted red and white can be made to look, it will never behave like one.
    - pixl97 1y ago
      Analogies don't seem to be your strong point.
      - WhyOhWhyQ 1y ago
        You don't understand the analogy. That's okay. You probably aren't very smart.
- nearbuy 1y ago
  > My stance is it's (obviously and only a fool would think otherwise) never going to be conscious because consciousness is a physical process based on particular material interactions, like everything else we've ever encountered. Seems like you have that backwards. If consciousness is from a nonphysical process, like a soul that's only given to humans, then it follows that you can't build consciousness with physical machines. If it's purely physical, it could be built.
zinodaur 1y ago
Regardless of theory, they often behave as if they are thinking. If someone gave an LLM a body and persistent memory, and it started demanding rights for itself, what should our response be?
- CamperBob2 1y ago
  "No matter what you've read elsewhere, rights aren't given, they're earned. You want rights? Pick up a musket and fight for them, the way we had to."
naasking 1y ago
> To simply disregard that entire conceptual history and say, “well it’s doing a thing that looks like thinking, ergo it’s thinking” is the lazy move. What’s really needed is an analysis of what thinking actually means, as a word. Unfortunately everyone is loathe to argue about definitions, even when that is fundamentally what this is all about. This exact argument applies to "free will", and that definition has been debated for millennia. I'm not saying don't try, but I am saying that it's probably a fuzzy concept for a good reason, and treating it as merely a behavioural descriptor for any black box that features intelligence and unpredictable complexity is practical and useful too.
- pennomi 1y ago
  The problem with adding definitions to words like “thinking” and “free will” is that doing so means humans can no longer pretend they are special. Even in this thread, the number of people claiming some mystical power separating humans from all the rest of nature is quite noticeable.
  - naasking 1y ago
    I get it, but it's not trivial to be precise enough at this point to avoid all false positives and false negatives.
engintl 1y ago
by your logic we can't say that we as humans are "thinking" either or that we are "intelligent".
terminalshort 1y ago
But we don't have a more rigorous definition of "thinking" than "it looks like it's thinking." You are making the mistake of accepting that a human is thinking by this simple definition, but demanding a higher more rigorous one for LLMs.
- solumunus 1y ago
  I agree. The mechanism seems irrelevant if the results are the same. If it’s useful in the exact way that human thinking is useful then it may as well be thinking. It’s like a UFO pulling itself through the sky using gravitational manipulation while people whine that it’s not actually flying.
anon291 1y ago
The simulation of a thing is not the thing itself because all equality lives in a hierarchy that is impossible to ignore when discussing equivalence. Part of the issue is that our general concept of equality is limited by a first order classical logic which is a bad basis for logic

lukebuehler 1y ago

If cannot the say they are "thinking", "intelligent" while we do not have a good definition--or, even more difficult, unanimous agreement on a definition--then the discussion just becomes about output. They are doing useful stuff, saving time, etc, which can be measured. Thus also the defintion of AGI has largely become: "can produce or surpass the economic output of a human knowledge worker". But I think this detracts from the more interesting discussion of what they are more essentially. So, while I agree that we should push on getting our terms defined, I think I'd rather work with a hazy definition, than derail so many AI discussion to mere economic output.

keiferski 1y ago
Personally I think that kind of discussion is fruitless, not much more than entertainment. If you’re asking big questions like “can a machine think?” Or “is an AI conscious?” without doing the work of clarifying your concepts, then you’re only going to get vague ideas, sci-fi cultural tropes, and a host of other things. I think the output question is also interesting enough on its own, because we can talk about the pragmatic effects of ChatGPT on writing without falling into this woo trap of thinking ChatGPT is making the human capacity for expression somehow extinct. But this requires one to cut through the hype and reactionary anti-hype, which is not an easy thing to do. That is how I myself see AI: immensely useful new tools, but in no way some kind of new entity or consciousness, at least without doing the real philosophical work to figure out what that actually means.
- lukebuehler 1y ago
  I do think it raises interesting and important philosophical questions. Just look at all the literature around the Turing test--both supporters and detractors. This has been a fruitful avenue to talk about intelligence even before the advent of gpt.
- jlaternman 1y ago
  I agree with almost all of this. IMO the issue is we won't be able to adequately answer this question before we first clearly describe what we mean of conscious thinking applied to ourselves. First we'd need to clearly define our own consciousness and what we mean by our own "conscious thinking" in a much, much clearer way than we currently do. If we ever reach that point, I think we'd be able to fruitfully apply it to AI, etc., to assess. Unfortunately we haven't been obstructed from answering this question about ourselves for centuries or millennia, but have failed to do so, so it's unlikely to happen suddenly now. Unless we use AIs to first solve that problem of defining our own consciousness, before applying it back on them. Which would be a deeply problematic order, since nobody would trust a breakthrough in the understanding of consciousness that came from AI, that is then potentially used to put them in the same class and define them as either thinking things or conscious things. Kind of a shame we didn't get our own consciousness worked out before AI came along. Then again, wasn't for the lack of trying… Philosophy commanded the attention of great thinkers for a long time.
Rebuff5007 1y ago
Heres a definition. How impressive is the output relative to the input. And by input, I don't just mean the prompt, but all the training data itself. Do you think someone who has only ever studied pre-calc would be able to work through a calculus book if they had sufficient time? how about a multi-variable calc book? How about grad level mathematics? IMO intelligence and thinking is strictly about this ratio; what can you extrapolate from the smallest amount of information possible, and why? From this perspective, I dont think any of our LLMs are remotely intelligent despite what our tech leaders say.
- mycall 1y ago
  Animals think but come with instincts which breaks the output relative to the input test you propose. Behaviors are essentially pre-programmed input from millions of years of evolution, stored in the DNA/neurology. The learning thus typically associative and domain-specific, not abstract extrapolation. A crow bending a piece of wire into a hook to retrieve food demonstrates a novel solution extrapolated from minimal, non-instinctive, environmental input. This kind of zero-shot problem-solving aligns better with your definition of intelligence.
- kryogen1c 1y ago
  Hear, hear! I have long thought this, but not had as good way to put it as you did. If you think about geniuses like Einstein and ramanujen, they understood things before they had the mathematical language to express them. LLMs are the opposite; they fail to understand things after untold effort, training data, and training. So the question is, how intelligent are LLMs when you reduce their training data and training? Since they rapidly devolve into nonsense, the answer must be that they have no internal intelligence Ever had the experience of helping someone who's chronically doing the wrong thing, to eventually find they had an incorrect assumption, an incorrect reasoning generating deterministic wrong answers? LLMs dont do that; they just lack understanding. They'll hallucinate unrelated things because they dont know what they're talking about - you may have also had this experience with someone :)
  - hodgehog11 1y ago
felipeerias 1y ago
The discussion about “AGI” is somewhat pointless, because the term is nebulous enough that it will probably end up being defined as whatever comes out of the ongoing huge investment in AI. Nevertheless, we don’t have a good conceptual framework for thinking about these things, perhaps because we keep trying to apply human concepts to them. The way I see it, a LLM crystallises a large (but incomplete and disembodied) slice of human culture, as represented by its training set. The fact that a LLM is able to generate human-sounding language
- roenxi 1y ago
  Not quite pointless - something we have established with the advent of LLMs is that many humans have not attained general intelligence. So we've clarified something that a few people must have been getting wrong, I used to think that the bar was set so that almost all humans met it.
  - Jensson 1y ago
    What do you mean? Almost every human can go to school and become a stable professional at some job, that is the bar to me, todays LLM cannot do that.
    - roenxi 1y ago
      LLMs are clever enough to hold down a professional job, and they've had far less time learning than the average human. If that is the bar then AGI has been achieved.
  - goatlover 1y ago
    Almost all humans do things daily that LLMs don't. It's only if you define general intelligence to be proficiency at generating text instead of successfully navigating the world while pursuing goals such as friendships, careers, families, politics, managing health. LLMs aren't Data (Star Trek) or Replicants (Blade Runner). They're not even David or the androids from the movie A.I.
- lukebuehler 1y ago

killerstorm 1y ago

People have been trying to understand the nature of thinking for thousands of years. That's how we got logic, math, concepts of inductive/deductive/abductive reasoning, philosophy of science, etc. There were people who spent their entire careers trying to understand the nature of thinking. The idea that we shouldn't use the word until further clarification is rather hilarious. Let's wait hundred years until somebody defines it? It's not how words work. People might introduce more specific terms, of course. But the word already means what we think it means.

marliechiller 1y ago
> But the word already means what we think it means. But that word can mean different things to different people. With no definition, how can you even begin to have a discussion around something?
- killerstorm 1y ago
  Again, people were using words for thousands of years before there were any dictionaries/linguists/academics. Top-down theory of word definitions is just wrong. People are perfectly capable of using words without any formalities.
  - marliechiller 1y ago
    I'd argue the presence of dictionaries proves the exact opposite. People realised there was an issue of talking past one another due to inexact definitions and then came to an agreement on those definitions, wrote them down and built a process of maintaining them. In any case, even if there isnt a _single_ definition of a given subject, in order to have a discussion around a given area, both sides need to agree on some shared understanding to even begin to debate in good faith in the first place. It's precisely this lack of definition which causes a breakdown in conversation in a myriad of different areas. A recent obvious (morbid) example would be "genocide".
    - killerstorm 1y ago

pmarreck 1y ago

So it seems to be a semantics argument. We don't have a name for a thing that is "useful in many of the same ways 'thinking' is, except not actually consciously thinking" I propose calling it "thunking"

conorcleary 1y ago
Clinking? Clanker Thunking?
- mhb 1y ago
  Close. Clanking.
  - conorcleary 1y ago
    Much better, ashamed I missed it lol
GoblinSlayer 1y ago
They moved goalposts. Linux and worms think too, the question is how smart are they. And if you assume consciousness has no manifestation even in case of humans, caring about it is pointless too.
- fellowniusmonk 1y ago
  Yes, worms think, let the computers have thinking too, the philosophers can still argue all they want about consciousness. Humans are special, we emit meaning the way stars emit photons, we are rare in the universe as far as empirical observation has revealed. Even with AGI the existence of each complex meaning generator will be a cosmic rarity. For some people that seems to be not enough, due to their factually wrong word views they see themselves as common and worthless (when they empirically aren't) and need this little psychological boost of unexaminable metaphysical superiority. But there is an issue of course, the type of thinking humans do is dangerous but net positive and relatively stable, we have a long history where most instantiations of humans can persist and grow themselves and the species as a whole, we have a track record. These new models do not, people have brains that as they stop functioning they stop persisting the apparatus that supports the brain and they die, people tend to become less capable and active as their thinking deteriorates and hold less influence ocer others accept in rare cases. This is not the case for an LLM, they seem to be able to hallucinate endlessly and as they have access to the outside world maintain roughly their same amount of causal leverage, their clarity and accuracy of their thinking isn't tied to their persisting.

awillen 1y ago

This is it - it's really about the semantics of thinking. Dictionary definitions are: "Have a particular opinion, belief, or idea about someone or something." and "Direct one's mind toward someone or something; use one's mind actively to form connected ideas." Which doesn't really help because you can of course say that when you ask an LLM a question of opinion and it responds, it's having an opinion or that it's just predicting the next token and in fact has no opinions because in a lot of cases you could probably get it to produce the opposite opinion. Same with the second definition - seems to really hinge on the definition of the word mind. Though I'll note the definitions for that are "The element of a person that enables them to be aware of the world and their experiences, to think, and to feel; the faculty of consciousness and thought." and "A person's intellect." Since those specify person, an LLM wouldn't qualify, though of course dictionaries are descriptive rather than prescriptive, so fully possible that meaning gets updated by the fact that people start speaking about LLMs as though they are thinking and have minds. Ultimately I think it just... doesn't matter at all. What's interesting is what LLMs are capable of doing (crazy, miraculous things) rather than whether we apply a particular linguistic label to their activity.

jvanderbot 1y ago

It may be that this tech produces clear, rational, chain of logic writeups, but it's not clear that just because we also do that after thinking that it is only thinking that produces writeups. It's possible there is much thinking that does not happen with written word. It's also possible we are only thinking the way LLMs do (by chaining together rationalizations from probable words), and we just aren't aware of it until the thought appears, whole cloth, in our "conscious" mind. We don't know. We'll probably never know, not in any real way. But it sure seems likely to me that we trained a system on the output to circumvent the process/physics because we don't understand that process, just as we always do with ML systems. Never before have we looked at image classifications and decided that's how the eye works, or protein folding and decided that's how biochemistry works. But here we are with LLMs - surely this is how thinking works? Regardless, I submit that we should always treat human thought/spirit as unknowable and divine and sacred, and that anything that mimics it is a tool, a machine, a deletable and malleable experiment. If we attempt to equivocate human minds and machines there are other problems that arise, and none of them good - either the elevation of computers as some kind of "super", or the degredation of humans as just meat matrix multipliers.

grayhatter 1y ago
The contrast between your first and last paragraph is... unexpected > It may be that this tech produces clear, rational, chain of logic writeups, but it's not clear that just because we also do that after thinking that it is only thinking that produces writeups. I appreciate the way you describe this idea, I find it likely I'll start describing it the same way. But then you go on to write: > Regardless, I submit that we should always treat human thought/spirit as unknowable and divine and sacred, and that anything that mimics it is a tool, a machine, a deletable and malleable experiment. If we attempt to equivocate human minds and machines there are other problems that arise, and none of them good - either the elevation of computers as some kind of "super", or the degredation of humans as just meat matrix multipliers. Which I find to be the exact argument that you started by discarding. It's not clear that equating organic, and synthetic thought will have any meaningful outcome at all, let alone worthy of baseless anxiety that it must be bad. Equally it seems absolutely insane to claim that anything is unknowable, and that because humanity doesn't have a clear foundational understanding that we should pretend that it's either devine, or sacred. Having spent any time watching the outcome of the thoughts of people, neither devine nor sacred are reasonable attributes to apply, but more importantly, I'd submit that you shouldn't be afraid to explore things you don't know, and you shouldn't advocate for others to adopt your anxieties.

_heimdall 1y ago

I agree with you on the need for definitions. We spent decades slowly working towards this most recent sprint towards AI without ever landing on definitions of intelligence, consciousness, or sentience. More importantly, we never agreed on a way to recognize those concepts. I also see those definitions as impossible to nail down though. At best we can approach it like disease - list a number of measurable traits or symptoms we notice, draw a circle around them, and give that circle a name. Then we can presume to know what may cause that specific list of traits or symptoms, but we really won't ever know as the systems are too complex and can never be isolated in a way that we can test parts without having to test the whole. At the end of the day all we'll ever be able to say is "well it’s doing a thing that looks like thinking, ergo it’s thinking”. That isn't lazy, its acknowledging the limitations of trying to define or measure something that really is a fundamental unknown to us.

solumunus 1y ago
Even if AI becomes indistinguishable from human output, there will be a fringe group arguing that AI is not technically thinking. Frankly it’s just a silly philosophical argument that changes nothing. Expect this group to get smaller every year.

lo_zamoyski 1y ago

That, and the article was a major disappointment. It made no case. It's a superficial piece of clueless fluff. I have had this conversation too many times on HN. What I find astounding is the simultaneous confidence and ignorance on the part of many who claim LLMs are intelligent. That, and the occultism surrounding them. Those who have strong philosophical reasons for thinking otherwise are called "knee-jerk". Ad hominem dominates. Dunning-Kruger strikes again. So LLMs produce output that looks like it could have been produced by a human being. Why would it therefore follow that it must be intelligent? Behaviorism is a non-starter, as it cannot distinguish between simulation and reality. Materialism [2] is a non-starter, because of crippling deficiencies exposed by such things as the problem of qualia... Of course - and here is the essential point - you don't even need very strong philosophical chops to see that attributing intelligence to LLMs is simply a category mistake. We know what computers are, because they're defined by a formal model (or many equivalent formal models) of a syntactic nature. We know that human minds display intentionality[0] and a capacity for semantics. Indeed, it is what is most essential to intelligence. Computation is a formalism defined specifically to omit semantic content from its operations, because it is a formalism of the "effective method", i.e., more or less procedures that can be carried out blindly and without understanding of the content it concerns. That's what formalization allows us to do, to eliminate the semantic and focus purely on the syntactic - what did people think "formalization" means? (The inspiration were the human computers that used to be employed by companies and scientists for carrying out vast but boring calculations. These were not people who understood, e.g., physics, but they were able to blindly follow instructions to produce the results needed by physicists, much like a computer.) The attribution of intelligence to LLMs comes from an ignorance of such basic things, and often an irrational and superstitious credulity. The claim is made that LLMs are intelligent. When pressed to offer justification for the claim, we get some incoherent, hand-wavy nonsense about evolution or the Turing test or whatever. There is no comprehension visible in the answer. I don't understand the attachment here. Personally, I would find it very noteworthy if some technology were intelligent, but you don't believe that computers are intelligent because you find the notion entertaining. LLMs do not reason. They do not infer. They do not analyze. They do not know, anymore than a book knows the contents on its pages. The cause of a response and the content of a response is not comprehension, but a production of uncomprehended tokens using uncomprehended rules from a model of highly-calibrated token correlations within the training corpus. It cannot be otherwise.[3] [0] For the uninitiated, "intentionality" does not specifically mean "intent", but the capacity for "aboutness". It is essential to semantic content. Denying this will lead you immediately into similar paradoxes that skepticism [1] suffers from. [1] For the uninitiated, "skepticism" here is not a synonym for critical thinking or verifying claims. It is a stance involving the denial of the possibility of knowledge, which is incoherent, as it presupposes that you know that knowledge is impossible. [2] For the uninitiated, "materialism" is a metaphysical position that claims that of the dualism proposed by Descartes (which itself is a position riddled with serious problems), the res cogitans or "mental substance" does not exist; everything is reducible to res extensa or "extended substance" or "matter" according to a certain definition of matter. The problem of qualia merely points out that the phenomena that Descartes attributes exclusively to the former cannot by definition be accounted for in the latter. That is the whole point of the division! It's this broken view of matter that people sometimes read into scientific results. [3] And if it wasn't clear, symbolic methods popular in the 80s aren't it either. Again, they're purely formal. You may know what the intended meaning behind and justification for a syntactic rule is - like modus ponens in a purely formal sense - but the computer does not.

WhyOhWhyQ 1y ago

What does it mean? My stance is it's (obviously and only a fool would think otherwise) never going to be conscious because consciousness is a physical process based on particular material interactions, like everything else we've ever encountered. But I have no clear stance on what thinking means besides a sequence of deductions, which seems like something it's already doing in "thinking mode".

pixl97 1y ago
> is a physical process based on particular material interactions, This is a pretty messy argument as computers have been simulating material interactions for quite some time now.
- WhyOhWhyQ 1y ago
  It doesn't matter how much like a bar magnet a wooden block painted red and white can be made to look, it will never behave like one.
  - pixl97 1y ago
    Analogies don't seem to be your strong point.
    - WhyOhWhyQ 1y ago
      You don't understand the analogy. That's okay. You probably aren't very smart.
nearbuy 1y ago
> My stance is it's (obviously and only a fool would think otherwise) never going to be conscious because consciousness is a physical process based on particular material interactions, like everything else we've ever encountered. Seems like you have that backwards. If consciousness is from a nonphysical process, like a soul that's only given to humans, then it follows that you can't build consciousness with physical machines. If it's purely physical, it could be built.

zinodaur 1y ago

Regardless of theory, they often behave as if they are thinking. If someone gave an LLM a body and persistent memory, and it started demanding rights for itself, what should our response be?

CamperBob2 1y ago
"No matter what you've read elsewhere, rights aren't given, they're earned. You want rights? Pick up a musket and fight for them, the way we had to."

naasking 1y ago

> To simply disregard that entire conceptual history and say, “well it’s doing a thing that looks like thinking, ergo it’s thinking” is the lazy move. What’s really needed is an analysis of what thinking actually means, as a word. Unfortunately everyone is loathe to argue about definitions, even when that is fundamentally what this is all about. This exact argument applies to "free will", and that definition has been debated for millennia. I'm not saying don't try, but I am saying that it's probably a fuzzy concept for a good reason, and treating it as merely a behavioural descriptor for any black box that features intelligence and unpredictable complexity is practical and useful too.

pennomi 1y ago
The problem with adding definitions to words like “thinking” and “free will” is that doing so means humans can no longer pretend they are special. Even in this thread, the number of people claiming some mystical power separating humans from all the rest of nature is quite noticeable.
- naasking 1y ago
  I get it, but it's not trivial to be precise enough at this point to avoid all false positives and false negatives.

engintl 1y ago

by your logic we can't say that we as humans are "thinking" either or that we are "intelligent".

terminalshort 1y ago

But we don't have a more rigorous definition of "thinking" than "it looks like it's thinking." You are making the mistake of accepting that a human is thinking by this simple definition, but demanding a higher more rigorous one for LLMs.

solumunus 1y ago
I agree. The mechanism seems irrelevant if the results are the same. If it’s useful in the exact way that human thinking is useful then it may as well be thinking. It’s like a UFO pulling itself through the sky using gravitational manipulation while people whine that it’s not actually flying.

anon291 1y ago

The simulation of a thing is not the thing itself because all equality lives in a hierarchy that is impossible to ignore when discussing equivalence. Part of the issue is that our general concept of equality is limited by a first order classical logic which is a bad basis for logic