3 ms·

I have two thoughts simultaneously about the anthropomorphisation of these systems: 1. we should do it less, because it distorts our ability to think about the

I have two thoughts simultaneously about the anthropomorphisation of these systems: 1. we should do it less, because it distorts our ability to think about them properly. Calling these processes 'thinking', 'holidays', etc invites the reader to bring along ideas and expectations that aren't justified by what's happening in the system. 2. it's good to keep doing it, because repeated use reduces the specialness or magic that people seem to reserve for our own behavior ("It's not really intelligent/thinking/reasoning/creative") without any justification for that position beyond feelings. I'm leaning towards the second.

E-Reverance 2mo ago
I definitely belong to the latter camp. After LLMs I view everything humans do very systematically and whenever said thing still feels fuzzy I just treat it as having a noise/smoothing term
- Mr_Minderbinder 2mo ago
  Thankfully for me it happened a while ago, after chess was conquered. One of the most amusing (and depressing) things in the past few years of the current “AI era” is people making the same claims and having the same discussions about “intelligence” and “creativity” from ten, twenty or thirty years ago as their field is finally coming under attack by AI; they are finally going through what the competitive board game communities (checkers, Othello, chess, shogi, go etc.) have already come to terms with. There was no single moment but a gradual process over decades as the incursion became greater and greater, forcing a recalibration as previously held positions became indefensible. In my own case it spurred me to educate myself in many areas of mathematics, philosophy and neuroscience in particular where I was ignorant and has undoubtedly shaped my understanding of “intelligence”. When people started claiming LLMs could not produce ideas outside of what they encountered in their training data, I was reminded of an old Chess Life article from the 60s reporting on the first chess computer to play in a tournament at a local chess club. One of the club members remarked that he did not believe that the computer could play moves that were not “put in” the computer in advance. There is an overwhelming sense of “here we go again”.
edg5000 2mo ago
I spent a long time with custom harness design and found the terminology to be a key part of the work, since the concepts are new. I dropped the term "agent" altogether in favour of "thread" for that very reason.
andai 2mo ago
So, what does thinking mean? Is there one thing we mean when we say thinking in humans? Are there several? Is it more of a spectrum? e.g. last year I tried inventing a "System 3"[0], a more rigorous way to approach problem solving, due to repeated painful experiences getting stuck solving problems the wrong way. I didn't get very far, but I definitely want to revisit the idea. [0] Based on System 1 and System 2, i.e. lossy pattern matching vs "actual thinking". Because I found my "actual thinking" was also ~~dogshit~~ frequently insufficient for the problems at hand. So as far as I'm concerned, thinking properly has not even been invented yet. (But I'd love to hear other perspectives!) https://thedecisionlab.com/reference-guide/philosophy/system-1-and-system-2-thinking https://thedecisionlab.com/reference-guide/philosophy/system...
- idiotsecant 2mo ago
  I suppose we are on the cusp of the first thing that can accurately think about thinking, that is have true self-access and introspection, not the post hoc rationalisation we do. That is legitimately a new kind of thinking, and to an entity capable of such a mode we might b prescribed as barely able to think at all
  - andai 2mo ago
    I had a conversation on my frustration with intellectual sparring partners today. The main insight was that I require people who think. That's a separate quality from intelligence or knowledge. The closest thing is "need for cognition" (which is apparently a dimension of personality). From where I'm standing, most people have a strong need to avoid cognition (and the machine described humans as "cognitive misers"!) However need for cognition is apparently a separate dimension from caring what is actually true. (10 hours of motivated reasoning satisfy NFC but not the latter.) And another dimension was, tolerance for ambiguity and lack of closure. Another was Openness, and Intellect, both of which correlated with experiencing confusion as motivating rather than aversive. All of which turned out to be basic prerequisites for thinking clearly about things. Apparently each of those qualities is rare and apparently you need all of them. So yeah, I think you might be right.
andai 2mo ago
Re: anthropomorphism: I frequently hear stories of LLMs [behaving as though they are] feeling self-doubt. I had a similar experience. Asked Claude Code what the weather was and it said, I don't know, I'm just a programmer. Added "You can do anything, believe in yourself!" to CLAUDE.md, suddenly it was able to google the dang weather. lmao
GPerson 2mo ago
Why would a secondary cultural goal take precedence over an apparently accurate description of what’s happening in the system? For one, you’re fighting some irrelevant battle.

E-Reverance 2mo ago

I definitely belong to the latter camp. After LLMs I view everything humans do very systematically and whenever said thing still feels fuzzy I just treat it as having a noise/smoothing term

Mr_Minderbinder 2mo ago
Thankfully for me it happened a while ago, after chess was conquered. One of the most amusing (and depressing) things in the past few years of the current “AI era” is people making the same claims and having the same discussions about “intelligence” and “creativity” from ten, twenty or thirty years ago as their field is finally coming under attack by AI; they are finally going through what the competitive board game communities (checkers, Othello, chess, shogi, go etc.) have already come to terms with. There was no single moment but a gradual process over decades as the incursion became greater and greater, forcing a recalibration as previously held positions became indefensible. In my own case it spurred me to educate myself in many areas of mathematics, philosophy and neuroscience in particular where I was ignorant and has undoubtedly shaped my understanding of “intelligence”. When people started claiming LLMs could not produce ideas outside of what they encountered in their training data, I was reminded of an old Chess Life article from the 60s reporting on the first chess computer to play in a tournament at a local chess club. One of the club members remarked that he did not believe that the computer could play moves that were not “put in” the computer in advance. There is an overwhelming sense of “here we go again”.

edg5000 2mo ago

I spent a long time with custom harness design and found the terminology to be a key part of the work, since the concepts are new. I dropped the term "agent" altogether in favour of "thread" for that very reason.

andai 2mo ago

So, what does thinking mean? Is there one thing we mean when we say thinking in humans? Are there several? Is it more of a spectrum? e.g. last year I tried inventing a "System 3"[0], a more rigorous way to approach problem solving, due to repeated painful experiences getting stuck solving problems the wrong way. I didn't get very far, but I definitely want to revisit the idea. [0] Based on System 1 and System 2, i.e. lossy pattern matching vs "actual thinking". Because I found my "actual thinking" was also ~~dogshit~~ frequently insufficient for the problems at hand. So as far as I'm concerned, thinking properly has not even been invented yet. (But I'd love to hear other perspectives!) https://thedecisionlab.com/reference-guide/philosophy/system-1-and-system-2-thinking https://thedecisionlab.com/reference-guide/philosophy/system...

idiotsecant 2mo ago
I suppose we are on the cusp of the first thing that can accurately think about thinking, that is have true self-access and introspection, not the post hoc rationalisation we do. That is legitimately a new kind of thinking, and to an entity capable of such a mode we might b prescribed as barely able to think at all
- andai 2mo ago
  I had a conversation on my frustration with intellectual sparring partners today. The main insight was that I require people who think. That's a separate quality from intelligence or knowledge. The closest thing is "need for cognition" (which is apparently a dimension of personality). From where I'm standing, most people have a strong need to avoid cognition (and the machine described humans as "cognitive misers"!) However need for cognition is apparently a separate dimension from caring what is actually true. (10 hours of motivated reasoning satisfy NFC but not the latter.) And another dimension was, tolerance for ambiguity and lack of closure. Another was Openness, and Intellect, both of which correlated with experiencing confusion as motivating rather than aversive. All of which turned out to be basic prerequisites for thinking clearly about things. Apparently each of those qualities is rare and apparently you need all of them. So yeah, I think you might be right.

andai 2mo ago

Re: anthropomorphism: I frequently hear stories of LLMs [behaving as though they are] feeling self-doubt. I had a similar experience. Asked Claude Code what the weather was and it said, I don't know, I'm just a programmer. Added "You can do anything, believe in yourself!" to CLAUDE.md, suddenly it was able to google the dang weather. lmao

GPerson 2mo ago

Why would a secondary cultural goal take precedence over an apparently accurate description of what’s happening in the system? For one, you’re fighting some irrelevant battle.