3 ms·

"The Illusion of Thinking" is bad, but so is the opposite "it's just math and code". They might not be reasoning like humans, but they are not reducible to just

"The Illusion of Thinking" is bad, but so is the opposite "it's just math and code". They might not be reasoning like humans, but they are not reducible to just code and math either. They do something new, something that just math and code did not do before.

jmsdnns 1y ago
> just code and math either this is literally what they are
- digbybk 1y ago
  I think it’s the “just” that they are taking issue with. We are “just” neurons. But we demonstrate interesting emergent behaviors that, in principle, can be reduced to firing neurons but in practice we don’t understand and shouldn’t diminish with the word “just”.
  - jmsdnns 1y ago
    fair point!
- visarga 1y ago
  To insist it's "just code and math" is to fundamentally mistake the rules for the game. Knowing the rules of Go without playing grants no insight into strategy, just as showing someone the code for Conway's Game of Life without execution reveals nothing about the gliders that emerge. The profound dynamics of a system are not found in its static instruction set. To ignore this is a fundamental error, because these are recursive processes that iteratively rewrite their own context, a reality the "it's just X" argument completely fails to grasp. Of course Turing undecidability is the canonical example - can't predict if a program will halt without execution.
cantor_S_drug 1y ago
Here's something to think about: Can we consider AI conscious similar to how Hardy recognised Ramanujan's genius? That if AI weren't to be conscious then they won't have the imagination to write what it wrote.
- mkl 1y ago
  It doesn't take conciousness to predict the next word repeatedly.
  - fc417fc802 1y ago
    That's reductive. Cherry pick the most lucid LLM output and the least lucid human output and I think at this point the former clearly exceeds the latter. If an LLM is "just" predicting the next word repeatedly then what does that say about humans?
    - tsimionescu 1y ago
      Nothing, since human output is adapted (better or worse) to the human's goals in a particular situation, while an LLM produces text that seems like a logical continuation of the text they are given as input. There is a fundamental difference in these two approaches that makes lessons from one very un-applicable from one to the other.
      - cantor_S_drug 1y ago
        Have you noticed how predictable humans are when they have Transient Global Amensia? https://www.radiolab.org/podcast/161744-loops/transcript https://www.radiolab.org/podcast/161744-loops/transcript *The Prisoner of Memory* In August 2010, Mary Sue Campbell became trapped in the most peculiar prison—a 90-second loop of her own consciousness. After experiencing transient global amnesia, she lost the ability to form new memories, resetting every minute and a half like a human record player stuck on repeat. What made her condition haunting wasn't just the forgetting, but the eerie predictability. Every cycle, she asked the same questions in identical phrasing: "What's the date?" followed by "My birthday's already passed? Darn." She'd listen to her daughter's explanation, widen her eyes at the mention of paramedics, and declare "This is creepy!" The nurses began mouthing her words along with her. Dr. Jonathan Vlahos, who treated similar cases, found this mechanical repetition deeply unsettling. It suggested that beneath our illusion of free will lies something more algorithmic—that given identical inputs, the brain produces identical outputs. Mary Sue's loop revealed the uncomfortable truth that our sense of spontaneous choice might be far more fragile than we imagine.
safety1st 1y ago
I don't really agree. I use multiple LLMs every day and I feel like I get the most mileage out of them when I think about them as exactly that. Super good text transformers that can paraphrase anything that's been posted to the Internet. There are a complexities beyond that of course. It can compare stuff, it can iterate through multiple steps ("Reason," though when you look at what it's doing, you definitely see how that term is a bit of a stretch). Lots of emergent applications yet to be discovered. But yeah, it's math (probabilities to generate the next token) and code (go run stuff). The best applications will likely not be the ones that anthropomorphize it, but take advantage of what it really is.
- XenophileJKO 1y ago
  Have you used them to code? I think that is the tipping point where the level of interaction is so high, when the models are troubleshooting. That is when you can start to get a little philosophical. The models make many mistakes and are not great software architects, but watch one or even two models work together to solve a bug and you'll quickly rethink "text transformers".
  - safety1st 1y ago
    All the time. And what stands out is that when I hit a problem in a framework that's new and badly documented, they tend to fall apart. When there's a lot of documentation, StackOverflow discussions etc. for years about how something works they do an excellent job of digging those insights up. So it fits within my mental model of "find stuff on the web and run a text transformer" pretty well. I don't mean to underestimate the capabilities, but I don't think we need philosophy to explain them either. If we ever develop a LLM that's able to apply symbolic logic to text, for instance to assess an argument's validity, develop an accurate proof step by step, and do this at least as well as many human beings, then I'll concede that we've invented a reasoning machine. Such a machine might very well be a miraculous invention in this age of misinformation, but I know of no work in that direction and I'm not at all convinced it's a natural outgrowth of LLMs which are so bad at math (perhaps they'd be of use in the implementation).
pepinator 1y ago
We call it breakthrough. And it's just math and code. We've had many of those.
jstanley 1y ago
What part of an LLM do you think is not made out of code and maths?
- victorbjorklund 1y ago
  I think the point is we wouldnt say "humans dont reason. It is just chemical and eletrical signals"
  - jstanley 1y ago
    Right. The error is in thinking that "just code and maths" can't reason. LLMs very obviously are reducible to "just code and maths". We know that, because that is how they are made.

jmsdnns 1y ago

> just code and math either this is literally what they are

digbybk 1y ago
I think it’s the “just” that they are taking issue with. We are “just” neurons. But we demonstrate interesting emergent behaviors that, in principle, can be reduced to firing neurons but in practice we don’t understand and shouldn’t diminish with the word “just”.
- jmsdnns 1y ago
  fair point!
visarga 1y ago
To insist it's "just code and math" is to fundamentally mistake the rules for the game. Knowing the rules of Go without playing grants no insight into strategy, just as showing someone the code for Conway's Game of Life without execution reveals nothing about the gliders that emerge. The profound dynamics of a system are not found in its static instruction set. To ignore this is a fundamental error, because these are recursive processes that iteratively rewrite their own context, a reality the "it's just X" argument completely fails to grasp. Of course Turing undecidability is the canonical example - can't predict if a program will halt without execution.

cantor_S_drug 1y ago

Here's something to think about: Can we consider AI conscious similar to how Hardy recognised Ramanujan's genius? That if AI weren't to be conscious then they won't have the imagination to write what it wrote.

mkl 1y ago
It doesn't take conciousness to predict the next word repeatedly.
- fc417fc802 1y ago
  That's reductive. Cherry pick the most lucid LLM output and the least lucid human output and I think at this point the former clearly exceeds the latter. If an LLM is "just" predicting the next word repeatedly then what does that say about humans?
  - tsimionescu 1y ago
    Nothing, since human output is adapted (better or worse) to the human's goals in a particular situation, while an LLM produces text that seems like a logical continuation of the text they are given as input. There is a fundamental difference in these two approaches that makes lessons from one very un-applicable from one to the other.
    - cantor_S_drug 1y ago
      Have you noticed how predictable humans are when they have Transient Global Amensia? https://www.radiolab.org/podcast/161744-loops/transcript https://www.radiolab.org/podcast/161744-loops/transcript *The Prisoner of Memory* In August 2010, Mary Sue Campbell became trapped in the most peculiar prison—a 90-second loop of her own consciousness. After experiencing transient global amnesia, she lost the ability to form new memories, resetting every minute and a half like a human record player stuck on repeat. What made her condition haunting wasn't just the forgetting, but the eerie predictability. Every cycle, she asked the same questions in identical phrasing: "What's the date?" followed by "My birthday's already passed? Darn." She'd listen to her daughter's explanation, widen her eyes at the mention of paramedics, and declare "This is creepy!" The nurses began mouthing her words along with her. Dr. Jonathan Vlahos, who treated similar cases, found this mechanical repetition deeply unsettling. It suggested that beneath our illusion of free will lies something more algorithmic—that given identical inputs, the brain produces identical outputs. Mary Sue's loop revealed the uncomfortable truth that our sense of spontaneous choice might be far more fragile than we imagine.

safety1st 1y ago

I don't really agree. I use multiple LLMs every day and I feel like I get the most mileage out of them when I think about them as exactly that. Super good text transformers that can paraphrase anything that's been posted to the Internet. There are a complexities beyond that of course. It can compare stuff, it can iterate through multiple steps ("Reason," though when you look at what it's doing, you definitely see how that term is a bit of a stretch). Lots of emergent applications yet to be discovered. But yeah, it's math (probabilities to generate the next token) and code (go run stuff). The best applications will likely not be the ones that anthropomorphize it, but take advantage of what it really is.

XenophileJKO 1y ago
Have you used them to code? I think that is the tipping point where the level of interaction is so high, when the models are troubleshooting. That is when you can start to get a little philosophical. The models make many mistakes and are not great software architects, but watch one or even two models work together to solve a bug and you'll quickly rethink "text transformers".
- safety1st 1y ago
  All the time. And what stands out is that when I hit a problem in a framework that's new and badly documented, they tend to fall apart. When there's a lot of documentation, StackOverflow discussions etc. for years about how something works they do an excellent job of digging those insights up. So it fits within my mental model of "find stuff on the web and run a text transformer" pretty well. I don't mean to underestimate the capabilities, but I don't think we need philosophy to explain them either. If we ever develop a LLM that's able to apply symbolic logic to text, for instance to assess an argument's validity, develop an accurate proof step by step, and do this at least as well as many human beings, then I'll concede that we've invented a reasoning machine. Such a machine might very well be a miraculous invention in this age of misinformation, but I know of no work in that direction and I'm not at all convinced it's a natural outgrowth of LLMs which are so bad at math (perhaps they'd be of use in the implementation).

pepinator 1y ago

We call it breakthrough. And it's just math and code. We've had many of those.

jstanley 1y ago

What part of an LLM do you think is not made out of code and maths?

victorbjorklund 1y ago
I think the point is we wouldnt say "humans dont reason. It is just chemical and eletrical signals"
- jstanley 1y ago
  Right. The error is in thinking that "just code and maths" can't reason. LLMs very obviously are reducible to "just code and maths". We know that, because that is how they are made.