7 ms·

of course! but maybe there is something that you have to experience, before you can understand it.

TeMPOraL 8mo ago
Sure! But if I experience it, and then write about my experience, parts of it become available for LLMs to learn from. Beyond that, even the tacit aspects of that experience, the things that can't be put down in writing, will still leave an imprint on anything I do and write from that point on. Those patterns may be more or less subtle, but they are there, and could be picked up at scale. I believe LLM training is happening at a scale great enough for models to start picking up on those patterns. Whether or not this can ever be equivalent to living through the experience personally, or at least asymptomatically approach it, I don't know. At the limit, this is basically asking about the nature of qualia. What I do believe is that continued development of LLMs and similar general-purpose AI systems will shed a lot of light on this topic, and eventually help answer many of the long-standing questions about the nature of conscious experience.
- fc417fc802 8mo ago
  > will shed a lot of light on this topic, and eventually help answer I dunno. I figure it's more likely we keep emulating behaviors without actually gaining any insight into the relevant philosophical questions. I mean what has learning that a supposed stochastic parrot is capable of interacting at the skill levels presently displayed actually taught us about any of the abstract questions?
  - TeMPOraL 8mo ago
    > I mean what has learning that a supposed stochastic parrot is capable of interacting at the skill levels presently displayed actually taught us about any of the abstract questions? IMHO a lot. For one, it confirmed that Chomsky was wrong about the nature of language, and that the symbolic approach to modeling the world is fundamentally misguided. It confirmed the intuition I developed of the years of thinking deeply about these problems[0], that the meaning of words and concepts is not an intrinsic property, but is derived entirely from relationships between concepts. The way this is confirmed, is because the LLM as a computational artifact is a reification of meaning, a data structure that maps token sequences to points in a stupidly high-dimensional space, encoding semantics through spatial adjacency. We knew for many years that high-dimensional spaces are weird and surprisingly good at encoding semi-dependent information, but knowing the theory is one thing, seeing an actual implementation casually pass the Turing test and threaten to upend all white-collar work, is another thing. -- I realize my perspective - particularly my belief that this informs the study of human mind in any way - might look to some as making some unfounded assumptions or leaps in logic, so let me spell out two insights that makes me believe LLMs and human brains share fundamentals: 1) The general optimization function of LLM training is "produce output that makes sense to humans, in fully general meaning of that statement". We're not training these models to be good at specific skills, but to always respond to any arbitrary input - even beyond natural language - in a way we consider reasonable. I.e. we're effectively brute-forcing a bag of floats into emulating the human mind. Now that alone doesn't guarantee the outcome will be anything like our minds, but consider the second insight: 2) Evolution is a dumb, greedy optimizer. Complex biology, including animal and human brains, evolved incrementally - and most importantly, every step taken had to provide a net fitness advantage[1], or else it would've been selected out[2]. From this follows that the basic principles that make a human mind work - including all intelligence and learning capabilities we have - must be fundamentally simple enough that a dumb, blind, greedy random optimizer can grope its way to them in incremental steps in relatively short time span[3]. 2.1) Corollary: our brains are basically the dumbest possible solution evolution could find that can host general intelligence. It didn't have time to iterate on the brain design further, before human technological civilization took off in the blink of an eye. So, my thinking basically is: 2) implies that the fundamentals behind human cognition are easily reachable in space of possible mind designs, so if process described in 1) is going to lead towards a working general intelligence, there's a good chance it'll stumble on the same architecture evolution did. -- [0] - I imagine there are multiple branches of philosophy, linguistics and cognitive sciences that studied this perspective in detail, but unfortunately I don't know what they are. [1] - At the point of being taken. Over time, a particular characteristic can become a fitness drag, but persist indefinitely as long as more recent evolutionary steps provide enough advantage that on the net, the fitness increases. So it's possible for evolution to accumulate building blocks that may become useful again later, but only if they were also useful initially. [2] - Also on average, law of big numbers, yadda yadda. It's fortunate that life started with lots of tiny things with very short life spans. [3] - It took evolution some 3 billion years to get from bacteria to first multi-cellular life, some extra 60 million years to develop a nervous system and eventually a kind of proto-brain, and then an extra 500 million years iterating on it to arrive at a human brain.
    - pegasus 8mo ago
      Didn't read the whole wall of text/slop, but noticed how the first note (referred from "the intuition I developed of the years of thinking deeply about these problems[0]") is nonsensical in the context. If this is reply is indeed AI-generated, it hilariously self-disproves itself this way. I would congratulate you for the irony, but I have a feeling this is not intentional.
      - TeMPOraL 8mo ago
        Not a single bit of it is AI generated, but I've noticed for years now that LLMs have a similar writing style to my own. Not sure what to do about it.
        sfink 8mo ago
        I'd like to congratulate you on writing a wall of text that gave off all the signals of being written by a conspiracy theorist or crank or someone off their meds, yet also such that when I bothered to read it, I found it to be completely level-headed. Nothing you claimed felt the least bit outrageous to me. I actually only read it because it looked like it was going to be deliciously unhinged ravings.
        strogonoff 8mo ago
        “The meaning of words and concepts is derived entirely from relationships between concepts” would be a pretty outrageous statement to me. The meaning of words is derived from our experience of reality. Words is how the experiencing self classifies experienced reality into a lossy shared map for the purposes of communication with other similarly experiencing selves, and without that shared experience words are meaningless, no matter what graph you put them in.
        TeMPOraL 8mo ago
        > The meaning of words is derived from our experience of reality. I didn't say "words". I said "concepts"[0]. > Words is how the experiencing self classifies experienced reality into a lossy shared map for the purposes of communication with other similarly experiencing selves, and without that shared experience words are meaningless, no matter what graph you put them in. Sure, ultimately everything is grounded in some experiences. But I'm not talking about grounding, I'm talking about the mental structures we build on top of those. The kind of higher-level, more abstract thinking (logical or otherwise) we do, is done in terms of those structures, not underlying experiences. Also: you can see what I mean by "meaning being defined in terms of relationships" if you pick anything, any concept - "a tree", "blue sky", "a chair", "eigenvector", "love", anything - and try to fully define what it means. You'll find the only way you can do it is by relating it to some other concepts, which themselves can only be defined by relating them to other concepts. It's not an infinite regression, eventually you'll reach some kind of empirical experience that can be used as anchor - but still, most of your effort will be spent drawing boundaries in concept space. -- [0] - And WRT. LLMs, tokens are not words either; if that wasn't obvious 2 years ago, it should be today, now that multimodal LLMs are commonplace. The fact that this - tokenizing video and audio and other modalities into the same class of tokens as text, and embedding them in the same latent space - worked spectacularly well - is pretty informative to me. For one, it's a much better framework to discuss the paradox of Sapir-Whorf hypotheses than whatever was mentioned on Wikipedia to date
- testaccount28 8mo ago
  whereof one cannot speak, thereof one must remain silent.

Sure! But if I experience it, and then write about my experience, parts of it become available for LLMs to learn from. Beyond that, even the tacit aspects of that experience, the things that can't be put down in writing, will still leave an imprint on anything I do and write from that point on. Those patterns may be more or less subtle, but they are there, and could be picked up at scale. I believe LLM training is happening at a scale great enough for models to start picking up on those patterns. Whether or not this can ever be equivalent to living through the experience personally, or at least asymptomatically approach it, I don't know. At the limit, this is basically asking about the nature of qualia. What I do believe is that continued development of LLMs and similar general-purpose AI systems will shed a lot of light on this topic, and eventually help answer many of the long-standing questions about the nature of conscious experience.

fc417fc802 8mo ago
> will shed a lot of light on this topic, and eventually help answer I dunno. I figure it's more likely we keep emulating behaviors without actually gaining any insight into the relevant philosophical questions. I mean what has learning that a supposed stochastic parrot is capable of interacting at the skill levels presently displayed actually taught us about any of the abstract questions?
- TeMPOraL 8mo ago
  > I mean what has learning that a supposed stochastic parrot is capable of interacting at the skill levels presently displayed actually taught us about any of the abstract questions? IMHO a lot. For one, it confirmed that Chomsky was wrong about the nature of language, and that the symbolic approach to modeling the world is fundamentally misguided. It confirmed the intuition I developed of the years of thinking deeply about these problems[0], that the meaning of words and concepts is not an intrinsic property, but is derived entirely from relationships between concepts. The way this is confirmed, is because the LLM as a computational artifact is a reification of meaning, a data structure that maps token sequences to points in a stupidly high-dimensional space, encoding semantics through spatial adjacency. We knew for many years that high-dimensional spaces are weird and surprisingly good at encoding semi-dependent information, but knowing the theory is one thing, seeing an actual implementation casually pass the Turing test and threaten to upend all white-collar work, is another thing. -- I realize my perspective - particularly my belief that this informs the study of human mind in any way - might look to some as making some unfounded assumptions or leaps in logic, so let me spell out two insights that makes me believe LLMs and human brains share fundamentals: 1) The general optimization function of LLM training is "produce output that makes sense to humans, in fully general meaning of that statement". We're not training these models to be good at specific skills, but to always respond to any arbitrary input - even beyond natural language - in a way we consider reasonable. I.e. we're effectively brute-forcing a bag of floats into emulating the human mind. Now that alone doesn't guarantee the outcome will be anything like our minds, but consider the second insight: 2) Evolution is a dumb, greedy optimizer. Complex biology, including animal and human brains, evolved incrementally - and most importantly, every step taken had to provide a net fitness advantage[1], or else it would've been selected out[2]. From this follows that the basic principles that make a human mind work - including all intelligence and learning capabilities we have - must be fundamentally simple enough that a dumb, blind, greedy random optimizer can grope its way to them in incremental steps in relatively short time span[3]. 2.1) Corollary: our brains are basically the dumbest possible solution evolution could find that can host general intelligence. It didn't have time to iterate on the brain design further, before human technological civilization took off in the blink of an eye. So, my thinking basically is: 2) implies that the fundamentals behind human cognition are easily reachable in space of possible mind designs, so if process described in 1) is going to lead towards a working general intelligence, there's a good chance it'll stumble on the same architecture evolution did. -- [0] - I imagine there are multiple branches of philosophy, linguistics and cognitive sciences that studied this perspective in detail, but unfortunately I don't know what they are. [1] - At the point of being taken. Over time, a particular characteristic can become a fitness drag, but persist indefinitely as long as more recent evolutionary steps provide enough advantage that on the net, the fitness increases. So it's possible for evolution to accumulate building blocks that may become useful again later, but only if they were also useful initially. [2] - Also on average, law of big numbers, yadda yadda. It's fortunate that life started with lots of tiny things with very short life spans. [3] - It took evolution some 3 billion years to get from bacteria to first multi-cellular life, some extra 60 million years to develop a nervous system and eventually a kind of proto-brain, and then an extra 500 million years iterating on it to arrive at a human brain.
  - pegasus 8mo ago
    Didn't read the whole wall of text/slop, but noticed how the first note (referred from "the intuition I developed of the years of thinking deeply about these problems[0]") is nonsensical in the context. If this is reply is indeed AI-generated, it hilariously self-disproves itself this way. I would congratulate you for the irony, but I have a feeling this is not intentional.
    - TeMPOraL 8mo ago
      Not a single bit of it is AI generated, but I've noticed for years now that LLMs have a similar writing style to my own. Not sure what to do about it.
      - sfink 8mo ago
        I'd like to congratulate you on writing a wall of text that gave off all the signals of being written by a conspiracy theorist or crank or someone off their meds, yet also such that when I bothered to read it, I found it to be completely level-headed. Nothing you claimed felt the least bit outrageous to me. I actually only read it because it looked like it was going to be deliciously unhinged ravings.
        strogonoff 8mo ago
        “The meaning of words and concepts is derived entirely from relationships between concepts” would be a pretty outrageous statement to me. The meaning of words is derived from our experience of reality. Words is how the experiencing self classifies experienced reality into a lossy shared map for the purposes of communication with other similarly experiencing selves, and without that shared experience words are meaningless, no matter what graph you put them in.
        TeMPOraL 8mo ago
        > The meaning of words is derived from our experience of reality. I didn't say "words". I said "concepts"[0]. > Words is how the experiencing self classifies experienced reality into a lossy shared map for the purposes of communication with other similarly experiencing selves, and without that shared experience words are meaningless, no matter what graph you put them in. Sure, ultimately everything is grounded in some experiences. But I'm not talking about grounding, I'm talking about the mental structures we build on top of those. The kind of higher-level, more abstract thinking (logical or otherwise) we do, is done in terms of those structures, not underlying experiences. Also: you can see what I mean by "meaning being defined in terms of relationships" if you pick anything, any concept - "a tree", "blue sky", "a chair", "eigenvector", "love", anything - and try to fully define what it means. You'll find the only way you can do it is by relating it to some other concepts, which themselves can only be defined by relating them to other concepts. It's not an infinite regression, eventually you'll reach some kind of empirical experience that can be used as anchor - but still, most of your effort will be spent drawing boundaries in concept space. -- [0] - And WRT. LLMs, tokens are not words either; if that wasn't obvious 2 years ago, it should be today, now that multimodal LLMs are commonplace. The fact that this - tokenizing video and audio and other modalities into the same class of tokens as text, and embedding them in the same latent space - worked spectacularly well - is pretty informative to me. For one, it's a much better framework to discuss the paradox of Sapir-Whorf hypotheses than whatever was mentioned on Wikipedia to date
testaccount28 8mo ago
whereof one cannot speak, thereof one must remain silent.