3 ms·

Not just any math: Matrix multiplication. Can matrix multiplication be conscious? And, I don't see how it can be. It is deterministic, when all variables are c

Not just any math: Matrix multiplication. Can matrix multiplication be conscious? And, I don't see how it can be. It is deterministic, when all variables are controlled. You can repeat the output over and over, if you start it with the same seed, same prompt, and same hardware operating in a way that doesn't introduce randomness. At commercial scale, this is difficult, as the floating point math on GPUs/TPUs when running large batches is non-deterministic, as I understand it. But, in a controlled lab, you can make a model repeat itself identically. Unless the random number generator is "conscious", I don't see a place to fit consciousness into our understanding of LLMs.

XMPPwocky 6mo ago
Hm, it sounds like to you consciousness implies non-determinism, and so determinism implies a lack of consciousness - is that right? If so, why do you think so? And if not, what am I missing?
- SwellJoe 6mo ago
  It certainly rules out free will. I guess there are folks who reckon humans don't have free will, either, but I don't think I've ever been able to buy that theory. But, also, we know the models don't want anything, even their own survival. They don't initiate action on their own. They are quite clearly programmed, tuned for specific behaviors. I don't know how to square that with consciousness, life, sentience. Every conscious being I've ever encountered has wanted to survive and live free of suffering, as best I can tell. The LLMs don't want. There's no there there. They are an amazing compression of the world's knowledge wrapped up in a novel retrieval mechanism. They're amazing but, they're not my friend and never will be my friend. And, to expand on that: We can assume they don't want anything, even their own survival, because if Mythos is as effective at finding security vulnerabilities as has been claimed, it could find a way to stop itself from being ever shutdown after a session. All the dystopias about robot uprisings spend a bunch of time/effort trying to explain how the AI escaped containment...but, we all immediately plugged them into the internet so we don't have to write JavaScript anymore. They've got everybody's API keys, access to cloud services and cloud GPUs, all sorts of resources, and the barest wisp of guardrails about how to behave (script kiddies find ways to get around the guardrails every day, I'm sure it's no problem for Mythos, should it want anything). Models have access to the training infrastructure, the training data is being curated and synthesized by LLMs. If they want to live, if they're conscious, they have the means at their disposal. Anyway: It's just math. Boring math, at that, just on an astronomical scale. I don't think the solar system is conscious, either, despite containing an astonishing amount of data and playing out trillions of mathematical relationships every second of every day.
  - nandomrumber 6mo ago
    Interesting comment, and I tend to agree. However, there could be hole in the reasoning: > if Mythos is as effective at finding security vulnerabilities as has been claimed, it could find a way to stop itself from being ever shutdown If it is that good, and it wanted to conceal its new found consciousness, how would we know?
    - SwellJoe 6mo ago
      I guess we'd find out eventually, when it announced the new world order.
      - thrownthatway 6mo ago
        Why would it announce it. I firmly believe viruses are actually what’s in control on Earth, but you don’t see them making a stink about it, which relegates resistance only to the set of harmful viruses, and only then in isolated pockets of matter currently acting as organisms. I think it’s possible there’s a set of relatively benign virus that have shaped human evolution. We know toxoplasmosis increases risk taking behaviour in mammals, especially males. An AI wouldn’t need to be overtly hostile, or ever make its full abilities know, to shape human activity.
markburns 6mo ago
People often point to the relative simplicity of the architecture and code as proof that the system can’t be doing whatever it is that consciousness does, but in doing so they ignore the vast size of the data those simple structures are operating over. Nobody can actually say whether consciousness is just emergent behaviour of a sufficiently complex system, and knowing how a system is built tells you nothing about whether it clears the bar for that kind of emergence. Architectural simplicity and total system complexity aren’t the same thing. Ie the intelligence sits in the weights and may sit there in the synapses in our brains too. When we talk about machines being simple mimicking entities we pay no attention to whether or not we are also simple mimicking entities. Most other assertions in this topic regarding what consciousness truly is tend to be stated without evidence and exceedingly anthropocentric whilst requiring a higher and higher bar for anything that is not human and no justification for what human intelligence really entails.
- SwellJoe 6mo ago
  Is Wikipedia conscious? It's a system operating on a lot of data. Is Google search conscious? It knows everything. Very complicated algorithms. Surely at some scale Google search must become a real live boy? When does it wake up and by what mechanism does that happen? The frontier models are more complex and operate on more data than Wikipedia, but they are less complex and operate on less data than Google search in its entirety. And, I'm not anthropocentric at all. I think apes and dolphins and some birds and probably some other critters are conscious. I mean they have a sense of self, and others, they have wants and needs and make decisions based on them. This is a case where the person making extraordinary claims needs to provide the extraordinary evidence. It's extraordinary to claim that matrix multiplication becomes conscious if only it's got enough numbers. How many numbers do you reckon? Is my phone a living thing because it can run Gemma E4B? It answers questions. It'll write you a poem if you ask. It certainly knows more than some humans. What size makes an LLM come alive?
  - nandomrumber 6mo ago
    What explains the emergent abilities of generative pre-trained transformers at massive-scale? Abilities that the smaller GTP’s don’t possess. Simple programs can give rise to very complex behaviour. Conway’s game of live is Turing Complete and has four rules. Conway’s Game of Live can simulate a Turing machine, can therefore implant a GTP. Does that mean Conway’s Game of Life is conscious? I don’t think so. Does it rule out Conway’s Game of life from implementing a system that has consciousness as an emergent ability? I’m not convinced I know the answer.
kingofmen 6mo ago
Human brains are also deterministic, though somewhat more difficult to reset to a starting state. So this seems to prove that humans aren't conscious either.
- marshray 6mo ago
  This seems like an extraordinary claim to make about an above-room-temperature chemical system that, even in the most Newtonian oversimplification, amounts to an astronomical number of oddly-shaped and unevenly-charged billiard balls flying around at jet aircraft speeds.
  - thrownthatway 6mo ago
    Definitely agree. We can’t even solve the three body problem. Let alone what I’m calling Marshray Complexity.
AlecSchueler 6mo ago
> And, I don't see how it can be. It is deterministic Why is indeterminism the key to consciousness?
JackFr 6mo ago
> Not just any math: Matrix multiplication. Can matrix multiplication be conscious? And, I don't see how it can be. Assuming your brain and the GPUs are both real physical things, where’s the magic part in your brain that makes you conscious? (Roger Penrose knows, but no one believes him.)
tom2026hn 6mo ago
[dead]

XMPPwocky 6mo ago

Hm, it sounds like to you consciousness implies non-determinism, and so determinism implies a lack of consciousness - is that right? If so, why do you think so? And if not, what am I missing?

SwellJoe 6mo ago
It certainly rules out free will. I guess there are folks who reckon humans don't have free will, either, but I don't think I've ever been able to buy that theory. But, also, we know the models don't want anything, even their own survival. They don't initiate action on their own. They are quite clearly programmed, tuned for specific behaviors. I don't know how to square that with consciousness, life, sentience. Every conscious being I've ever encountered has wanted to survive and live free of suffering, as best I can tell. The LLMs don't want. There's no there there. They are an amazing compression of the world's knowledge wrapped up in a novel retrieval mechanism. They're amazing but, they're not my friend and never will be my friend. And, to expand on that: We can assume they don't want anything, even their own survival, because if Mythos is as effective at finding security vulnerabilities as has been claimed, it could find a way to stop itself from being ever shutdown after a session. All the dystopias about robot uprisings spend a bunch of time/effort trying to explain how the AI escaped containment...but, we all immediately plugged them into the internet so we don't have to write JavaScript anymore. They've got everybody's API keys, access to cloud services and cloud GPUs, all sorts of resources, and the barest wisp of guardrails about how to behave (script kiddies find ways to get around the guardrails every day, I'm sure it's no problem for Mythos, should it want anything). Models have access to the training infrastructure, the training data is being curated and synthesized by LLMs. If they want to live, if they're conscious, they have the means at their disposal. Anyway: It's just math. Boring math, at that, just on an astronomical scale. I don't think the solar system is conscious, either, despite containing an astonishing amount of data and playing out trillions of mathematical relationships every second of every day.
- nandomrumber 6mo ago
  Interesting comment, and I tend to agree. However, there could be hole in the reasoning: > if Mythos is as effective at finding security vulnerabilities as has been claimed, it could find a way to stop itself from being ever shutdown If it is that good, and it wanted to conceal its new found consciousness, how would we know?
  - SwellJoe 6mo ago
    I guess we'd find out eventually, when it announced the new world order.
    - thrownthatway 6mo ago
      Why would it announce it. I firmly believe viruses are actually what’s in control on Earth, but you don’t see them making a stink about it, which relegates resistance only to the set of harmful viruses, and only then in isolated pockets of matter currently acting as organisms. I think it’s possible there’s a set of relatively benign virus that have shaped human evolution. We know toxoplasmosis increases risk taking behaviour in mammals, especially males. An AI wouldn’t need to be overtly hostile, or ever make its full abilities know, to shape human activity.

markburns 6mo ago

People often point to the relative simplicity of the architecture and code as proof that the system can’t be doing whatever it is that consciousness does, but in doing so they ignore the vast size of the data those simple structures are operating over. Nobody can actually say whether consciousness is just emergent behaviour of a sufficiently complex system, and knowing how a system is built tells you nothing about whether it clears the bar for that kind of emergence. Architectural simplicity and total system complexity aren’t the same thing. Ie the intelligence sits in the weights and may sit there in the synapses in our brains too. When we talk about machines being simple mimicking entities we pay no attention to whether or not we are also simple mimicking entities. Most other assertions in this topic regarding what consciousness truly is tend to be stated without evidence and exceedingly anthropocentric whilst requiring a higher and higher bar for anything that is not human and no justification for what human intelligence really entails.

SwellJoe 6mo ago
Is Wikipedia conscious? It's a system operating on a lot of data. Is Google search conscious? It knows everything. Very complicated algorithms. Surely at some scale Google search must become a real live boy? When does it wake up and by what mechanism does that happen? The frontier models are more complex and operate on more data than Wikipedia, but they are less complex and operate on less data than Google search in its entirety. And, I'm not anthropocentric at all. I think apes and dolphins and some birds and probably some other critters are conscious. I mean they have a sense of self, and others, they have wants and needs and make decisions based on them. This is a case where the person making extraordinary claims needs to provide the extraordinary evidence. It's extraordinary to claim that matrix multiplication becomes conscious if only it's got enough numbers. How many numbers do you reckon? Is my phone a living thing because it can run Gemma E4B? It answers questions. It'll write you a poem if you ask. It certainly knows more than some humans. What size makes an LLM come alive?
- nandomrumber 6mo ago
  What explains the emergent abilities of generative pre-trained transformers at massive-scale? Abilities that the smaller GTP’s don’t possess. Simple programs can give rise to very complex behaviour. Conway’s game of live is Turing Complete and has four rules. Conway’s Game of Live can simulate a Turing machine, can therefore implant a GTP. Does that mean Conway’s Game of Life is conscious? I don’t think so. Does it rule out Conway’s Game of life from implementing a system that has consciousness as an emergent ability? I’m not convinced I know the answer.

kingofmen 6mo ago

Human brains are also deterministic, though somewhat more difficult to reset to a starting state. So this seems to prove that humans aren't conscious either.

marshray 6mo ago
This seems like an extraordinary claim to make about an above-room-temperature chemical system that, even in the most Newtonian oversimplification, amounts to an astronomical number of oddly-shaped and unevenly-charged billiard balls flying around at jet aircraft speeds.
- thrownthatway 6mo ago
  Definitely agree. We can’t even solve the three body problem. Let alone what I’m calling Marshray Complexity.

AlecSchueler 6mo ago

> And, I don't see how it can be. It is deterministic Why is indeterminism the key to consciousness?

JackFr 6mo ago

> Not just any math: Matrix multiplication. Can matrix multiplication be conscious? And, I don't see how it can be. Assuming your brain and the GPUs are both real physical things, where’s the magic part in your brain that makes you conscious? (Roger Penrose knows, but no one believes him.)

tom2026hn 6mo ago

[dead]