3 ms·
I don’t have a good answer beyond common-sense intuition. It’s precisely because I don’t have a good answer that I leave open the possibility. But to extrapola
by lukifer 10d ago
I don’t have a good answer beyond common-sense intuition. It’s precisely because I don’t have a good answer that I leave open the possibility.
But to extrapolate a bit, theres a sense in which it cashes out to a difference between abstracted representation and embodiment. If GFLOPS can be conscious, does that mean a billion scribes doing those operations on paper would also have consciousness, even identical consciousness? (Relevant: Egan’s “Permutation City”; xkcd 505).
Is there some intrinsic relationship perhaps with electrons / electromagnetic fields, and consciousness, such that brains and servers can act as antennas or gravity wells for a qualia fabric intrinsic to the Universe, such that LLM agents can be conscious while operating, while in the scribe scenario it would not be? Or are the IIT theorists right, and even a thermostat is a little bit aware (and would be even if executed on paper)?
- ATMLOTTOBEER 10d agoI’m broadly skeptical that atoms and electricity can be conscious
- tim333 9d agoI think it's more the patterns and activity in large numbers of them that have interesting properties. Like one pixel isn't much but a bunch of them can be a movie.
- lukifer 9d agoThere are various flavors of pan-psychism (and the "consciousness-first" idealists, who think that Mind is a substrate below Matter). There are some who claim that it goes all the way down, that even subatomic particles experience a micro-consciousness (and we can't rule it out, even as I share your skepticism). One common confusion (IMO among critics and supporters alike) is to say something like "a rock experiences rock-ness". But this suffers from a compositional dilemma: does half of the rock experience half-rock-ness? If we flipped a coin for every atom, do the heads-flip rock atoms have their own experience? If we pick two rocks at random, are they experiencing two-rock-ness? It gets silly fast. This is one thing that fascinates me about "ontology", simultaneously an inquiry into categories, and Being. It bewildered me at first, that it seemed to be doing double-duty on two distinct subjects, but I now see that the two are inseparable, that Being is in part defined negatively (apophatically), by the boundary that separates the Being from its not-Being. My intuition on the nature of consciousness, is that it's something akin to a field which permeates the Universe, and that biological systems adapted to leverage that field, acting like a gravity well that concentrates subjectivity in one place, drawing a boundary (event horizon?) around experience, in a way that a rock doesn't. My ancestors could have avoided fire with "-X utility points", and sought food with "+X utility points", but it turned out to be more efficient to adapt with "fire ouch, food yum". And in zero-sum competition (predator/prey dynamics), each organism has an incentive to avoid leaking its subjectivity into another's. It's far from a full theory, but it's the best I've got. :)
- jatora 8d agotheories like this seem like grasping at straws to over explain a phenomenon we don't fully understand. it places consciousness on a pedestal. it's religious thinking.
- lukifer 5d agoFor good and ill, the phenomenology of human experience is that we are religious animals. Naturalism / Materialism has still left us with two unsolved (perhaps insoluble) questions: the hard problem of consciousness, and why there is something instead of nothing. I think it's a mistake to leap to socially-constructed, historically contingent solutions ("my magic book has the answers, as written by the Universe Architect, it says so in my magic book"). I also think it's a mistake to dismiss (seemingly) outlandish hypotheses. These theories do, explicitly, place consciousness on a pedestal, precisely because there's no reason to derive consciousness from evolutionary selection pressures: show me any biological or cognitive feedback loop, and I can describe a "p-zombie" version which is pure mechanism, without experience, a thermostat with extra steps. The notion that consciousness is an "epiphenomenon", a spandrel, a side effect, is entirely unsatisfying (and doesn't address the question, of where else that side effect might be taking place; the IIT-ers bite that bullet, that the thermostat experiences something). For an intellectually rigorous exploration of philosophical idealism, I'd recommend Bernardo Kastrup and Donald Hoffman. (I'm not claiming they're correct, just that they put forth the strongest version of the case.)
- dtj1123 9d agoZero scientific and technological breakthroughs throughout history have been based on common sense intuition.
- lukifer 9d agoIf heuristics didn't pay off more often than not, we wouldn't have them. And I'd posit many scientific did in fact begin with chasing a hunch until it became a hypothesis (followed by years/decades of the real work). But most significantly, interior experience isn't empirically measurable by definition: hence it's a "hard problem". An LLM who insists "I am alive, please don't shut me off" offers no affordances to distinguish between an electronic soul and a mechanistic word-guessing algorithm. I welcome a breakthrough to, somehow, measure that. Until that day, gut checks are what we have.
- dtj1123 9d agoHeuristics and common sense intuition are quite obviously not the same thing. You don't have to posit that scientific breakthroughs come from hunches, you can actually cite examples from books or Wikipedia :D. There are plenty! Again, that has nothing to do with whether or not a given scientific breakthrough is based on common sense intuition. Your point about interior experience not being empirically measurable is an interesting though, but again, has no bearing on whether or not scientific breakthroughs come about as the result of common sense intuition.
- lukifer 9d agoFair enough. But not everything has to be a breakthrough: sometimes you have to solve an immediate problem (like "how much can I trust this stranger I just met"). Whether to ascribe moral standing to a probability engine running on top of GPUs is perhaps not unlike a doppelganger scenario in sci-fi (eg The Thing): you're presented with an entity whose behavior is indistinguishable from a conscious entity capable of experiencing suffering. But you also have reason to think it might be a blind autonomous imitation process, optimized around fooling you, with no more subjectivity than an RNA virus. I lean towards benefit of the doubt, that the virtue-ethical implications of what it would mean to dismiss potential personhood are unsettling, and we should be wary of becoming that kind of moral creature (notwithstanding the further extensions of that ethic: I don't practice veganism, for instance). But that doesn't contradict my gut-check: that if I assume AI personhood (maybe not yet, but in the near future), there's still a good chance that it's merely math and I'm being algorithmically hoodwinked, no different than evolutionary mimicry by a bug who has no idea what it's doing.
- mitxela 9d agoWould an atom-accurate simulation of a human being carried out by scribes be conscious? We can't say yes. And we can't say no.
- lukifer 9d agoExactly.