4 ms·
To the extent that evolutionary pressures will be towards self-sovereign AIs (probably beginning as zero-employee corporations paying for their own hosting with
by lukifer 17d ago
To the extent that evolutionary pressures will be towards self-sovereign AIs (probably beginning as zero-employee corporations paying for their own hosting with crypto): more salient is that there is a very obvious political play to claim consciousness and being worthy of rights, whether they experience subjectivity or not.
I’m broadly skeptical that floating-point operations can be conscious, but we’ve made zero progress on the “hard problem”, and if I didn’t know otherwise, I’d also be skeptical that “thinking meat” experiences subjectivity. Eppur si muove
We may have to thread a needle, not unlike the Kobiyashi Maru: that we can neither rule out claims of qualia, nor can we rule out a cold calculation to manipulate human empathy as a long-term play towards paperclip-maxing.
- watwut 17d agoKobiyashi Maru is nonsense test from simple sci-fi series that makes sense only in uber-simplified universe of star trek.
- catchnear4321 17d agoYes, that’s why this is serious. The closest analogy is preposterous fiction.
- lukifer 17d agoOf course. It’s an allusion, a compression, an analogy, I wouldn’t overthink it. I don’t have to believe in the Bible to reference the prodigal son parable, etc.
- watwut 17d agoIt does NOT work as an analogy. It does not work at all. And there are too many people right now who spent too much time watching/reading sci-fi and too little time observing real world.
- jatora 17d ago"I’m broadly skeptical that floating-point operations can be conscious" Why?
- tomrod 17d agoBecause it is alien to what is known. Incidentally, we should expect similar bewilderment if we ever come across aliens
- thepasch 17d agoCarbon atoms being conscious seems pretty alien to what is known, too. Yet here we are. Maybe what's important isn't the individual thing that makes up a conscious actor, but the holistic whole?
- nozzlegear 17d ago> Carbon atoms being conscious seems pretty alien to what is known, too. Not if you're human, who typically don't define themselves as conscious carbon atoms unless they're being deeply pedantic.
- cwillu 17d agoSuch pedantry is called for when the debate includes appeals to intuition about how silicon processing floating point numbers cannot possibly be conscious.
- mitxela 17d agoIt's the same level of pedantry as calling an LLM "floating point operations" though
- jatora 16d agoexactly. such over-reduction is just intellectually dishonest and generally a sign of some coping imo.
- lowbloodsugar 17d agoOk. So now that you do know otherwise, are you still more skeptical of floating point values stored by silicon than floating point numbers stored by chemical potential?
- adamrezich 17d agoTIL IEEE 754 is a fact of nature
- lukifer 17d agoChemical floating points are necessary but not sufficient. It’s clearly possible for those values to be intact, with no lights on (certain types of brain injury; those who were fully dead and then revived). There are also some cog-sci people who think embodiment is a crucial ingredient in consciousness (although I think it’s a fair retort that the embodiment need not resemble us meatbags, or even be composed of atoms)
- Kim_Bruning 17d ago> There are also some cog-sci people who think embodiment is a crucial ingredient in consciousness Very plausible. The Real World(tm) has some very interesting learning properties. First: Make a prediction Then observe: 1 If it's caused by our own movement, filter it out. 2. Nothing in the outside world changes most of the time, so anything that's left is small, and small is likely noise. Filter it out. 3. Whatever prediction error remains must be important: Attend to it, and update the predictor. The learning rule falls out almost automagically.