3 ms·

This article is dripping with the same kind of cringey techno-engineering naivete you find in hollywood movies. The author is totally lost in the sauce of compl

This article is dripping with the same kind of cringey techno-engineering naivete you find in hollywood movies. The author is totally lost in the sauce of complex surface level analyses mixed with romantic ideals of human exceptionalism, and completely blind to the deeper abstractions and common under girding systems that an expertise in computation would reveal (and don't have any care for emotional concepts). The takeaway seems to be "Only meat brains can be conscious because I can feel it and computers aren't made of meat". Which is basically the plot line of every human/robot movie for the last 80 years.

adamzwasserman 9mo ago
The interesting version of the argument isn't about substrate: it's about motivation. Present the trolley problem to GPT-4 and it gives you a philosophy survey answer. Present it to a human and their palms sweat. The gap isn't computation, it's that humans are value-making machines shaped by millions of years of selection pressure. Pollan lands on the wrong argument (biology vs. silicon) when the real one is: where do the values come from, and can they emerge without a reproductive lineage that stakes survival on getting them right?
- ath3nd 9mo ago
  [dead]
- judahmeek 9mo ago
  I refer to this as moral grounding. I'm not sure I would call it a requirement for consciousness, but knowing that most beings with general intelligence (humans) have a form of it similar to my own does make it easier to sleep at night.
  - adamzwasserman 8mo ago
    To clarify: I'm not talking about morals specifically. I mean value in the broader sense of spontaneously assigning relative importance to things, producing a hierarchy that drives action. You're thirsty. There's pond water in the forest and clean well water in the town square, but you're an escaped prisoner. Suddenly the value hierarchy flips: safety trumps water quality. You do this instantly, with incomplete information, integrating survival, context, and preference in a way that no one programmed into you. Morality is one expression of this capacity, but so is aesthetic judgment, risk assessment, curiosity, and the decision to walk down a dark alley or not. The trolley problem is just a dramatic example. The mundane examples are actually more telling, because we do them thousands of times a day without noticing. No current AI has any form of this. It has no mechanism for deciding that anything matters more than anything else except through weightings that were derived from human-generated training data. It borrows our value hierarchies statistically. It doesn't have its own.
tim333 8mo ago
I'd hazard a counter prediction that we'll have AI seeming pretty conscious within a decade. People will say it's not real in the same way they say said driving cars will never work when they were driving around but it will become hard to argue against when you can hang out with real examples.

adamzwasserman 9mo ago

The interesting version of the argument isn't about substrate: it's about motivation. Present the trolley problem to GPT-4 and it gives you a philosophy survey answer. Present it to a human and their palms sweat. The gap isn't computation, it's that humans are value-making machines shaped by millions of years of selection pressure. Pollan lands on the wrong argument (biology vs. silicon) when the real one is: where do the values come from, and can they emerge without a reproductive lineage that stakes survival on getting them right?

ath3nd 9mo ago
[dead]
judahmeek 9mo ago
I refer to this as moral grounding. I'm not sure I would call it a requirement for consciousness, but knowing that most beings with general intelligence (humans) have a form of it similar to my own does make it easier to sleep at night.
- adamzwasserman 8mo ago
  To clarify: I'm not talking about morals specifically. I mean value in the broader sense of spontaneously assigning relative importance to things, producing a hierarchy that drives action. You're thirsty. There's pond water in the forest and clean well water in the town square, but you're an escaped prisoner. Suddenly the value hierarchy flips: safety trumps water quality. You do this instantly, with incomplete information, integrating survival, context, and preference in a way that no one programmed into you. Morality is one expression of this capacity, but so is aesthetic judgment, risk assessment, curiosity, and the decision to walk down a dark alley or not. The trolley problem is just a dramatic example. The mundane examples are actually more telling, because we do them thousands of times a day without noticing. No current AI has any form of this. It has no mechanism for deciding that anything matters more than anything else except through weightings that were derived from human-generated training data. It borrows our value hierarchies statistically. It doesn't have its own.

tim333 8mo ago

I'd hazard a counter prediction that we'll have AI seeming pretty conscious within a decade. People will say it's not real in the same way they say said driving cars will never work when they were driving around but it will become hard to argue against when you can hang out with real examples.