4 ms·

> "But we do know exactly what we'd like to formally model" Do we really? I thought it was a deeply intractable problem for cognitive theorists and other such

> "But we do know exactly what we'd like to formally model" Do we really? I thought it was a deeply intractable problem for cognitive theorists and other such experts. Haven't you ever encountered a challenge that you at first thought was straight-forward and obvious to solve, but after digging in find it was unexpectedly nuanced and difficult? > "The limits of this are harder to pin down, as, again, we lack a deep formal understanding." Doesn't that contradict? If you were to try pinning down precisely what you mean by "the kind of sentience that living, awake humans surely have, most animals probably have, and rocks definitely don't have", I suspect it will not be nearly as easy as you think. How do you know rocks don't have it, for instance? How do you know you do, for that matter?

tsimionescu 4y ago
> How do you know rocks don't have it, for instance? How do you know you do, for that matter? These are both part of the (incomplete) definition I gave you. There have been problems like this before in the history of science - we have concepts that we intuitively understand, and want to give a formal definition for. This can take a long time of careful study - such as coming up with a complete definition for "insect". Other times it can indeed turn out that the concept was ill-defined - there is no possible formal definition for the concept "fish" that doesn't appeal to the human mind (the various organisms we think of as fish are too diverse genetically to find a satisfactory one-to-one mapping with any group of animals, even if you exclude a small set of them, like the aquatic mammals).
- rendall 4y ago
  Ok. I'm responding to the statement that these are "relatively clear concepts". AI will by definition never have "the kind of sentience that living, awake humans have" (and I do acknowledge that this is only a working definition) because the goalposts will continue to move according to whim in the absence of a more formal definition. I remember when one prevailing formal test of self-awareness was "independently claiming to have self-awareness". There were caveats, of course, but that goalpost has evidently moved.

tsimionescu 4y ago

> How do you know rocks don't have it, for instance? How do you know you do, for that matter? These are both part of the (incomplete) definition I gave you. There have been problems like this before in the history of science - we have concepts that we intuitively understand, and want to give a formal definition for. This can take a long time of careful study - such as coming up with a complete definition for "insect". Other times it can indeed turn out that the concept was ill-defined - there is no possible formal definition for the concept "fish" that doesn't appeal to the human mind (the various organisms we think of as fish are too diverse genetically to find a satisfactory one-to-one mapping with any group of animals, even if you exclude a small set of them, like the aquatic mammals).

rendall 4y ago
Ok. I'm responding to the statement that these are "relatively clear concepts". AI will by definition never have "the kind of sentience that living, awake humans have" (and I do acknowledge that this is only a working definition) because the goalposts will continue to move according to whim in the absence of a more formal definition. I remember when one prevailing formal test of self-awareness was "independently claiming to have self-awareness". There were caveats, of course, but that goalpost has evidently moved.