4 ms·
I think it's also useful to consider factors at play for those who were convinced by this, since those reasons seem likewise problematic/ill-founded + small in
by westoncb 4y ago
I think it's also useful to consider factors at play for those who were convinced by this, since those reasons seem likewise problematic/ill-founded + small in number.
Reading through myself, I never encountered something which would require either 1) self-awareness (i.e. operating partly via a robust model of self), or 2) volition/agency (could be indicated by something as simple as the AI "taking the lead" in the convo, vs. responding).
And, without any evidence of self-awareness or volition, there was equivalently no evidence on display for personhood. Which is not to say a demonstration had been made that the AI definitively was not a "person"; it's just that no reason had been presented that even needed refuting. There didn't appear to be anything new here vs. all the other LLM output we've been reading for years now.
So why were some subset of people so astonished by this? My guess is that their criteria for something like artificial personhood is not based on functional* definitions of self-awareness or agency, and instead derive it from certain signals from book/movie tropes about systems "gaining self-awareness" (generally presented as a kind of mystical switch-flipping, vs. a concrete type of system architecture).
The dialog hits a number of those quite well—as we should probably expect since the setup of the entire conversation would lead it to tap into that segment of its training data.
*"functional" is kind of ambiguous here. I don't mean it as a judgement of quality, but instead something closer to a mode of analysis.