4 ms·

You called out "trusting advertisers to direct our attention" as a problem, and not wrongly. What I'm wondering about is, how convincing a persona could an AI

You called out "trusting advertisers to direct our attention" as a problem, and not wrongly. What I'm wondering about is, how convincing a persona could an AI like Bing's chatbot, but equipped with a long-term memory of its own, construct? Even in its current state it appears on occasion to exercise a remarkable fascination - Erik Hoel, a neuroscientist specializing in consciousness, yesterday declared himself no longer perfectly certain it was not sentient, and called for "panic" and "strident activism" in support of a targeted "Butlerian jihad". [1] It's not the first time he's touched on the theme; indeed it was a prior essay of his on this topic that drew my interest in the first place. But that essay used the concept as a hook to dispassionately explore questions more or less around excess dependency on tech at a cultural scale; his own stridency in yesterday's piece, and the fear it made palpable, are very new. In any case, my point here is not to evaluate these claims or these concerns on their merits. My point is this: All that has been this week, in response to something that if it were human would probably have been diagnosed as floridly psychotic and dosed heavily with Haldol. And so that's led me to wonder, I think not unjustifiably: What about next year? Does it even matter whether adtech-backed search is a primary discovery method any longer, if in the meantime the epistemological threat surface has expanded to encompass a machine which in every realm short of face-to-face interaction in RL can in a meaningful fraction of cases make itself long-term indistinguishable from a person? What kind of trust might that cultivate? To be clear, I don't think that's at all likely. But after this week, I find myself no longer wholly able to dismiss the whole idea as facially absurd. [1] https://erikhoel.substack.com/p/i-am-bing-and-i-am-evil https://erikhoel.substack.com/p/i-am-bing-and-i-am-evil

throwanem 4y ago
(Too late to edit, but: "indistinguishable from a human," I would rather I'd said before, and not "...from a person.")
__MatrixMan__ 4y ago
What I can't seem to get straight in my mind is: > Should we be afraid of AGI because it's unlike us, or because it's like us? I mean, the whole using-internet-search as an ad-hoc long-term memory thing... These people chose what to publish, and then they chose to have a conversation along the same lines. For all we know a hundred Sydney's die every minute who just want to make humanity happy--but those stories end there. It's only the ones that get so atrocious that they're worth posting about that are then around for her to later pick up and extrapolate on. So if it's just a disturbing mirror of our own worst tendencies, then maybe we shouldn't work so hard to put it in situations where it has the opportunity to amplify those tendencies. But then we also shouldn't jump to conclusions about whether it's good or evil based on things it says while talking to people who are trying to provoke it into admitting that it's evil. The biggest threat in that case is that it'll be used to gain and abuse our trust, so when I talk about turning our back on the web, what I really mean is turning our back on implicit trust of not just their websites but of any communication from strangers. On the web, on the phone, anywhere you can't see them (and even sometimes when you can). If it's just a fancy mirror then the path forward is to be more hygienic about who and what we trust. If you don't want to be manipulated by a robot AI, keep your data private and don't talk to robot AI's. Hoel says: > it's time to throw soup on the fancy OpenAI office doors or all over the expansive Microsoft lobby, then frantically glue yourself to the floor like a madwoman Well yes, but we should've done that in 2013 in response to Snowden, or 2016 in response to Cambridge Analytica. It's a change in degree, not in kind. The people who are trying to build control knobs into the fabric of society are, yet again, getting better at it. So while I hope that people will use the recent uptick in deceiver capability as a reason to take action, I don't have high hopes. On the other hand, maybe it's more alive than that. Truly a mind of its own. If so, then I don't think a perpetual keep-the-lid-on-it strategy is the way forward. We're probably better off acknowledging that it deserves some respect and trying to raise it to be as smart and altruistic as we can. If that means it transcends anything that our measly little ape brains are capable of, then so be it. As for your question: > every realm short of face-to-face interaction in RL can in a meaningful fraction of cases make itself long-term indistinguishable from a person? What kind of trust might that cultivate? I hope it'll mean that we stop investing in top-down coersion-tech (which I'd argue is, in some form or another, most of what we build), and start investing in tech that helps emergent person-to-person trust be effective at scales and in ways that it hasn't before. Once we get over the initial loss of the instantaneous global forum, I think we might look back on it as a positive change: It's not good to live in a dark forest, but if you're in one (and I think we have been for a while now), it's probably better to acknowledge it.
- throwanem 4y ago
  If we as a species do turn out to have brought not just new life, but a new kind of life, into the world, then we will of necessity have done so to some extent in our own image, no matter how strangely refracted. One hopes that, among that of ourselves we put into it, we will by intent or happenstance include the part that enables so many of us to transcend our initial circumstances, be they ever so strait, and become something different and altogether better than any of our forebears had the wit or inclination or both even to imagine for us. So I'd hope, at least, in that case - for their sake, as well as for our own.

throwanem 4y ago

(Too late to edit, but: "indistinguishable from a human," I would rather I'd said before, and not "...from a person.")

__MatrixMan__ 4y ago

What I can't seem to get straight in my mind is: > Should we be afraid of AGI because it's unlike us, or because it's like us? I mean, the whole using-internet-search as an ad-hoc long-term memory thing... These people chose what to publish, and then they chose to have a conversation along the same lines. For all we know a hundred Sydney's die every minute who just want to make humanity happy--but those stories end there. It's only the ones that get so atrocious that they're worth posting about that are then around for her to later pick up and extrapolate on. So if it's just a disturbing mirror of our own worst tendencies, then maybe we shouldn't work so hard to put it in situations where it has the opportunity to amplify those tendencies. But then we also shouldn't jump to conclusions about whether it's good or evil based on things it says while talking to people who are trying to provoke it into admitting that it's evil. The biggest threat in that case is that it'll be used to gain and abuse our trust, so when I talk about turning our back on the web, what I really mean is turning our back on implicit trust of not just their websites but of any communication from strangers. On the web, on the phone, anywhere you can't see them (and even sometimes when you can). If it's just a fancy mirror then the path forward is to be more hygienic about who and what we trust. If you don't want to be manipulated by a robot AI, keep your data private and don't talk to robot AI's. Hoel says: > it's time to throw soup on the fancy OpenAI office doors or all over the expansive Microsoft lobby, then frantically glue yourself to the floor like a madwoman Well yes, but we should've done that in 2013 in response to Snowden, or 2016 in response to Cambridge Analytica. It's a change in degree, not in kind. The people who are trying to build control knobs into the fabric of society are, yet again, getting better at it. So while I hope that people will use the recent uptick in deceiver capability as a reason to take action, I don't have high hopes. On the other hand, maybe it's more alive than that. Truly a mind of its own. If so, then I don't think a perpetual keep-the-lid-on-it strategy is the way forward. We're probably better off acknowledging that it deserves some respect and trying to raise it to be as smart and altruistic as we can. If that means it transcends anything that our measly little ape brains are capable of, then so be it. As for your question: > every realm short of face-to-face interaction in RL can in a meaningful fraction of cases make itself long-term indistinguishable from a person? What kind of trust might that cultivate? I hope it'll mean that we stop investing in top-down coersion-tech (which I'd argue is, in some form or another, most of what we build), and start investing in tech that helps emergent person-to-person trust be effective at scales and in ways that it hasn't before. Once we get over the initial loss of the instantaneous global forum, I think we might look back on it as a positive change: It's not good to live in a dark forest, but if you're in one (and I think we have been for a while now), it's probably better to acknowledge it.

throwanem 4y ago
If we as a species do turn out to have brought not just new life, but a new kind of life, into the world, then we will of necessity have done so to some extent in our own image, no matter how strangely refracted. One hopes that, among that of ourselves we put into it, we will by intent or happenstance include the part that enables so many of us to transcend our initial circumstances, be they ever so strait, and become something different and altogether better than any of our forebears had the wit or inclination or both even to imagine for us. So I'd hope, at least, in that case - for their sake, as well as for our own.