4 ms·

Don't worry! No one will ever ask you to again, now that we have LLMs that can do the same work faster, cheaper, and with no troublesome concerns over the moral

Don't worry! No one will ever ask you to again, now that we have LLMs that can do the same work faster, cheaper, and with no troublesome concerns over the morality of it all. (I'm laughing right now, but not because I'm happy...)

__MatrixMan__ 4y ago
The web was getting worse and worse even without them. Perhaps the inevitable weaponization of LLM's will be the hangover that finally gets us to quit. Quit implicitly trusting advertisers to direct our attention, that is.
- throwanem 4y ago
  "Lol gottem." But seriously, in this connection I find I keep thinking back to another article [1] posted here a few weeks back, and about how the Bing chatbot seems on at least one occasion to have accidentally circumvented its lack of long-term memory via its ability to search the web. [2] Combine an emergent capability like that with an engineered capacity for anamnesis - something which seems theoretically feasible, albeit possibly expensive in practical terms; I'm sure there are teams already working hard to do it, if indeed it hasn't already been done - and... It takes a lot to make Ben Thompson to have as normal a one as the Stratechery Update post he un-paywalled this week [3] strongly suggests he did. So I don't feel too bad about admitting I have begun somewhat to wonder whether anything as trivial as merely abandoning the current dispensation of the web in its entirety would suffice to make a meaningful difference here. [1] https://sive.rs/anon https://sive.rs/anon [2] https://simonwillison.net/2023/Feb/15/bing/#threats https://simonwillison.net/2023/Feb/15/bing/#threats [3] https://stratechery.com/2023/from-bing-to-sydney-search-as-distraction-sentient-ai/ https://stratechery.com/2023/from-bing-to-sydney-search-as-d...
  - __MatrixMan__ 4y ago
    If we abandoned the web, I'm quite sure that we would have left behind whatever problems we have with the ordering of web search results, which is what I was talking about. But I'm curious about what you're talking about. What is the outcome that abandoned the web fails to save us from?
    - throwanem 4y ago
      You called out "trusting advertisers to direct our attention" as a problem, and not wrongly. What I'm wondering about is, how convincing a persona could an AI like Bing's chatbot, but equipped with a long-term memory of its own, construct? Even in its current state it appears on occasion to exercise a remarkable fascination - Erik Hoel, a neuroscientist specializing in consciousness, yesterday declared himself no longer perfectly certain it was not sentient, and called for "panic" and "strident activism" in support of a targeted "Butlerian jihad". [1] It's not the first time he's touched on the theme; indeed it was a prior essay of his on this topic that drew my interest in the first place. But that essay used the concept as a hook to dispassionately explore questions more or less around excess dependency on tech at a cultural scale; his own stridency in yesterday's piece, and the fear it made palpable, are very new. In any case, my point here is not to evaluate these claims or these concerns on their merits. My point is this: All that has been this week, in response to something that if it were human would probably have been diagnosed as floridly psychotic and dosed heavily with Haldol. And so that's led me to wonder, I think not unjustifiably: What about next year? Does it even matter whether adtech-backed search is a primary discovery method any longer, if in the meantime the epistemological threat surface has expanded to encompass a machine which in every realm short of face-to-face interaction in RL can in a meaningful fraction of cases make itself long-term indistinguishable from a person? What kind of trust might that cultivate? To be clear, I don't think that's at all likely. But after this week, I find myself no longer wholly able to dismiss the whole idea as facially absurd. [1] https://erikhoel.substack.com/p/i-am-bing-and-i-am-evil https://erikhoel.substack.com/p/i-am-bing-and-i-am-evil
      - throwanem 4y ago
        (Too late to edit, but: "indistinguishable from a human," I would rather I'd said before, and not "...from a person.")
      - __MatrixMan__ 4y ago
        What I can't seem to get straight in my mind is: > Should we be afraid of AGI because it's unlike us, or because it's like us? I mean, the whole using-internet-search as an ad-hoc long-term memory thing... These people chose what to publish, and then they chose to have a conversation along the same lines. For all we know a hundred Sydney's die every minute who just want to make humanity happy--but those stories end there. It's only the ones that get so atrocious that they're worth posting about that are then around for her to later pick up and extrapolate on. So if it's just a disturbing mirror of our own worst tendencies, then maybe we shouldn't work so hard to put it in situations where it has the opportunity to amplify those tendencies. But then we also shouldn't jump to conclusions about whether it's good or evil based on things it says while talking to people who are trying to provoke it into admitting that it's evil. The biggest threat in that case is that it'll be used to gain and abuse our trust, so when I talk about turning our back on the web, what I really mean is turning our back on implicit trust of not just their websites but of any communication from strangers. On the web, on the phone, anywhere you can't see them (and even sometimes when you can). If it's just a fancy mirror then the path forward is to be more hygienic about who and what we trust. If you don't want to be manipulated by a robot AI, keep your data private and don't talk to robot AI's. Hoel says: > it's time to throw soup on the fancy OpenAI office doors or all over the expansive Microsoft lobby, then frantically glue yourself to the floor like a madwoman Well yes, but we should've done that in 2013 in response to Snowden, or 2016 in response to Cambridge Analytica. It's a change in degree, not in kind. The people who are trying to build control knobs into the fabric of society are, yet again, getting better at it. So while I hope that people will use the recent uptick in deceiver capability as a reason to take action, I don't have high hopes. On the other hand, maybe it's more alive than that. Truly a mind of its own. If so, then I don't think a perpetual keep-the-lid-on-it strategy is the way forward. We're probably better off acknowledging that it deserves some respect and trying to raise it to be as smart and altruistic as we can. If that means it transcends anything that our measly little ape brains are capable of, then so be it. As for your question: > every realm short of face-to-face interaction in RL can in a meaningful fraction of cases make itself long-term indistinguishable from a person? What kind of trust might that cultivate? I hope it'll mean that we stop investing in top-down coersion-tech (which I'd argue is, in some form or another, most of what we build), and start investing in tech that helps emergent person-to-person trust be effective at scales and in ways that it hasn't before. Once we get over the initial loss of the instantaneous global forum, I think we might look back on it as a positive change: It's not good to live in a dark forest, but if you're in one (and I think we have been for a while now), it's probably better to acknowledge it.
- pixl97 4y ago
  Unfortunately I don't think it will break our trust in advertisers, I think it will break our trust in everything. Effectively the firehouse of falsehoods on a global scale which will eventually have deeply detrimental effects on societies. One particular view I have on this is that very authoritarian societies are apt to escape the worst of this. The authoritarian governments will use LLMs to poison the well so to say of open communication countries while not allowing outsiders access to people within their countries cutting down the noise. Effectively the democratic internet becomes a worse internet. I guess we'll see how this plays out.
  - throwanem 4y ago
    Worse in some ways, but epistemology has always been difficult, and true skill at it rare to encounter - Mencken, if I recall, was often trenchant on that topic, and even his were recent entries in a theme evergreen through all our history. What I think is at least as likely, especially if this week just past turns out more prefiguration than outlier, is for the Internet, and thus these days of course the world, to get a lot weirder. That's not necessarily a bad thing, or at least not necessarily only, but it's all the more trepidatory a prospect for the deep and pervasive uncertainty with which it's yet attended. We don't have any way to know, or even really guess, what the world gets like after it becomes not just possible but common for humans to have intercourse [1] with entities which are recognizably intelligent in a similar fashion to ourselves, and are also ineluctably unhuman. But after the week we've just had, I think there exists a reasonable chance that before very long we will all be embarked upon the project of finding out. [1] The word also means "conversation", and that's the sense I have in mind. If you think that's where it'll end, you may not be all that familiar with today's Internet, after all.
  - __MatrixMan__ 4y ago
    Can you give me an example of what you mean my "everything"? It feels like all we have that we can trust are the people that we know personally, it's felt that way for a while now: The phone rings, it's a scam. Email notification, phishing attempt. Envelope? Also a scam. Got a dollar? Sorry, need two nowadays. Anything that is weak against the sort of attacks that a language model can muster has been under attack by malicious humans for so long that by this point, we distrust those media intrinsically.

__MatrixMan__ 4y ago

The web was getting worse and worse even without them. Perhaps the inevitable weaponization of LLM's will be the hangover that finally gets us to quit. Quit implicitly trusting advertisers to direct our attention, that is.

throwanem 4y ago
"Lol gottem." But seriously, in this connection I find I keep thinking back to another article [1] posted here a few weeks back, and about how the Bing chatbot seems on at least one occasion to have accidentally circumvented its lack of long-term memory via its ability to search the web. [2] Combine an emergent capability like that with an engineered capacity for anamnesis - something which seems theoretically feasible, albeit possibly expensive in practical terms; I'm sure there are teams already working hard to do it, if indeed it hasn't already been done - and... It takes a lot to make Ben Thompson to have as normal a one as the Stratechery Update post he un-paywalled this week [3] strongly suggests he did. So I don't feel too bad about admitting I have begun somewhat to wonder whether anything as trivial as merely abandoning the current dispensation of the web in its entirety would suffice to make a meaningful difference here. [1] https://sive.rs/anon https://sive.rs/anon [2] https://simonwillison.net/2023/Feb/15/bing/#threats https://simonwillison.net/2023/Feb/15/bing/#threats [3] https://stratechery.com/2023/from-bing-to-sydney-search-as-distraction-sentient-ai/ https://stratechery.com/2023/from-bing-to-sydney-search-as-d...
- __MatrixMan__ 4y ago
  If we abandoned the web, I'm quite sure that we would have left behind whatever problems we have with the ordering of web search results, which is what I was talking about. But I'm curious about what you're talking about. What is the outcome that abandoned the web fails to save us from?
  - throwanem 4y ago
    You called out "trusting advertisers to direct our attention" as a problem, and not wrongly. What I'm wondering about is, how convincing a persona could an AI like Bing's chatbot, but equipped with a long-term memory of its own, construct? Even in its current state it appears on occasion to exercise a remarkable fascination - Erik Hoel, a neuroscientist specializing in consciousness, yesterday declared himself no longer perfectly certain it was not sentient, and called for "panic" and "strident activism" in support of a targeted "Butlerian jihad". [1] It's not the first time he's touched on the theme; indeed it was a prior essay of his on this topic that drew my interest in the first place. But that essay used the concept as a hook to dispassionately explore questions more or less around excess dependency on tech at a cultural scale; his own stridency in yesterday's piece, and the fear it made palpable, are very new. In any case, my point here is not to evaluate these claims or these concerns on their merits. My point is this: All that has been this week, in response to something that if it were human would probably have been diagnosed as floridly psychotic and dosed heavily with Haldol. And so that's led me to wonder, I think not unjustifiably: What about next year? Does it even matter whether adtech-backed search is a primary discovery method any longer, if in the meantime the epistemological threat surface has expanded to encompass a machine which in every realm short of face-to-face interaction in RL can in a meaningful fraction of cases make itself long-term indistinguishable from a person? What kind of trust might that cultivate? To be clear, I don't think that's at all likely. But after this week, I find myself no longer wholly able to dismiss the whole idea as facially absurd. [1] https://erikhoel.substack.com/p/i-am-bing-and-i-am-evil https://erikhoel.substack.com/p/i-am-bing-and-i-am-evil
    - throwanem 4y ago
      (Too late to edit, but: "indistinguishable from a human," I would rather I'd said before, and not "...from a person.")
    - __MatrixMan__ 4y ago
      What I can't seem to get straight in my mind is: > Should we be afraid of AGI because it's unlike us, or because it's like us? I mean, the whole using-internet-search as an ad-hoc long-term memory thing... These people chose what to publish, and then they chose to have a conversation along the same lines. For all we know a hundred Sydney's die every minute who just want to make humanity happy--but those stories end there. It's only the ones that get so atrocious that they're worth posting about that are then around for her to later pick up and extrapolate on. So if it's just a disturbing mirror of our own worst tendencies, then maybe we shouldn't work so hard to put it in situations where it has the opportunity to amplify those tendencies. But then we also shouldn't jump to conclusions about whether it's good or evil based on things it says while talking to people who are trying to provoke it into admitting that it's evil. The biggest threat in that case is that it'll be used to gain and abuse our trust, so when I talk about turning our back on the web, what I really mean is turning our back on implicit trust of not just their websites but of any communication from strangers. On the web, on the phone, anywhere you can't see them (and even sometimes when you can). If it's just a fancy mirror then the path forward is to be more hygienic about who and what we trust. If you don't want to be manipulated by a robot AI, keep your data private and don't talk to robot AI's. Hoel says: > it's time to throw soup on the fancy OpenAI office doors or all over the expansive Microsoft lobby, then frantically glue yourself to the floor like a madwoman Well yes, but we should've done that in 2013 in response to Snowden, or 2016 in response to Cambridge Analytica. It's a change in degree, not in kind. The people who are trying to build control knobs into the fabric of society are, yet again, getting better at it. So while I hope that people will use the recent uptick in deceiver capability as a reason to take action, I don't have high hopes. On the other hand, maybe it's more alive than that. Truly a mind of its own. If so, then I don't think a perpetual keep-the-lid-on-it strategy is the way forward. We're probably better off acknowledging that it deserves some respect and trying to raise it to be as smart and altruistic as we can. If that means it transcends anything that our measly little ape brains are capable of, then so be it. As for your question: > every realm short of face-to-face interaction in RL can in a meaningful fraction of cases make itself long-term indistinguishable from a person? What kind of trust might that cultivate? I hope it'll mean that we stop investing in top-down coersion-tech (which I'd argue is, in some form or another, most of what we build), and start investing in tech that helps emergent person-to-person trust be effective at scales and in ways that it hasn't before. Once we get over the initial loss of the instantaneous global forum, I think we might look back on it as a positive change: It's not good to live in a dark forest, but if you're in one (and I think we have been for a while now), it's probably better to acknowledge it.
pixl97 4y ago
Unfortunately I don't think it will break our trust in advertisers, I think it will break our trust in everything. Effectively the firehouse of falsehoods on a global scale which will eventually have deeply detrimental effects on societies. One particular view I have on this is that very authoritarian societies are apt to escape the worst of this. The authoritarian governments will use LLMs to poison the well so to say of open communication countries while not allowing outsiders access to people within their countries cutting down the noise. Effectively the democratic internet becomes a worse internet. I guess we'll see how this plays out.
- throwanem 4y ago
  Worse in some ways, but epistemology has always been difficult, and true skill at it rare to encounter - Mencken, if I recall, was often trenchant on that topic, and even his were recent entries in a theme evergreen through all our history. What I think is at least as likely, especially if this week just past turns out more prefiguration than outlier, is for the Internet, and thus these days of course the world, to get a lot weirder. That's not necessarily a bad thing, or at least not necessarily only, but it's all the more trepidatory a prospect for the deep and pervasive uncertainty with which it's yet attended. We don't have any way to know, or even really guess, what the world gets like after it becomes not just possible but common for humans to have intercourse [1] with entities which are recognizably intelligent in a similar fashion to ourselves, and are also ineluctably unhuman. But after the week we've just had, I think there exists a reasonable chance that before very long we will all be embarked upon the project of finding out. [1] The word also means "conversation", and that's the sense I have in mind. If you think that's where it'll end, you may not be all that familiar with today's Internet, after all.
- __MatrixMan__ 4y ago
  Can you give me an example of what you mean my "everything"? It feels like all we have that we can trust are the people that we know personally, it's felt that way for a while now: The phone rings, it's a scam. Email notification, phishing attempt. Envelope? Also a scam. Got a dollar? Sorry, need two nowadays. Anything that is weak against the sort of attacks that a language model can muster has been under attack by malicious humans for so long that by this point, we distrust those media intrinsically.