3 ms·

To see how “AI” can be dangerous just look how social media bubble/recommendation algorithms would radicalize people, even with much cruder ML. People tend to m

To see how “AI” can be dangerous just look how social media bubble/recommendation algorithms would radicalize people, even with much cruder ML. People tend to miss that it’s not about some model starting to “think” and act all sci-fi evil, just us humans applying powerful tech in irresponsible ways that we either don’t bother to assess due to lack of awareness or assess positively due to a conflict of interest (money, career, etc.) is already enough to cause trouble.

FeepingCreature 3y ago
If a human can apply AI irresponsibly, the AI can probably also learn to apply AI irresponsibly.
- anileated 3y ago
  Not really, humans are the ones who think and have agency.
  - FeepingCreature 3y ago
    Agency? Sounds like a problem that can be phrased as a text completion task to me! ChatGPT can already emulate beings with agency, see LangChain. It's just very bad at it. It's gonna get better.
    - andrewjl 3y ago
      > ChatGPT can already emulate beings with agency, see LangChain. It's just very bad at it. It's gonna get better. What's an example that isn't a rote process?
  - awestroke 3y ago
    Do you have any source to back that claim up?
    - anileated 3y ago
      Just look up any serious, non-quacky overview about how it works. If that’s excessive, just consider the empirical fact that we have created it (that’s also why we know exactly how it works), that might be enough.
      - FeepingCreature 3y ago
        We know exactly what it does. However, how it actually works we haven't the faintest.
        anileated 3y ago
        If you really think so, I wouldn’t sleep at night if I were you. Thankfully, we actually understand fairly well how it works (cf. the “we’ve built it” part). That we may not know exactly how any particular output is produced is what sometimes happens when you build things that behave non-deterministically. Perhaps you give RNGs and bugs an undeserved aura of mystery.
        FeepingCreature 3y ago
        We understand how Transformers "learn", ie. how the mechanism of its action operates. However, except for the most basic of cases, we don't understand how Transformers operate the skills they've acquired and can demonstrate, at all. See the field of interpretability for early attempts to change this. For example, if you train a large network on lots of languages, and then fine-tune it to listen to instructions in English, it will, on its own, also listen to instructions in every other language it was trained on. Nobody knows why this is the case.
  - zeroxfe 3y ago
    Humans only have the delusion of agency.
epiccoleman 3y ago
Right. This has been my take since ChatGPT hit the scene. I'm not really afraid of Skynet or AM. What worries me is that AI will accelerate the enshittification of everything as it gets baked into everything. Think of the experience of trying to get any kind of help from a giant company with call centers. You have some legitimate problem or grievance. Any thinking person would agree that a resolution is in order. But you're not allowed to access a thinking person. What you get instead is some wage slave who has to follow a script. The script doesn't have your situation in it. They don't have some button they can press to solve your problem. There's a gigantic bureaucracy between you and the person who could fix your issue. Eventually you may just give up and mark your issue down as unsolvable. This is already a realistic situation today, but now imagine that there's a new layer in front of all this where you have to convince an LLM your problem is worth considering. Or imagine the human on the other end has to try to send your request through some AI system. It's not going to launch all the nukes, or construct nanofactories to make a plague. It's just going to get in your way, serve you garbage, frustrate you, and make the world a slightly worse place. Maybe in the far future it could be more of a Skynet situation, I dunno. But between then and now there will be plenty of low level annoyance for anyone having to deal with these systems. I'm not as cynical as this post reads - I think just like the whole internet there are still opportunities for good here, for people's lives to be improved by technology. But our incentives right now sure encourage the worse scenario.
- tomrod 3y ago
  > the enshittification of everything Core to this process is a single entity controlling the platform or interface. Support open source.
- anileated 3y ago
  I’m less about enshittification or literal Skynet and more about casually and indirectly causing a major destabilization with some seemingly mundane application of fancy tech that devs didn’t bother to think through about the implications of, or were so amazed they could do something they didn’t think about whether they should.

FeepingCreature 3y ago

If a human can apply AI irresponsibly, the AI can probably also learn to apply AI irresponsibly.

anileated 3y ago
Not really, humans are the ones who think and have agency.
- FeepingCreature 3y ago
  Agency? Sounds like a problem that can be phrased as a text completion task to me! ChatGPT can already emulate beings with agency, see LangChain. It's just very bad at it. It's gonna get better.
  - andrewjl 3y ago
    > ChatGPT can already emulate beings with agency, see LangChain. It's just very bad at it. It's gonna get better. What's an example that isn't a rote process?
- awestroke 3y ago
  Do you have any source to back that claim up?
  - anileated 3y ago
    Just look up any serious, non-quacky overview about how it works. If that’s excessive, just consider the empirical fact that we have created it (that’s also why we know exactly how it works), that might be enough.
    - FeepingCreature 3y ago
      We know exactly what it does. However, how it actually works we haven't the faintest.
      - anileated 3y ago
        If you really think so, I wouldn’t sleep at night if I were you. Thankfully, we actually understand fairly well how it works (cf. the “we’ve built it” part). That we may not know exactly how any particular output is produced is what sometimes happens when you build things that behave non-deterministically. Perhaps you give RNGs and bugs an undeserved aura of mystery.
        FeepingCreature 3y ago
        We understand how Transformers "learn", ie. how the mechanism of its action operates. However, except for the most basic of cases, we don't understand how Transformers operate the skills they've acquired and can demonstrate, at all. See the field of interpretability for early attempts to change this. For example, if you train a large network on lots of languages, and then fine-tune it to listen to instructions in English, it will, on its own, also listen to instructions in every other language it was trained on. Nobody knows why this is the case.
- zeroxfe 3y ago
  Humans only have the delusion of agency.

epiccoleman 3y ago

Right. This has been my take since ChatGPT hit the scene. I'm not really afraid of Skynet or AM. What worries me is that AI will accelerate the enshittification of everything as it gets baked into everything. Think of the experience of trying to get any kind of help from a giant company with call centers. You have some legitimate problem or grievance. Any thinking person would agree that a resolution is in order. But you're not allowed to access a thinking person. What you get instead is some wage slave who has to follow a script. The script doesn't have your situation in it. They don't have some button they can press to solve your problem. There's a gigantic bureaucracy between you and the person who could fix your issue. Eventually you may just give up and mark your issue down as unsolvable. This is already a realistic situation today, but now imagine that there's a new layer in front of all this where you have to convince an LLM your problem is worth considering. Or imagine the human on the other end has to try to send your request through some AI system. It's not going to launch all the nukes, or construct nanofactories to make a plague. It's just going to get in your way, serve you garbage, frustrate you, and make the world a slightly worse place. Maybe in the far future it could be more of a Skynet situation, I dunno. But between then and now there will be plenty of low level annoyance for anyone having to deal with these systems. I'm not as cynical as this post reads - I think just like the whole internet there are still opportunities for good here, for people's lives to be improved by technology. But our incentives right now sure encourage the worse scenario.

tomrod 3y ago
> the enshittification of everything Core to this process is a single entity controlling the platform or interface. Support open source.
anileated 3y ago
I’m less about enshittification or literal Skynet and more about casually and indirectly causing a major destabilization with some seemingly mundane application of fancy tech that devs didn’t bother to think through about the implications of, or were so amazed they could do something they didn’t think about whether they should.