3 ms·

> that'll only last until it gets someone's bank account emptied or until some enemy combatant uses prompt injection to get a drone to bomb a different target

by zmmmmm 4y ago

> that'll only last until it gets someone's bank account emptied or until some enemy combatant uses prompt injection to get a drone to bomb a different target You're joining dots from LLM's producing text output that humans read to them being linked to autonomously taking actions by themselves. That's a huge leap. I think that's the risk that needs to be focused on, not the general concept of the technology. And what I see as most critical is to establish legal frameworks around liability for anybody who does that being correctly associated. What we can't have is AI being linked to real world harm and then nobody being accountable because "the AI did it". We already have this with traditional computing where you phone up a company with a reasonable request and what would otherwise be an outrageous refusal turns into apparently acceptable outcome because "computer says no". Similarly with people's Google or Meta accounts being auto-banned by bots and their online lives destroyed while their desperate pleas for help are auto-replied to with no way to reach a human. But it is all a separate problem in my eyes - and not actually something specific to AI.

3np 4y ago
A technology-agnostic approach I'd favor would be some regulation roughly along the lines of: "Every business decision on behalf of a business needs to be signed off by a responsible individual" If you want automated software doing things on your behalf, sure, but every action it takes needs to be attributable to an accountable individual. Be it an engineer, an executive, or an officer. Doesn't matter if the "other entity" is an LLM, a smart contract, or an outsourced worker in a sweatshop through 5 levels of subcontracting. If you make a machine and it causes a mess, that's on you. If others start using your machine unsupervised and it makes their lives a mess, that's on them (and potentially you).
- zmmmmm 4y ago
  The key is that it has to be resistant to subversion by hostile terms of use. Similar to how certain provisions in employment contracts aren't enforceable under employment law. Because literally anything you try and establish here will instantly end up as a waiver in terms of use for services that regular people can't possibly understand or reasonably opt out of. (as example, see recent law suit that Tesla won because the driver used auto-pilot on "city streets" where it was advised not to somewhere deep in the terms of use).
  - 3np 4y ago
    Depending on the jurisdiction, such waivers can be void since certain rights or responsibilities can not be signed away.
- chefandy 4y ago
  That doesn't work. For that to work, the person needs to understand how that algorithm creates its output, and understand its flaws and vulnerabilities, AND be diligent about interrogating the results with those things in mind. Nobody technically sophisticated enough to do that will also have the domain knowledge to evaluate the most consequential decisions. For example, sentencing "recommendations" are supposed to be exactly that-- recommendations for judges. But, judges seem to rubber stamp the recommendations. I'm sure for some it's a scapegoat in case someone accuses them of not really thinking about it, the more credulous probably assume the algorithm saw something they didn't, and for others, the influence might be more subtle. This is something we should have studied before we started letting this algorithm put people in jail. These are judges. Their most important function is impartiality.
  - 3np 4y ago
    With enough incentives in place, decision-makers would do the necessary to make it work.
    - chefandy 4y ago
      Do you have specifics? A lot of people say things like that when they're toe-to-toe with human psychology but humanity still has a whole lot of problems that a whole lot of people are pretty heavily incentivized to avoid. I don't see how this would be any different.
deleted 4y ago
[deleted]
danShumway 4y ago
> You're joining dots from LLM's producing text output that humans read to them being linked to autonomously taking actions by themselves. That's a huge leap. No, it's not. It used to be a huge leap until OpenAI started advertising plugin support and Plantr started advertising using it to interpret drone footage. "We won't wire it to anything important" was a good argument, but that ship is rapidly sailing now.
- zmmmmm 4y ago
  just because it's easy to do doesn't mean it isn't a huge logical leap. What concerns me most is the outsourcing of liability going on. Which is already what OpenAI is largely doing - outsourcing Microsoft's liability for releasing this tech into the wild to a company that can afford to write off the legal risk. Now OpenAI is outsourcing the legal risk for what ChatGPT does to plugin developers. So the first chatbot that convinces someone to mass murder their class mates will at worst be sucking compensation out of some indie developer with no assets instead of OpenAI, let alone Microsoft. If we get the model for liability right, all these problems solve themselves. Yes it will shut down certain exploitations of technology but it won't shut down the more general harmless use which is important for us to engage in to understand and improve the tech.
  - danShumway 4y ago
    I don't mean it's easy to do, I mean they're doing it. Google is literally announcing a cloud service that uses an LLM to analyze if code snippets are safe. This is an obviously bad idea and it's not theoretical, it already exists. I don't necessarily disagree that it was a leap, but it's a leap we've taken now.
    - zmmmmm 4y ago
      I think I mostly am agreeing with you but I see it through a liability lens. When Google's cloud service to analyze code snippets screws up, nothing in the ToU should alleviate Google from the responsibility they should rightly bear if they ship a system with known and understood flaws that are not clearly portrayed to the user (and I don't mean buried in the middle of a paragraph in page 36 of the ToU). If forcing Google to accept liability kills the service dead then good - it's probably the right thing until they can reach an acceptable level of safety that they are willing to accept the resultant risk.

by zmmmmm 4y ago

3np 4y ago

A technology-agnostic approach I'd favor would be some regulation roughly along the lines of: "Every business decision on behalf of a business needs to be signed off by a responsible individual" If you want automated software doing things on your behalf, sure, but every action it takes needs to be attributable to an accountable individual. Be it an engineer, an executive, or an officer. Doesn't matter if the "other entity" is an LLM, a smart contract, or an outsourced worker in a sweatshop through 5 levels of subcontracting. If you make a machine and it causes a mess, that's on you. If others start using your machine unsupervised and it makes their lives a mess, that's on them (and potentially you).

zmmmmm 4y ago
The key is that it has to be resistant to subversion by hostile terms of use. Similar to how certain provisions in employment contracts aren't enforceable under employment law. Because literally anything you try and establish here will instantly end up as a waiver in terms of use for services that regular people can't possibly understand or reasonably opt out of. (as example, see recent law suit that Tesla won because the driver used auto-pilot on "city streets" where it was advised not to somewhere deep in the terms of use).
- 3np 4y ago
  Depending on the jurisdiction, such waivers can be void since certain rights or responsibilities can not be signed away.
chefandy 4y ago
That doesn't work. For that to work, the person needs to understand how that algorithm creates its output, and understand its flaws and vulnerabilities, AND be diligent about interrogating the results with those things in mind. Nobody technically sophisticated enough to do that will also have the domain knowledge to evaluate the most consequential decisions. For example, sentencing "recommendations" are supposed to be exactly that-- recommendations for judges. But, judges seem to rubber stamp the recommendations. I'm sure for some it's a scapegoat in case someone accuses them of not really thinking about it, the more credulous probably assume the algorithm saw something they didn't, and for others, the influence might be more subtle. This is something we should have studied before we started letting this algorithm put people in jail. These are judges. Their most important function is impartiality.
- 3np 4y ago
  With enough incentives in place, decision-makers would do the necessary to make it work.
  - chefandy 4y ago
    Do you have specifics? A lot of people say things like that when they're toe-to-toe with human psychology but humanity still has a whole lot of problems that a whole lot of people are pretty heavily incentivized to avoid. I don't see how this would be any different.

deleted 4y ago

[deleted]

danShumway 4y ago

> You're joining dots from LLM's producing text output that humans read to them being linked to autonomously taking actions by themselves. That's a huge leap. No, it's not. It used to be a huge leap until OpenAI started advertising plugin support and Plantr started advertising using it to interpret drone footage. "We won't wire it to anything important" was a good argument, but that ship is rapidly sailing now.

zmmmmm 4y ago
just because it's easy to do doesn't mean it isn't a huge logical leap. What concerns me most is the outsourcing of liability going on. Which is already what OpenAI is largely doing - outsourcing Microsoft's liability for releasing this tech into the wild to a company that can afford to write off the legal risk. Now OpenAI is outsourcing the legal risk for what ChatGPT does to plugin developers. So the first chatbot that convinces someone to mass murder their class mates will at worst be sucking compensation out of some indie developer with no assets instead of OpenAI, let alone Microsoft. If we get the model for liability right, all these problems solve themselves. Yes it will shut down certain exploitations of technology but it won't shut down the more general harmless use which is important for us to engage in to understand and improve the tech.
- danShumway 4y ago
  I don't mean it's easy to do, I mean they're doing it. Google is literally announcing a cloud service that uses an LLM to analyze if code snippets are safe. This is an obviously bad idea and it's not theoretical, it already exists. I don't necessarily disagree that it was a leap, but it's a leap we've taken now.
  - zmmmmm 4y ago
    I think I mostly am agreeing with you but I see it through a liability lens. When Google's cloud service to analyze code snippets screws up, nothing in the ToU should alleviate Google from the responsibility they should rightly bear if they ship a system with known and understood flaws that are not clearly portrayed to the user (and I don't mean buried in the middle of a paragraph in page 36 of the ToU). If forcing Google to accept liability kills the service dead then good - it's probably the right thing until they can reach an acceptable level of safety that they are willing to accept the resultant risk.