4 ms·

> But don't you think that there should be regulation saying that delegating decisions to a technology system What is a "decision"? My computer is making billi

> But don't you think that there should be regulation saying that delegating decisions to a technology system What is a "decision"? My computer is making billions of decisions per second. If nothing else, I could see this crippling FOSS. No one is going to want to be "responsible" to some EU bureaucrat pursuing a personal political agenda, certainly not for work you've done for free.

moritzwarhier 3y ago
Yeah I wanted to expand but got tired: Of course any computation can be considered a decision in some way. But this is conflating a narrow and IT-specific meaning of that word with legalese and philosophy, or my intended meaning of the word. This reminds me of the ancient catastrophic Therac-25 software bug, and other such cases. Maybe this case is a good example for thinking about what part of the responsibility is on the side of the operator, apart from the obvious failure of the implementer. For more modern examples involving actual ML, look at Meta, TikTok and their recommendations: where is the line to draw, what excuses are allowed, when the outcome is obviously negative, and the algorithm claims to fulfill a goal? It doesn't matter if it's a rule-based system without "intelligence" or ML. What matters is the responsibility of the human operator. And making assumptions about correctness. Humans make egregious errors as well, but the kind of errors AI causes are a significant concern where current legislation is insufficient. It's one thing to have a bug in your airplane controller code or whatever. It's another to knowingly accept malicious errors, either unpredictable or even intentional, but without proper responsibility?
- candiodari 3y ago
  > It's another to knowingly accept malicious errors, either unpredictable or even intentional, but without proper responsibility? Is this law making the users of software responsible (e.g. law enforcement, government departments) ? It seems to me to make authors of software responsible, absolving the users. Not the other way around. They keep the law secret, of course, so anyone can claim what they want, but the article talks directly about the accountability of OpenAI. It seems to focus on rules on authors of AI software, presumably to mostly absolve governments and users of that software. "Rules around generative AI", "transparency requirements for any developer of a large language model" ... nothing that would make governments responsible for abusing AI software. Which is strange, because that's the concern the last paragraph of the article focuses on. I must say ... this "complete ban" voted by the EU parliament last spring, doesn't seem to have stopped governments from using live facial recognition [1]. [1] https://www.euronews.com/next/2023/02/21/new-french-facial-recognition-reveals-identity-in-a-click-ahead-of-paris-2024-olympics https://www.euronews.com/next/2023/02/21/new-french-facial-r...
  - troupo 3y ago
    > It seems to me to make authors of software responsible, absolving the users. Only the authors of foundational models. > They keep the law secret, of course You mean this secret law: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52021PC0206 https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52... that people have actually read https://softwarecrisis.dev/letters/the-truth-about-the-eu-act/ https://softwarecrisis.dev/letters/the-truth-about-the-eu-ac... ?
    - soulofmischief 3y ago
      > Developers (not deployers) of foundation models need to register their models, with documentation, prior to making it available on the market or as a service. No, thanks. Utterly insane, we are sliding backwards into the "code is a munitions export" territory that I so hoped we'd escaped after the 90s. > If you take a foundation model, fine-tune it for a specialised purpose, and deploy it as a part of your software, it won’t count as a foundation model, and you’ll probably be fine, as long as the original provider of the foundation model was compliant. This interpretation would render the whole exercise pointless, there is quite a blurry line between "fine-tuning" and "training".
      - troupo 3y ago
        > Utterly insane, we are sliding backwards into the "code is a munitions export" Of course not. This goes hand in hand with such things as "being accountable". You can't spit out a black box that no one knows what it does and how it was trained? Too bad for you. Perhaps this will lead to fewer things like this: https://news.ycombinator.com/item?id=38595751 https://news.ycombinator.com/item?id=38595751
        soulofmischief 3y ago
        > You can't spit out a black box that no one knows what it does and how it was trained? What if I want to? Who on earth has the ethical authority to claim the right to control my ability to release a set of weights? Only what is done with those weights should be legislated, and extremely conservatively. Maybe a lot of software developers got into this career for the money, or because they like solving problems. But for me, politics inseparable from software engineering. Politics are why I devoted myself to the craft. This is the exact kind of situation in which my knowledge and skill become tools of protest.
Tainnor 3y ago
Nobody cares about your CMS, online shop, code collaboration tool, etc. If your software however makes life-altering decisions for its users - such as targeting people for investigation, deciding on asylum requests, etc. - then you have to be responsible for the decisions your software makes. You can't hide behind "I don't know how the AI works, but it decided that person X is likely a criminal" - nope, you need to be able to explain the reasoning process behind this, because the system is your responsibility.
- deleted 3y ago
  [deleted]
- deleted 3y ago
  [deleted]
troupo 3y ago
> If nothing else, I could see this crippling FOSS. If anything, this should strengthen FOSS, because one of stipulations is "document your foundational models and training sets"

moritzwarhier 3y ago

Yeah I wanted to expand but got tired: Of course any computation can be considered a decision in some way. But this is conflating a narrow and IT-specific meaning of that word with legalese and philosophy, or my intended meaning of the word. This reminds me of the ancient catastrophic Therac-25 software bug, and other such cases. Maybe this case is a good example for thinking about what part of the responsibility is on the side of the operator, apart from the obvious failure of the implementer. For more modern examples involving actual ML, look at Meta, TikTok and their recommendations: where is the line to draw, what excuses are allowed, when the outcome is obviously negative, and the algorithm claims to fulfill a goal? It doesn't matter if it's a rule-based system without "intelligence" or ML. What matters is the responsibility of the human operator. And making assumptions about correctness. Humans make egregious errors as well, but the kind of errors AI causes are a significant concern where current legislation is insufficient. It's one thing to have a bug in your airplane controller code or whatever. It's another to knowingly accept malicious errors, either unpredictable or even intentional, but without proper responsibility?

candiodari 3y ago
> It's another to knowingly accept malicious errors, either unpredictable or even intentional, but without proper responsibility? Is this law making the users of software responsible (e.g. law enforcement, government departments) ? It seems to me to make authors of software responsible, absolving the users. Not the other way around. They keep the law secret, of course, so anyone can claim what they want, but the article talks directly about the accountability of OpenAI. It seems to focus on rules on authors of AI software, presumably to mostly absolve governments and users of that software. "Rules around generative AI", "transparency requirements for any developer of a large language model" ... nothing that would make governments responsible for abusing AI software. Which is strange, because that's the concern the last paragraph of the article focuses on. I must say ... this "complete ban" voted by the EU parliament last spring, doesn't seem to have stopped governments from using live facial recognition [1]. [1] https://www.euronews.com/next/2023/02/21/new-french-facial-recognition-reveals-identity-in-a-click-ahead-of-paris-2024-olympics https://www.euronews.com/next/2023/02/21/new-french-facial-r...
- troupo 3y ago
  > It seems to me to make authors of software responsible, absolving the users. Only the authors of foundational models. > They keep the law secret, of course You mean this secret law: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52021PC0206 https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52... that people have actually read https://softwarecrisis.dev/letters/the-truth-about-the-eu-act/ https://softwarecrisis.dev/letters/the-truth-about-the-eu-ac... ?
  - soulofmischief 3y ago
    > Developers (not deployers) of foundation models need to register their models, with documentation, prior to making it available on the market or as a service. No, thanks. Utterly insane, we are sliding backwards into the "code is a munitions export" territory that I so hoped we'd escaped after the 90s. > If you take a foundation model, fine-tune it for a specialised purpose, and deploy it as a part of your software, it won’t count as a foundation model, and you’ll probably be fine, as long as the original provider of the foundation model was compliant. This interpretation would render the whole exercise pointless, there is quite a blurry line between "fine-tuning" and "training".
    - troupo 3y ago
      > Utterly insane, we are sliding backwards into the "code is a munitions export" Of course not. This goes hand in hand with such things as "being accountable". You can't spit out a black box that no one knows what it does and how it was trained? Too bad for you. Perhaps this will lead to fewer things like this: https://news.ycombinator.com/item?id=38595751 https://news.ycombinator.com/item?id=38595751
      - soulofmischief 3y ago
        > You can't spit out a black box that no one knows what it does and how it was trained? What if I want to? Who on earth has the ethical authority to claim the right to control my ability to release a set of weights? Only what is done with those weights should be legislated, and extremely conservatively. Maybe a lot of software developers got into this career for the money, or because they like solving problems. But for me, politics inseparable from software engineering. Politics are why I devoted myself to the craft. This is the exact kind of situation in which my knowledge and skill become tools of protest.

Tainnor 3y ago

Nobody cares about your CMS, online shop, code collaboration tool, etc. If your software however makes life-altering decisions for its users - such as targeting people for investigation, deciding on asylum requests, etc. - then you have to be responsible for the decisions your software makes. You can't hide behind "I don't know how the AI works, but it decided that person X is likely a criminal" - nope, you need to be able to explain the reasoning process behind this, because the system is your responsibility.

deleted 3y ago
[deleted]
deleted 3y ago
[deleted]

troupo 3y ago

> If nothing else, I could see this crippling FOSS. If anything, this should strengthen FOSS, because one of stipulations is "document your foundational models and training sets"