5 ms·

> I think what it's really trying to say is, "You can't pass laws saying overseas companies have to share their source code if they want to sell software or Saa

> I think what it's really trying to say is, "You can't pass laws saying overseas companies have to share their source code if they want to sell software or SaaS in your country." This is terrifying. Foreign-made self-driving cars can deliberately kill people via software algorithms, and even the government can't insist on seeing the source code to find out whether the car maker is incompetent or malicious.

bunderbunder 8y ago
I see nothing to suggest that this provision would prohibit a subpoena.
kentm 8y ago
There’s a difference between requiring source code as a precondition for selling and stopping sales of a product on documented safety concerns and requiring participation in an investigation to resume sales. It’s down to whether the law is correctly written to distinguish the two.
- yellowapple 8y ago
  Personally, I'd much prefer proactive auditing (i.e. requiring source code) over waiting until someone is injured and possibly killed to launch an investigation. Transparency is a dependency of trust. There's no going around that. If a product is not transparent, then it is not trustworthy, period. I'm not at all inclined to wait for something to violate my trust further before demanding a prerequisite for that trust. Governments shouldn't be so inclined, either, especially when it comes to human safety.
  - adrianN 8y ago
    I'd also like that, but even today there is lots of software that can kill people if it contains bugs. It is usually enough that the makers can prove that they followed some process like EN50128. Source code is not required. I don't think there are competent people available for auditing source code.
    - gregknicholson 8y ago
      I agree. I just don't think we should collectively allow companies to hide their source code and still be considered trustworthy. — especially for innovative safety algorithms that could save the lives of their competitors' customers, if only the code was reusable. Competent auditors will emerge if there's demand. There were once no competent astronauts!
    - yellowapple 8y ago
      "I don't think there are competent people available for auditing source code." Probably not, but by requiring companies to disclose source code on request, effective independent audit of that code is merely improbable versus effectively impossible.
pc86 8y ago
I'd rather have an algorithm deliberately kill someone 1 in 1,000 cases than have distraction, negligence, and poor reaction times of a human driver accidentally kill someone 1 in 100.
- deleted 8y ago
  [deleted]
- mcguire 8y ago
  Hypothetical example: Entities in Country A provide entities in Country B with software for, say, self-driving cars. Such cars become sufficently popular in Country B. ("Sufficiently" being read as numerous enough for the next step to become important.) Country A and Country B come into conflict. Potential causes: resources (oil, fresh water, rare earths, whatever), ideology, etc. Country A uses its legal (and extra-legal) powers to force the entities providing said software to push out an over-the-air update of the software. When Country A flips a switch, the self-driving cars begin moving offensively, attempting to kill passengers and pedestrians and jam the road infrastructure, doing as much economic and political damage as possible. (Whoo; suppose they target specific people and infrastructure.) It's an entertaining scenario for your favorite TV series or something, but possibly more troubling for software controlling information networks or the power grid. And the proof-of-concept of such an attack has already been shown to be successful. P.S. We are both assuming that self-driving cars are better than human-driven cars, which fact is not currently in evidence.
- simion314 8y ago
  Possible issues with your ideas: - false numbers, similar with Tesla PR where they use bad statistics to prove their system is safer - too general statistics for your situation, say car X is safer then human globally but locally in your region humans are much safer, will you send your children with car X where is probably not safer then average human(including drunks, teens and tired people) - say car X is super safe, so you allow them to kill some people deliberately because they saved lives, so CEO of X hates people from a specific group(say gun owners, or teachers, or developers) so he decides that on 13 every month 666 people will be sacrificed

bunderbunder 8y ago

I see nothing to suggest that this provision would prohibit a subpoena.

kentm 8y ago

There’s a difference between requiring source code as a precondition for selling and stopping sales of a product on documented safety concerns and requiring participation in an investigation to resume sales. It’s down to whether the law is correctly written to distinguish the two.

yellowapple 8y ago
Personally, I'd much prefer proactive auditing (i.e. requiring source code) over waiting until someone is injured and possibly killed to launch an investigation. Transparency is a dependency of trust. There's no going around that. If a product is not transparent, then it is not trustworthy, period. I'm not at all inclined to wait for something to violate my trust further before demanding a prerequisite for that trust. Governments shouldn't be so inclined, either, especially when it comes to human safety.
- adrianN 8y ago
  I'd also like that, but even today there is lots of software that can kill people if it contains bugs. It is usually enough that the makers can prove that they followed some process like EN50128. Source code is not required. I don't think there are competent people available for auditing source code.
  - gregknicholson 8y ago
    I agree. I just don't think we should collectively allow companies to hide their source code and still be considered trustworthy. — especially for innovative safety algorithms that could save the lives of their competitors' customers, if only the code was reusable. Competent auditors will emerge if there's demand. There were once no competent astronauts!
  - yellowapple 8y ago
    "I don't think there are competent people available for auditing source code." Probably not, but by requiring companies to disclose source code on request, effective independent audit of that code is merely improbable versus effectively impossible.

pc86 8y ago

I'd rather have an algorithm deliberately kill someone 1 in 1,000 cases than have distraction, negligence, and poor reaction times of a human driver accidentally kill someone 1 in 100.

deleted 8y ago
[deleted]
mcguire 8y ago
Hypothetical example: Entities in Country A provide entities in Country B with software for, say, self-driving cars. Such cars become sufficently popular in Country B. ("Sufficiently" being read as numerous enough for the next step to become important.) Country A and Country B come into conflict. Potential causes: resources (oil, fresh water, rare earths, whatever), ideology, etc. Country A uses its legal (and extra-legal) powers to force the entities providing said software to push out an over-the-air update of the software. When Country A flips a switch, the self-driving cars begin moving offensively, attempting to kill passengers and pedestrians and jam the road infrastructure, doing as much economic and political damage as possible. (Whoo; suppose they target specific people and infrastructure.) It's an entertaining scenario for your favorite TV series or something, but possibly more troubling for software controlling information networks or the power grid. And the proof-of-concept of such an attack has already been shown to be successful. P.S. We are both assuming that self-driving cars are better than human-driven cars, which fact is not currently in evidence.
simion314 8y ago
Possible issues with your ideas: - false numbers, similar with Tesla PR where they use bad statistics to prove their system is safer - too general statistics for your situation, say car X is safer then human globally but locally in your region humans are much safer, will you send your children with car X where is probably not safer then average human(including drunks, teens and tired people) - say car X is super safe, so you allow them to kill some people deliberately because they saved lives, so CEO of X hates people from a specific group(say gun owners, or teachers, or developers) so he decides that on 13 every month 666 people will be sacrificed