3 ms·

It seems like it's impossible to prove that it's safe on public roads without actually using it on public roads, though? Wouldn't that be an impossible ask, by

It seems like it's impossible to prove that it's safe on public roads without actually using it on public roads, though? Wouldn't that be an impossible ask, by definition?

kibwen 4y ago
It is entirely possible to test things under controlled trials to simulate real-world conditions. To use an analogy, we don't just test new medicines by putting them on the shelves and waiting for consumers to report side effects. We have strenuous trials first to give us baseline confidence in the product.
- try_the_bass 4y ago
  I would argue that it's impossible to simulate real-world conditions for a task as complex as driving. There are too many edge cases and different combinations of environmental factors, many of which involve the inherent unpredictability of other drivers. I would assume medicines have similar or higher levels of complexity involved. However, the medicine analogy is poor. We often release medicines for use even when they have incredibly harmful side-effects. We do the testing to quantify and understand those side-effects, not prove that each medicine does exactly what it should with no side effects.
cptcobalt 4y ago
Exactly. There's no better facsimilie for the sheer variety of public roads than, well, public roads. Tesla even has rigorous simulation in place for hostile situations that should never occur on public roads: A good general overview from two minute papers: https://www.youtube.com/watch?v=6hkiTejoyms https://www.youtube.com/watch?v=6hkiTejoyms Tesla's AI Day 2022 video, timestamped around labeling and simulation https://youtu.be/ODSJsviD_SU?t=6037 https://youtu.be/ODSJsviD_SU?t=6037
kaba0 4y ago
I still see no reason why should a private company be allowed to risk the lives of those who never willingly participated in their study. There are human trials for drugs as well, but those are only the latest phase, and with explicit permission.
- try_the_bass 4y ago
  Isn't the usage of Autopilot "willing participation"? Or are you referring to the other people on the road? Wouldn't that also be true of your medicine analogy? If someone takes an experimental medicine but experiences an unknown side effect and passes out while driving, isn't the risk profile equivalent? That hypothetical certainly is far less probable than the Autopilot case, but for any experiment that takes place outside controlled conditions, there can always be unknown and unforeseen consequences that affect people who have not given explicit consent to being part of said experiment. To argue that we cannot test things unless all potentially-involved parties have given consent is to argue for an impossibility. There is always some small amount of risk of higher-order consequences. I would even go further and say that arguing for such is an unnecessarily conservative approach that hamstrings any attempt at improving the world. See, for example, this commentary on the current state of IRBs [1] [1] https://astralcodexten.substack.com/p/book-review-from-oversight-to-overkill https://astralcodexten.substack.com/p/book-review-from-overs...
  - kaba0 4y ago
    There are plenty of drugs after which you can’t drive. Also, the “danger to others” aspect is even then negligible as opposed to some faulty software controlling a ton of steel/battery in a system that depends heavily on predictable behavior of each other.
    - try_the_bass 3y ago
      Safe driving for a human also depends heavily on predictable behavior of others. If that's an important metric, it seems like the optimal solution is a single self-driving system with a monopoly on the market... that way each and every driver is predictable, correct? I guess I don't understand your argument. We put faulty humans in control of a ton of steel/battery in the same system all the time; humans have the disadvantage that when lessons are learned, they aren't transferrable without great amounts of effort. Self-driving systems can share learnings across the whole fleet with software updates, which seems like a strictly better solution in the long term. It seems like you're making perfect the enemy of "better".

It seems like it's impossible to prove that it's safe on public roads without actually using it on public roads, though? Wouldn't that be an impossible ask, by definition?

kibwen 4y ago

It is entirely possible to test things under controlled trials to simulate real-world conditions. To use an analogy, we don't just test new medicines by putting them on the shelves and waiting for consumers to report side effects. We have strenuous trials first to give us baseline confidence in the product.

try_the_bass 4y ago
I would argue that it's impossible to simulate real-world conditions for a task as complex as driving. There are too many edge cases and different combinations of environmental factors, many of which involve the inherent unpredictability of other drivers. I would assume medicines have similar or higher levels of complexity involved. However, the medicine analogy is poor. We often release medicines for use even when they have incredibly harmful side-effects. We do the testing to quantify and understand those side-effects, not prove that each medicine does exactly what it should with no side effects.

cptcobalt 4y ago

Exactly. There's no better facsimilie for the sheer variety of public roads than, well, public roads. Tesla even has rigorous simulation in place for hostile situations that should never occur on public roads: A good general overview from two minute papers: https://www.youtube.com/watch?v=6hkiTejoyms https://www.youtube.com/watch?v=6hkiTejoyms Tesla's AI Day 2022 video, timestamped around labeling and simulation https://youtu.be/ODSJsviD_SU?t=6037 https://youtu.be/ODSJsviD_SU?t=6037

kaba0 4y ago

I still see no reason why should a private company be allowed to risk the lives of those who never willingly participated in their study. There are human trials for drugs as well, but those are only the latest phase, and with explicit permission.

try_the_bass 4y ago
Isn't the usage of Autopilot "willing participation"? Or are you referring to the other people on the road? Wouldn't that also be true of your medicine analogy? If someone takes an experimental medicine but experiences an unknown side effect and passes out while driving, isn't the risk profile equivalent? That hypothetical certainly is far less probable than the Autopilot case, but for any experiment that takes place outside controlled conditions, there can always be unknown and unforeseen consequences that affect people who have not given explicit consent to being part of said experiment. To argue that we cannot test things unless all potentially-involved parties have given consent is to argue for an impossibility. There is always some small amount of risk of higher-order consequences. I would even go further and say that arguing for such is an unnecessarily conservative approach that hamstrings any attempt at improving the world. See, for example, this commentary on the current state of IRBs [1] [1] https://astralcodexten.substack.com/p/book-review-from-oversight-to-overkill https://astralcodexten.substack.com/p/book-review-from-overs...
- kaba0 4y ago
  There are plenty of drugs after which you can’t drive. Also, the “danger to others” aspect is even then negligible as opposed to some faulty software controlling a ton of steel/battery in a system that depends heavily on predictable behavior of each other.
  - try_the_bass 3y ago
    Safe driving for a human also depends heavily on predictable behavior of others. If that's an important metric, it seems like the optimal solution is a single self-driving system with a monopoly on the market... that way each and every driver is predictable, correct? I guess I don't understand your argument. We put faulty humans in control of a ton of steel/battery in the same system all the time; humans have the disadvantage that when lessons are learned, they aren't transferrable without great amounts of effort. Self-driving systems can share learnings across the whole fleet with software updates, which seems like a strictly better solution in the long term. It seems like you're making perfect the enemy of "better".