8 ms·

The irony is that this is actually a good reason to drag the car manufacturers to the table and compel them to fix it from their end. But pragmatically, they l

by kwdc 6y ago

The irony is that this is actually a good reason to drag the car manufacturers to the table and compel them to fix it from their end. But pragmatically, they likely don't answer the phone. Or email. So, it is still easier to fix the road. And hope the issue doesn't resurface elsewhere and inadvertantly injure or kill too many people.

mhh__ 6y ago
> The irony is that this is actually a good reason to drag the car manufacturers to the table and compel them to fix it from their end. The solution is having the balls to lay down good policy/regulation. There is no point forcing car companies to use a certain algorithm, for example, but it's not unreasonable to build a standard basket of problematic road signs which the car must pass - e.g. a cattle grid isn't too bad, but if a driverless car doesn't stop for a speed limit outside a school the company should have some liability.
- p1necone 6y ago
  I feel the "driver" of the driverless car should be responsible too. We are nowhere near actual full unsupervised autonomy - people should still be actively looking out for the car doing the wrong thing. You don't get to abdicate responsibility on the road just because you bought in to some corporations marketing.
  - mhh__ 6y ago
    Absolutely true, although the corporate liability thing really depends on whether the engineering takes after software or aviation (to pick two widely contrasting examples).
    - bigiain 6y ago
      And I notice the "phone with wheels" built by mostly software engineers chooses to use words with well known meanings from the aviation industry without a) any of the rigour behind the regulations and certifications or b) much in the way of acknowledging the differences between the challenges of an "autopilot" safely managing to traverse the airspace 37,000 feet above Hawaii, compared to a rural lane in the UK (or 280 between the valley and the city). (While, sadly, at the same time it seems from the outside that the ACAS code got written by the same outsourced dev team that work on their website the rest of the time. :sigh: )
  - Swizec 6y ago
    non autonomous cars have driver assists now that slam on the brakes when they think you’re about to crash. Statistically a huge improvement to active safety, and then there’s cattle grids. Imagine you’re driving along in full control on an empty country road. Out of nowhere your car panics, slams in the brakes, deploys seatbelt tensioners, and flashes bright warning lights because clearly you fell asleep or something if you cant see that big wall. Would you react? Maybe swerve coz you’re startled?
  - bigiain 6y ago
    You don't get to abdicate responsibility for your vehicle's "autopilot" just because that was only corporate marketing. Words mean things. You don't get to say they mean different things when you say them just because it helps you sell more cars. Not in the fine print, and not in a court.
  - tremon 6y ago
    There's no way a "driver" can autocorrect for a vehicle that makes an emergency stop out of the blue -- the vehicle is still stopped, and now poses a safety hazard for the surrounding traffic.
- bigiain 6y ago
  I'd go further than that. If a vehicle's automated systems caused or contributed to an accident or a road rule violation, the car manufacturer should be held responsible. If your company's automated emergency braking, or lane assist, or adaptive cruise control - causes or contributes to a problem, someone from that company should be in court next to the driver facing judgement for the consequences. (I'd even go so far as to suggest if the existence of a system leads to a situation where an incident occurred because of the driver's assumption that the automated system would prevent it, then the company should need to justify why the driver should be held responsible instead of the software. If your advertising claims "autopilot!", you're going to be apportioned a significant portion of liability even if your fine print says "Drivers must keep alert and ready to take control at all times" - you should be judged based on a reasonable persons expectation from the language you use to promote your vehicles, not some w4easel words in your T&Cs.)
  - nickff 6y ago
    Automation issues may fall into the category of manufacturer/developer negligence; you seem to be suggesting some sort of burden shifting. When you say that a manufacturer should be held liable if their system '... contributes to a problem...', it seems like you're suggesting adoption of the (notoriously problematic) 'if not but for' test. This sort of test is used in negligence cases, but can be useless, as almost anything satisfies it (i.e. slow service at a coffee shop may have been a necessary condition for the timing of that exact accident).
  - HALtheWise 6y ago
    Unfortunately, pretty much every automobile safety improvement in the last century (which collectively have saved ~millions of lives) also occasionally kills people. Airbags can propel objects into people, seatbelts can trap someone in a burning car, antilock brakes perform worse on sand, etc. In practice, putting the responsibilty on auto manufacturers for every specific injury or death that happens due to a safety system, without somehow giving them "positive credit" for lives saved, massively delays or entirely prevents these sort of advancements from being made. We definitely need processes to verify that new features are actually functional, typically through recalls and product liability suits, but simply putting blame on manufacturers for all accidents is a terrible idea.
- akira2501 6y ago
  > but if a driverless car doesn't stop for a speed limit outside a school the company should have some liability. This is a particular challenge in California because our school zone signs specify that the limit only applies "when children are present."
  - taneq 6y ago
    How are you meant to know whether children are present in time if you're not doing the lower speed limit?
    - colejohnson66 6y ago
      The idea is that you would be going slow enough already (and if not, slow down). Then, if there’s no children, you can speed up.
    - aidenn0 6y ago
      It weirded me out when I moved here, but in practice, it's enforced like anywhere else: at the beginning and end of school days. Since a large fraction of kids are driven to school (at least in southern California), you know that this is happening because of the massive traffic jam near the school.
    - jimktrains2 6y ago
      Illinois has these "no turn on red when pedestrians present" signs and I have no clue how thats supposed to help anyone because pedestrians usually get hit because they aren't seen by the car.
  - csunbird 6y ago
    In Germany, when a speed limit of 30kmh is introduced because of a school, it is usually on applicable for school days and hours (e.g. Mo-Fri 8-18) only. After school hours or non school days you are allowed to drive at normal speed of 50kmh.
- sideshowb 6y ago
  > it's not unreasonable to build a standard basket of problematic road signs which the car must pass I'm wondering how you would get around metaparameter optimization here, or the fact that it's too easy to optimize to a known test procedure rather than the real world?
bigiain 6y ago
> But pragmatically, they likely don't answer the phone. Or email. > So, it is still easier to ... hold them responsible in court for damage/injury/deaths caused by the actions their vehicles take which are not under control of the driver.
- bumby 6y ago
  I prefer proactive policy. As much as I would have liked to see more people go to jail for the 2009 financial crisis, for example, I’d prefer to see smarter regulation to have prevented it in the first place
  - bluGill 6y ago
    Regulations are too complex to get right like that. You need to figure out how to enforce following the spirit of them not the letter or you will forever be seeing more crises that you didn't foresee.
    - bumby 6y ago
      I think that’s the ideal, but tends to break down in practice. The problem with expecting the spirit of the law to be the compass is that it opens up a lot for interpretation. The end result is that it incentivizes gamesmanship where the organization with the best lawyers win
  - bigiain 6y ago
    While I agree - I reckon it's far past time that we pierce the outdated "corporate personhood" idea, and let Tesla/Uber/Apple/Waymo et al. know that society has chosen to hold their "corporate personhood" responsible in the same way people are held responsible. If your corporation (via it's self driving software) kills someone, the corporation will "go to jail", a C level exec does the jailtime and the entire company will be unable to carry on any of it's normal business for the duration of the sentence (while still being held responsible for it's obligations to staff/suppliers/cerditors). If that self driving software is judged to be liable for a traffic infringement, the corporation will pay an amount equal to the percentage of income that a human driver would have paid. If it's an infringement with a ~$400 fine, the corporation should pay ~0.5% of their annual income - the same sort of penalty as "an average salary" person. A couple of $20mil fines and a 6 month total company shutdown would be very "proactive" in encouraging Tesla to get Elon to shut the fuck up with his big bold claims that cannot be backed up. If Uber had been slapped with the "corporate personhood" equivalent of whatever penalty a human driver who'd killed that person crossing the road with a bicycle they'd be far more circumspect about using minimum wage contractors as "safety drivers" in their prototypes. (I can dream, right?)
toast0 6y ago
In the small, it's dramatically more cost effective to fix the road. Ignoring cost to develop and test, just the cost to deploy a fix to all the affected cars would probably be a lot more than £70,000. In the large, it's always a shame to fix a good test case.
colechristensen 6y ago
Or simply decertify all of the vehicles that behave incorrectly from driving on public roads. This is the kind of issue which will persist, vehicles which fail to recognize a real world situation shouldn’t get the benefit of changing the world to fit them. An inquiry should be done into how these vehicles came to fail to recognize this road feature and the upstream cause (not just the bug by the systematic issues in the organizations which produced vehicles with this bug) should be fixed before they’re rectified to operate on roads.
- kwdc 6y ago
  I like the idea of a warning that particular models of cars are not permitted past this point for safety reasons. That by itself is enough for some frustrated driver to see the sign, take a photo and upload it to social media where those car manufacturers can be suitably laughed at and ridiculed. The good ones would then sort out why their braking system is faulty. Perhaps it gets confused about other things in other serious ways.
  - Arnt 6y ago
    I can't tell from the article whether the automatic braking happens when a sudden incline, decline or both is detected in front of the car. It was one of those three, though, that much is clear. Do you really regard braking in front of a sudden incline/decline as ridiculous?
    - fuzxi 6y ago
      It seems like there was a steep hill, but the cattle gate was level, and the sudden, extreme difference in gradient is what caused the sudden braking. Although from the photos, it all looks fairly flat to me.
      - Arnt 6y ago
        Yes, that's my impression too, and braking seems sensible. From uphill: "There's something weird in front of the car, best drive very slowly". From downhill: "WTF I can't see the road any more, best drive very slowly." Both seem quite sensible to me, I don't understand why so many people on the thread seem to disagree.
rlpb 6y ago
For unusual fixed features like this, perhaps car manufacturers should start putting exceptions into their navigation databases, rather than tweaking their automatic detection. You generally don't want to be driving over cattle grids, with their sudden gradient changes, at high speed anyway. So tell the car to slow down as it approaches (or warn the driver loudly to slow down), and briefly disable automatic braking. If it's a rare event, it'll have the drivers' attention. I see the future of self-driving cars to be one where highway authorities map and sign features in a way that is designed for self-driving cars to be able to parse, rather than requiring them to heuristically sense as much as they do today. This would be a step down the path that's inevitable anyway.
- netsharc 6y ago
  Giant QR codes on highway signs? Haha, an attack by spoofing them would be interesting. Next trick: QR codes with cryptographic signatures, and a huge traffic jam when the traffic authorities forget to renew their certificate...
- HPsquared 6y ago
  Having a database of known trouble spots would certainly serve as defense in depth, but would only be secondary.
rkangel 6y ago
While the question of who pays for it is important, we should be focusing on the best way to get safe driving. If car sensors and algorithms can get to 99.99% of road features, then changing the remaining 0.01% to work with modern car technologies seems like a reasonable use of resource. No algorithm is ever going to be completely perfect.
- Hamuko 6y ago
  I'm more worried about the roads that can't be changed.
- rtx 6y ago
  What happens if there are bugs or we find code being not up to the mark? Should all the models using that version be disabled till OTA updates are rolled out. For reference bugs in Boeing's plane resulted in the whole fleet being grounded.
  - fuzxi 6y ago
    I don't see a problem with autonomous vehicles having a manual control mode, unless we're talking about a scenario some decades in the future where you don't actually need a driver's license for an autonomous vehicle.
narcissismo 6y ago
As a programmer, I find it comforting to hear that people are prepared to terraform in order to cancel out my bugs.

by kwdc 6y ago

mhh__ 6y ago

> The irony is that this is actually a good reason to drag the car manufacturers to the table and compel them to fix it from their end. The solution is having the balls to lay down good policy/regulation. There is no point forcing car companies to use a certain algorithm, for example, but it's not unreasonable to build a standard basket of problematic road signs which the car must pass - e.g. a cattle grid isn't too bad, but if a driverless car doesn't stop for a speed limit outside a school the company should have some liability.

p1necone 6y ago
I feel the "driver" of the driverless car should be responsible too. We are nowhere near actual full unsupervised autonomy - people should still be actively looking out for the car doing the wrong thing. You don't get to abdicate responsibility on the road just because you bought in to some corporations marketing.
- mhh__ 6y ago
  Absolutely true, although the corporate liability thing really depends on whether the engineering takes after software or aviation (to pick two widely contrasting examples).
  - bigiain 6y ago
    And I notice the "phone with wheels" built by mostly software engineers chooses to use words with well known meanings from the aviation industry without a) any of the rigour behind the regulations and certifications or b) much in the way of acknowledging the differences between the challenges of an "autopilot" safely managing to traverse the airspace 37,000 feet above Hawaii, compared to a rural lane in the UK (or 280 between the valley and the city). (While, sadly, at the same time it seems from the outside that the ACAS code got written by the same outsourced dev team that work on their website the rest of the time. :sigh: )
- Swizec 6y ago
  non autonomous cars have driver assists now that slam on the brakes when they think you’re about to crash. Statistically a huge improvement to active safety, and then there’s cattle grids. Imagine you’re driving along in full control on an empty country road. Out of nowhere your car panics, slams in the brakes, deploys seatbelt tensioners, and flashes bright warning lights because clearly you fell asleep or something if you cant see that big wall. Would you react? Maybe swerve coz you’re startled?
- bigiain 6y ago
  You don't get to abdicate responsibility for your vehicle's "autopilot" just because that was only corporate marketing. Words mean things. You don't get to say they mean different things when you say them just because it helps you sell more cars. Not in the fine print, and not in a court.
- tremon 6y ago
  There's no way a "driver" can autocorrect for a vehicle that makes an emergency stop out of the blue -- the vehicle is still stopped, and now poses a safety hazard for the surrounding traffic.
bigiain 6y ago
I'd go further than that. If a vehicle's automated systems caused or contributed to an accident or a road rule violation, the car manufacturer should be held responsible. If your company's automated emergency braking, or lane assist, or adaptive cruise control - causes or contributes to a problem, someone from that company should be in court next to the driver facing judgement for the consequences. (I'd even go so far as to suggest if the existence of a system leads to a situation where an incident occurred because of the driver's assumption that the automated system would prevent it, then the company should need to justify why the driver should be held responsible instead of the software. If your advertising claims "autopilot!", you're going to be apportioned a significant portion of liability even if your fine print says "Drivers must keep alert and ready to take control at all times" - you should be judged based on a reasonable persons expectation from the language you use to promote your vehicles, not some w4easel words in your T&Cs.)
- nickff 6y ago
  Automation issues may fall into the category of manufacturer/developer negligence; you seem to be suggesting some sort of burden shifting. When you say that a manufacturer should be held liable if their system '... contributes to a problem...', it seems like you're suggesting adoption of the (notoriously problematic) 'if not but for' test. This sort of test is used in negligence cases, but can be useless, as almost anything satisfies it (i.e. slow service at a coffee shop may have been a necessary condition for the timing of that exact accident).
- HALtheWise 6y ago
  Unfortunately, pretty much every automobile safety improvement in the last century (which collectively have saved ~millions of lives) also occasionally kills people. Airbags can propel objects into people, seatbelts can trap someone in a burning car, antilock brakes perform worse on sand, etc. In practice, putting the responsibilty on auto manufacturers for every specific injury or death that happens due to a safety system, without somehow giving them "positive credit" for lives saved, massively delays or entirely prevents these sort of advancements from being made. We definitely need processes to verify that new features are actually functional, typically through recalls and product liability suits, but simply putting blame on manufacturers for all accidents is a terrible idea.
akira2501 6y ago
> but if a driverless car doesn't stop for a speed limit outside a school the company should have some liability. This is a particular challenge in California because our school zone signs specify that the limit only applies "when children are present."
- taneq 6y ago
  How are you meant to know whether children are present in time if you're not doing the lower speed limit?
  - colejohnson66 6y ago
    The idea is that you would be going slow enough already (and if not, slow down). Then, if there’s no children, you can speed up.
  - aidenn0 6y ago
    It weirded me out when I moved here, but in practice, it's enforced like anywhere else: at the beginning and end of school days. Since a large fraction of kids are driven to school (at least in southern California), you know that this is happening because of the massive traffic jam near the school.
  - jimktrains2 6y ago
    Illinois has these "no turn on red when pedestrians present" signs and I have no clue how thats supposed to help anyone because pedestrians usually get hit because they aren't seen by the car.
- csunbird 6y ago
  In Germany, when a speed limit of 30kmh is introduced because of a school, it is usually on applicable for school days and hours (e.g. Mo-Fri 8-18) only. After school hours or non school days you are allowed to drive at normal speed of 50kmh.
sideshowb 6y ago
> it's not unreasonable to build a standard basket of problematic road signs which the car must pass I'm wondering how you would get around metaparameter optimization here, or the fact that it's too easy to optimize to a known test procedure rather than the real world?

bigiain 6y ago

> But pragmatically, they likely don't answer the phone. Or email. > So, it is still easier to ... hold them responsible in court for damage/injury/deaths caused by the actions their vehicles take which are not under control of the driver.

bumby 6y ago
I prefer proactive policy. As much as I would have liked to see more people go to jail for the 2009 financial crisis, for example, I’d prefer to see smarter regulation to have prevented it in the first place
- bluGill 6y ago
  Regulations are too complex to get right like that. You need to figure out how to enforce following the spirit of them not the letter or you will forever be seeing more crises that you didn't foresee.
  - bumby 6y ago
    I think that’s the ideal, but tends to break down in practice. The problem with expecting the spirit of the law to be the compass is that it opens up a lot for interpretation. The end result is that it incentivizes gamesmanship where the organization with the best lawyers win
- bigiain 6y ago
  While I agree - I reckon it's far past time that we pierce the outdated "corporate personhood" idea, and let Tesla/Uber/Apple/Waymo et al. know that society has chosen to hold their "corporate personhood" responsible in the same way people are held responsible. If your corporation (via it's self driving software) kills someone, the corporation will "go to jail", a C level exec does the jailtime and the entire company will be unable to carry on any of it's normal business for the duration of the sentence (while still being held responsible for it's obligations to staff/suppliers/cerditors). If that self driving software is judged to be liable for a traffic infringement, the corporation will pay an amount equal to the percentage of income that a human driver would have paid. If it's an infringement with a ~$400 fine, the corporation should pay ~0.5% of their annual income - the same sort of penalty as "an average salary" person. A couple of $20mil fines and a 6 month total company shutdown would be very "proactive" in encouraging Tesla to get Elon to shut the fuck up with his big bold claims that cannot be backed up. If Uber had been slapped with the "corporate personhood" equivalent of whatever penalty a human driver who'd killed that person crossing the road with a bicycle they'd be far more circumspect about using minimum wage contractors as "safety drivers" in their prototypes. (I can dream, right?)

toast0 6y ago

In the small, it's dramatically more cost effective to fix the road. Ignoring cost to develop and test, just the cost to deploy a fix to all the affected cars would probably be a lot more than £70,000. In the large, it's always a shame to fix a good test case.

colechristensen 6y ago

Or simply decertify all of the vehicles that behave incorrectly from driving on public roads. This is the kind of issue which will persist, vehicles which fail to recognize a real world situation shouldn’t get the benefit of changing the world to fit them. An inquiry should be done into how these vehicles came to fail to recognize this road feature and the upstream cause (not just the bug by the systematic issues in the organizations which produced vehicles with this bug) should be fixed before they’re rectified to operate on roads.

kwdc 6y ago
I like the idea of a warning that particular models of cars are not permitted past this point for safety reasons. That by itself is enough for some frustrated driver to see the sign, take a photo and upload it to social media where those car manufacturers can be suitably laughed at and ridiculed. The good ones would then sort out why their braking system is faulty. Perhaps it gets confused about other things in other serious ways.
- Arnt 6y ago
  I can't tell from the article whether the automatic braking happens when a sudden incline, decline or both is detected in front of the car. It was one of those three, though, that much is clear. Do you really regard braking in front of a sudden incline/decline as ridiculous?
  - fuzxi 6y ago
    It seems like there was a steep hill, but the cattle gate was level, and the sudden, extreme difference in gradient is what caused the sudden braking. Although from the photos, it all looks fairly flat to me.
    - Arnt 6y ago
      Yes, that's my impression too, and braking seems sensible. From uphill: "There's something weird in front of the car, best drive very slowly". From downhill: "WTF I can't see the road any more, best drive very slowly." Both seem quite sensible to me, I don't understand why so many people on the thread seem to disagree.

rlpb 6y ago

For unusual fixed features like this, perhaps car manufacturers should start putting exceptions into their navigation databases, rather than tweaking their automatic detection. You generally don't want to be driving over cattle grids, with their sudden gradient changes, at high speed anyway. So tell the car to slow down as it approaches (or warn the driver loudly to slow down), and briefly disable automatic braking. If it's a rare event, it'll have the drivers' attention. I see the future of self-driving cars to be one where highway authorities map and sign features in a way that is designed for self-driving cars to be able to parse, rather than requiring them to heuristically sense as much as they do today. This would be a step down the path that's inevitable anyway.

netsharc 6y ago
Giant QR codes on highway signs? Haha, an attack by spoofing them would be interesting. Next trick: QR codes with cryptographic signatures, and a huge traffic jam when the traffic authorities forget to renew their certificate...
HPsquared 6y ago
Having a database of known trouble spots would certainly serve as defense in depth, but would only be secondary.

rkangel 6y ago

While the question of who pays for it is important, we should be focusing on the best way to get safe driving. If car sensors and algorithms can get to 99.99% of road features, then changing the remaining 0.01% to work with modern car technologies seems like a reasonable use of resource. No algorithm is ever going to be completely perfect.

Hamuko 6y ago
I'm more worried about the roads that can't be changed.
rtx 6y ago
What happens if there are bugs or we find code being not up to the mark? Should all the models using that version be disabled till OTA updates are rolled out. For reference bugs in Boeing's plane resulted in the whole fleet being grounded.
- fuzxi 6y ago
  I don't see a problem with autonomous vehicles having a manual control mode, unless we're talking about a scenario some decades in the future where you don't actually need a driver's license for an autonomous vehicle.

narcissismo 6y ago

As a programmer, I find it comforting to hear that people are prepared to terraform in order to cancel out my bugs.