5 ms·

I could see working for for Tesla, I couldn't work on Tesla's AutoPilot though. I have deep reservations about using members of the public (and others on the r

I could see working for for Tesla, I couldn't work on Tesla's AutoPilot though. I have deep reservations about using members of the public (and others on the road) as guinea pigs. Before the Walter Huang fatality, maybe, but after that and how Tesla acted about what was clearly a software bug in software that wasn't fit to be on public roads ("beta" label or otherwise), I could never justify working for such a company. Imagine being a software engineer on AutoPilot at the time, and then having the company you work for excuse your bugs by blaming the victim for not circumventing AutoPilot controlled steering into a wall. Software development ethics don't come up a lot. But I feel like AutoPilot is up there with medical software, aviation, and military, in that mistakes can easily cost lives. I want to be able to sleep at night. It is a cool problem-space, but only academically.

alexpetralia 7y ago
By this logic, wouldn't you want an opportunity to help direct this space more positively, rather than being completely hands off?
- michaelt 7y ago
  To me it seems unlikely a newly hired C developer could overrule Elon Musk on things like releasing the product to customers or having LIDAR.
raxxorrax 7y ago
The way he demonstrated to treat his engineers would make me more suspicious. I have experience in c/c++ and especially imaging, but no way in hell would I want to work there. Maybe the guys he publicly fired were idiots, but even then you can manage that differently. It was a to a time when he got a lot of crap from media, but that is not a good excuse. Seriously lost faith in the company for that.
- rurban 7y ago
  I've worked im automotive in similar fields, even more advanced, and Tesla is not an outlier, probably even better than the rest. Tesla has a huge advantage in its flat modern management structure. They are at least 10x faster and cheaper than the rest. The results are visible to everyone. Bad midmgmt apples are everywhere, esp. in those kind of industries.
deweller 7y ago
The ethics here are not as simple as "don't ship any self-driving AI until you are 100% sure they will never make a fatal mistake." There is no such thing as 100% assurance in this field. If you are so cautious that you never ship any self-driving code, there are lives that may be lost due to accidents caused by human failure. These accidents might have been avoided if an AI was at the wheel. I respect this type of work is not for all people. I applaud the brave companies plowing ahead.
- majormjr 7y ago
  There is a difference between 99% and 99.99% assurance in this field though. You can test your self-driving AI to a reasonably high degree of certainty before ever testing in a public uncontrolled environment.
- semi-extrinsic 7y ago
  I've argued in several of these threads on self-driving the following point: safety in terms of fatalities per billion vehicle miles travelled (VMT) is already in the single-digit range in Europe, and likely far below 1 fatality per billion VMT when restricted to "highways AND modern cars AND good conditions". So you need ridiculously huge samples to get statistically significant data on the safety of a single manufacturer's system. As in the order of magnitude of a hundred thousand miles per car for every one of the ~ 900k Teslas sold so far. Thus you reach a paradox: with current trends in "normal" car safety improvement, you won't know for sure if a particular self-driving model is safer than non-self-driving models until they are close to end-of-life. And then it's a bit late to find out.
SEJeff 7y ago
The inverse is also true. There are plenty of videos of autopilot preventing wrecks that the human drivers said they likely would have wrecked on themselves. I happen to be one of those drivers when someone pulled onto the highway from a field at night. AP swerved and missed the idiot but I'd not even seen them and would have almost certainly hit them doing about 70mph. A T-bone that fast would likely have messed both of us up really good. AP will never prevent all injuries / fatalities due to road conditions and other human drivers simply being imperfect. It has a good chance to prevent a lot of otherwise fatal crashes in the meantime however. One could say it would be unethical to not even consider the alternative here. Just some food for thought. I'm a software engineer myself and Model 3 owner.
- BoorishBears 7y ago
  I always see this argument and it's like everyone's forgotten that AP isn't the only automatic safety system. My Volt had the same exact hardware as AP1 (from Mobileye). GM decided not to play games with marketing, and had it only intervene in cases of immediate danger. For example, you could try and get it to follow lanes, but it's guidance would be directly proportional to your steering input, and it would intentionally stop intervening the moment you were back in your lane, eventually disengaging if you weren't trying to correct at all. - At night in pouring rain coming around a tight highway turn, my car stopped for an overturned box truck covering 2 of 3 lanes and allowed me to go around it. It saved my life with very little fanfare. AP didn't have to be called AP. AP didn't have to be something you can let take over (except jk don't let it take over or we'll victim blame you when it kills you). AP could provide traffic-aware cruise control, could take over when you drift out of your lane, swerve away from imminent crashes, and you would still have been saved from that T-bone, and Walter Huang would still be alive today. But then Tesla couldn't charge extra for Advanced AP, then extra on top of that for FSD.
- semi-extrinsic 7y ago
  On the flipside, in these cases where people self-report "I likely would have wrecked myself", we can reasonably expect that they are paying less attention to the driving situation while using AP than if they were driving a car without AP. So we expect them to be strongly biased towards underestimating their own abilities in the situation.
nexuist 7y ago
> mistakes can easily cost lives. I want to be able to sleep at night. Everyone who works in the safety field has to deal with this. How do you think the MCAS engineers feel? But let's rewrite history, and say Boeing decided to scrap MCAS before it ever shipped on a 737 MAX. Now let's say a MAX crashes from stalling on takeoff - the exact scenario MCAS was meant to prevent. How would those engineers feel? How would we be looking at Boeing if we knew there was work done on a prevention device, and the higher-ups decided to scrap it before it was completed? No one engineer or manager is responsible for a tragedy. Organizations, sure, but no organization can be perfect. You can only try to use whatever power you have to guide it towards better practices and regulations.

alexpetralia 7y ago

By this logic, wouldn't you want an opportunity to help direct this space more positively, rather than being completely hands off?

michaelt 7y ago
To me it seems unlikely a newly hired C developer could overrule Elon Musk on things like releasing the product to customers or having LIDAR.

raxxorrax 7y ago

The way he demonstrated to treat his engineers would make me more suspicious. I have experience in c/c++ and especially imaging, but no way in hell would I want to work there. Maybe the guys he publicly fired were idiots, but even then you can manage that differently. It was a to a time when he got a lot of crap from media, but that is not a good excuse. Seriously lost faith in the company for that.

rurban 7y ago
I've worked im automotive in similar fields, even more advanced, and Tesla is not an outlier, probably even better than the rest. Tesla has a huge advantage in its flat modern management structure. They are at least 10x faster and cheaper than the rest. The results are visible to everyone. Bad midmgmt apples are everywhere, esp. in those kind of industries.

deweller 7y ago

The ethics here are not as simple as "don't ship any self-driving AI until you are 100% sure they will never make a fatal mistake." There is no such thing as 100% assurance in this field. If you are so cautious that you never ship any self-driving code, there are lives that may be lost due to accidents caused by human failure. These accidents might have been avoided if an AI was at the wheel. I respect this type of work is not for all people. I applaud the brave companies plowing ahead.

majormjr 7y ago
There is a difference between 99% and 99.99% assurance in this field though. You can test your self-driving AI to a reasonably high degree of certainty before ever testing in a public uncontrolled environment.
semi-extrinsic 7y ago
I've argued in several of these threads on self-driving the following point: safety in terms of fatalities per billion vehicle miles travelled (VMT) is already in the single-digit range in Europe, and likely far below 1 fatality per billion VMT when restricted to "highways AND modern cars AND good conditions". So you need ridiculously huge samples to get statistically significant data on the safety of a single manufacturer's system. As in the order of magnitude of a hundred thousand miles per car for every one of the ~ 900k Teslas sold so far. Thus you reach a paradox: with current trends in "normal" car safety improvement, you won't know for sure if a particular self-driving model is safer than non-self-driving models until they are close to end-of-life. And then it's a bit late to find out.

SEJeff 7y ago

The inverse is also true. There are plenty of videos of autopilot preventing wrecks that the human drivers said they likely would have wrecked on themselves. I happen to be one of those drivers when someone pulled onto the highway from a field at night. AP swerved and missed the idiot but I'd not even seen them and would have almost certainly hit them doing about 70mph. A T-bone that fast would likely have messed both of us up really good. AP will never prevent all injuries / fatalities due to road conditions and other human drivers simply being imperfect. It has a good chance to prevent a lot of otherwise fatal crashes in the meantime however. One could say it would be unethical to not even consider the alternative here. Just some food for thought. I'm a software engineer myself and Model 3 owner.

BoorishBears 7y ago
I always see this argument and it's like everyone's forgotten that AP isn't the only automatic safety system. My Volt had the same exact hardware as AP1 (from Mobileye). GM decided not to play games with marketing, and had it only intervene in cases of immediate danger. For example, you could try and get it to follow lanes, but it's guidance would be directly proportional to your steering input, and it would intentionally stop intervening the moment you were back in your lane, eventually disengaging if you weren't trying to correct at all. - At night in pouring rain coming around a tight highway turn, my car stopped for an overturned box truck covering 2 of 3 lanes and allowed me to go around it. It saved my life with very little fanfare. AP didn't have to be called AP. AP didn't have to be something you can let take over (except jk don't let it take over or we'll victim blame you when it kills you). AP could provide traffic-aware cruise control, could take over when you drift out of your lane, swerve away from imminent crashes, and you would still have been saved from that T-bone, and Walter Huang would still be alive today. But then Tesla couldn't charge extra for Advanced AP, then extra on top of that for FSD.
semi-extrinsic 7y ago
On the flipside, in these cases where people self-report "I likely would have wrecked myself", we can reasonably expect that they are paying less attention to the driving situation while using AP than if they were driving a car without AP. So we expect them to be strongly biased towards underestimating their own abilities in the situation.

nexuist 7y ago

> mistakes can easily cost lives. I want to be able to sleep at night. Everyone who works in the safety field has to deal with this. How do you think the MCAS engineers feel? But let's rewrite history, and say Boeing decided to scrap MCAS before it ever shipped on a 737 MAX. Now let's say a MAX crashes from stalling on takeoff - the exact scenario MCAS was meant to prevent. How would those engineers feel? How would we be looking at Boeing if we knew there was work done on a prevention device, and the higher-ups decided to scrap it before it was completed? No one engineer or manager is responsible for a tragedy. Organizations, sure, but no organization can be perfect. You can only try to use whatever power you have to guide it towards better practices and regulations.