3 ms·
There’s really no parallel whatsoever between pushing code to production and a military operation like this. It’s a terrible analogy.
by mdhb 3y ago
There’s really no parallel whatsoever between pushing code to production and a military operation like this. It’s a terrible analogy.
- svrtknst 3y agoThere are parallells to human complacency, and there have been military incidents due to it.
- acdha 3y agoI agree that it’s not a great analogy but would caution that some of the same concerns about complacency apply. If this tool produces some decent looking output, it’s not hard to imagine a mistake being okayed because it looked good similar to someone hitting “LGTM” on Friday afternoon because it’s been fine the last 500 times OR miss something else because they were assuming that the AI would flag the important areas. I know that people in the military are very familiar with problems like that so I think the main concern here is making sure that vendors don’t successfully oversell their products’ capabilities.
- robertlagrant 3y agoIt's not hard to imagine, but then the military already uses loads of automation that that applies to. I don't think this increases that risk.
- acdha 3y agoAt the very least, it would increase the risk by expanding usage. I think there’s an additional angle, however: the better an automated system like this gets, the more people assume it’ll always work. Tesla drivers get in FSD crashes they’d never get in with a basic cruise control because it’s good enough to handle enough road situations that they assume it can handle all of them, despite periodic fatal reminders that it isn’t.
- robertlagrant 3y agoI get that unfamiliarity can breed trust. Same as drivers who blindly follow GPS into a lake. But I don't think the military should stop using GPS.