3 ms·
I talked about this several years ago[1], but I strongly believe that we, as a profession, don't invest nearly enough into thinking through all the possible con
by hlieberman 3y ago
I talked about this several years ago[1], but I strongly believe that we, as a profession, don't invest nearly enough into thinking through all the possible consequences of the things that we design. It's easy to write off that what you're doing "doesn't matter" or "can't hurt", but the world is far too interconnected for us to be so nonchalant about our work.
[1]: https://blog.setec.io/2015/11/01/ethics.html https://blog.setec.io/2015/11/01/ethics.html
- renewiltord 3y agoI think we do actually. In terms of economic value / accidental death this is probably the most rigorous engineering discipline. The big difference with software engineering that other engineering disciplines fail at is asking the question "Is it worth the risk?" and then answering it well. For instance, historically and presently, bridge builders accept a much higher risk of killing someone than software engineers. Given the risk that most bridge engineers take, most software engineers would instead just do something else.
- giantrobot 3y agoThere's also the issue that code designed for one purpose often gets used for different purposes. You literally can't think through all of the consequences for a design because the total number of combinations is enormous. Even when you do think through all of the possible combinations you can't know the runtime state of all of those combinations. Formal verification of software exists but can only really be trusted if running on hardware with some multiple redundancy and formal verification of its own.
- keyringlight 3y agoMike Monteiro did a talk [1] along these lines - "How Designers Destroyed the World". [1] https://www.youtube.com/watch?v=qIcM21l61TE https://www.youtube.com/watch?v=qIcM21l61TE