4 ms·
Contrary to what that link says, the software was not thoroughly tested. Normal testing was bypassed - per management request after a small code change. This w
by phkahler 6y ago
Contrary to what that link says, the software was not thoroughly tested. Normal testing was bypassed - per management request after a small code change.
This was covered in a book (perhaps Safeware, but maybe another one I dont recall) along with the Therac 25, the Ariane V, and several others. Unfortunately these lessons need to be relearned by each generation. See the 737-Max...
- pacificmint 6y agoProbably not the book you are thinking off, since it’s just about the AT&T incident, but “The Day the Phones Stopped Ringing” by Leonard Lee is a detailed description of the event. It’s been many years since I read it, but I recall it being a very interesting read.
- tinco 6y agoFor some reason in my university almost every CS class would start with an anecdote about the Therac 25, Ariane V, and/or a couple others as a motivation on why we the class existed. It was sort of a meme. The lessons are definitely still taught, I don't know if they're actually learned of course.. And who knows who actually taught the 737-Max software devs, I don't suppose they're fresh out of uni.
- TheSpiceIsLife 6y agoDo management typical typically study Computer Science?
- tinco 6y agoTypically yes actually, the director of engineering should always be an engineer. Of course, these are hardware companies so it would probably be some kind of hardware engineer.
- TheSpiceIsLife 6y agoShould. Sure.
- _heimdall 6y agoUnfortunately most people become a manager by bring a stellar independent contributor. People management and engineering are very different skills, I'm always impressed when I see someone make that jump smoothly. I always wanted companies to hire people managers as its own career path. An engineer can be an excellent technical lead or architect, but it can feel like you started over once you're responsible for the employees, their growth, and their career path.
- thatfunkymunki 6y agoYeah, it just sucks that you eventually have someone making significant people management decisions without the technical knowledge of what the consequences could end up being. This would be even worse if you had people manager hiring be completely decoupled. The US military works this way and I have to say it's not the best mode.
- jacquesm 6y ago> Normal testing was bypassed - per management request after a small code change. That lesson will really never be learned. This happens on a daily basis all over the planet with people who have not been bitten - yet.
- edoceo 6y agoManagers who have been bitten still make this choice
- cortesoft 6y agoIt isn't learned because 99% of the time, it works fine and nothing bad happens. We are very bad at avoiding these sorts of rare, catastrophic events.
- coryrc 6y agoOr most of what we do isn't really important so it doesn't matter if it breaks every once in a while.
- eru 6y agoThat's why the most reliable way to instil this lesson is to instil it into our tools. Automate as much testing as possible, so that bypassing the tests becomes more work than running them.
- smaudet 6y agoI disagree, it's in part a people problem - more draconian test suites just make developers more inclined to cheat and they tend to write tests which are not valid or just get the tool passing... It's more important to visually model and test than to enforce some arbitrary set of rules that don't apply universally - then you have at least the visual impetus of 'this is wrong' or 'I need to test this right'. A lot of time is spent visually testing UIs and yet these same people struggle with testing the code that matters...
- monkpit 6y agoUntil a manager is told about how hard the automation makes it to accomplish their goal...
- _heimdall 6y agoAs a former AT&T contractor, albeit from years later, this checks out. Sat in a "red jeopardy" meeting once because a certain higher-up couldn't access the AT&T branded security system at one of his many houses. The build that broke it was rushed out and never fully tested, adding a fairly useless feature for said higher-up that improved the UX for users with multiple houses on their account.