4 ms·
Prime Finance (at Amazon) did that. Wrote fancy math on a PhD economist’s laptop. Then when we added testing when developing a platform, we found out we’d bee
by zmgsabst 3y ago
Prime Finance (at Amazon) did that.
Wrote fancy math on a PhD economist’s laptop.
Then when we added testing when developing a platform, we found out we’d been 5% off in allocating Prime revenue between organizations — and had the correct amount gone to Retail, the 2018 hiring freeze might have been avoided. (According to a very angry Wilke.)
Whoops.
Turned out we did need a team and all those guardrails, processes, code reviews, etc.
There’s a time and a place for “move fast and break things” — but real trouble can come from taking those academic practices into the real world.
- aragilar 3y agoDid no-one check the maths? There's a difference between having a full test suite, and someone trying some choice values, but I'd expect both would pick up something like that.
- adrianN 3y agoIf simple checks could find all bugs we would not have so much buggy software.
- zmgsabst 3y agoThe PhD economists on our team looked at it. Economists from other teams looked at it and signed off. There was manual testing — ie, trying some “choice values”. We discovered their error in convexity when our test suite allowed us to randomly sample the models at scale. (Actually, I had questions before that — but unsurprisingly, when it was just me questioning a PhD economist, the lowly SDE was ignored.) Good intentions aren’t enough; you need mechanisms.