4 ms·

Well surely there were tests...

• sillysaurusx 6y ago
  Nope! They solved it the opposite way. Their bug tracking system consisted of individual customers who called up to report bugs. I.e. Joe Blow would call up and say "Hey, I'm supposed to see 231 shares, but I only see 219 shares." That would result in a bug report. The bug report was that Joe Blow saw that outcome. In other words, the bug report didn't contain anything about the code itself. They had an entire division of support people whose job was to track these bugs, and to fix them. In order to fix the bug, they would manually edit the database, or do whatever was necessary to make sure Joe Blow saw the right value again. But they never changed the code; they weren't programmers. Once Joe Blow was happy, they closed the bug report as "fixed." Therefore, no tests were ever required. It was ... impressive? I think? I couldn't mentally process what I was seeing at the time. But "impressive" is probably the right word. After all, the system worked.
  • lonnyk 6y ago
    How did they verify that Joe Blow was actually supposed to have 231 shares instead of 219?
    • sillysaurusx 6y ago
      I always wondered that! I assume they had a "ground truth" financial order book somewhere (which presumably was held to much higher standards of correctness) and that the support staff manually verified their balance. But ... that logic doesn't work if you chase down the implications. And sadly I was both too shocked and too young to press my coworker for details. (He was a cool older fellow who seemed as amused with the craziness.) Eventually I became a pentester at Matasano. During my one-year stint, I was parachuted into around 70 codebases. I got to see first-hand that Scottrade wasn't an outlier; they were the average. Most companies have similar WTFs, and the codebases are just as onerous. The world is held together with duct-tape. That's why pentesting is so crucial.