3 ms·

I still believe Hanlons razor applies. I've seen products that have serious performance affecting bugs caused by similar mistakes.

ncallaway 6y ago
I still think—even when applying Hanlon's razor—there's an imbalance in incentives that leads to a weight in favor of the interests of the party paying for the test. Take the store pricing example. Suppose the store's pricing & labeling process produce an equal number of bugs at checkout in favor of the store and in opposition to the store. The store is heavily incentivized to detect the errors that are opposed to them. They are much less likely to detect the errors in their favor. Consider the manager that looks at the cash at the end of the day and notices they are $500 short. They likely dig hard to find the root cause of the issue, detect the pricing disparity and correct it. Now consider the manager that is $500 over at the end of the day. They are much more likely to say: "that's weird", shrug their shoulders and move on. The same applies to forensic tools. Even if they originally produced bugs in both directions, their own internal QA and the market of police officers are likely to work hard to detect bugs that make them less likely to allow them to make an arrest. The net result is that the tools end up with a bias in one direction, even if the original developers made an equal number of mistakes in both directions.
- jfengel 6y ago
  Most store managers get as grumpy about overages as undercounts. They mean that some customer got shortchanged. For $500, it probably means a lot of customers got shortchanged, or something even worse is going on. That makes customers grumpy, and it affects your future. There are plenty of lazy managers who would sweep it under the rug once. But if it happens more than once, it can become their job on the line. They start looking for who's counting wrong. And if they can't figure that out, they get really worried. I have no idea about police officers and prosecutors. But store managers care about accuracy of counts, not just profits.

ncallaway 6y ago

I still think—even when applying Hanlon's razor—there's an imbalance in incentives that leads to a weight in favor of the interests of the party paying for the test. Take the store pricing example. Suppose the store's pricing & labeling process produce an equal number of bugs at checkout in favor of the store and in opposition to the store. The store is heavily incentivized to detect the errors that are opposed to them. They are much less likely to detect the errors in their favor. Consider the manager that looks at the cash at the end of the day and notices they are $500 short. They likely dig hard to find the root cause of the issue, detect the pricing disparity and correct it. Now consider the manager that is $500 over at the end of the day. They are much more likely to say: "that's weird", shrug their shoulders and move on. The same applies to forensic tools. Even if they originally produced bugs in both directions, their own internal QA and the market of police officers are likely to work hard to detect bugs that make them less likely to allow them to make an arrest. The net result is that the tools end up with a bias in one direction, even if the original developers made an equal number of mistakes in both directions.

jfengel 6y ago
Most store managers get as grumpy about overages as undercounts. They mean that some customer got shortchanged. For $500, it probably means a lot of customers got shortchanged, or something even worse is going on. That makes customers grumpy, and it affects your future. There are plenty of lazy managers who would sweep it under the rug once. But if it happens more than once, it can become their job on the line. They start looking for who's counting wrong. And if they can't figure that out, they get really worried. I have no idea about police officers and prosecutors. But store managers care about accuracy of counts, not just profits.