> The bigger problem with this is that the IT systems are not as well defined or a closed system as the other examples might be. A ledger would be simple to ver

> The bigger problem with this is that the IT systems are not as well defined or a closed system as the other examples might be. A ledger would be simple to verify in comparison: you apply a fixed set of rules to a fixed period of data and it must result in something within a known range. With software you can do a little bit of determining that the 'borders' of your 'box' of functionality would be, but to do an exhaustive verification is so far from profitable that it's only done in a few highly regulated systems like flight controllers. That's a really good point. We haven't even settled on how to weed out bad programmers through a formal structure yet, and we still sometimes hear about companies setting KPIs based on LoC or bugs fixed. I'm not sure if it's even possible to have an objective framework to evaluate an outsourced system apart from "meets the spec" which is usually not very helpful unless there is in-house expertise spending a mountain of effort building the spec in the first place.