> In my 28 years of professional experience working in 9 different organizations, only one of them did fully run the core system in development, and another two

> In my 28 years of professional experience working in 9 different organizations, only one of them did fully run the core system in development, and another two of them could have done that with enough time and effort. For the other six, there was no reasonable way to run the full system in development. Only ~20 years here but I've worked almost exclusively on three or four large systems with test environments despite their complexity. I think it may be the nature of the companies/business that makes the difference. Ancient (1980's code) hospital ADT+billing system (medseries 4 if anyone cares) on an AS/400. We kept the last generation of hardware around to run the test/demo environment which was end-to-end connected via HL7 to all the other test environments. That included the other thing I managed, Cerner Millennium, a hodgepodge of a medical records system on AIX and Oracle, where again the old hardware got used as the test and build systems. The test environment was critical because even though it was made up of a dozen commercial software packages talking to each other over HL7 and flat files (and serial for lab devices and who knows what else) it was critical to make sure the whole environment worked when doing upgrades or new development. We kept a full copy of prod data for final integration testing. The tooling for (most of) the software supported replicating environments. The largest example is Photon/Ubiq at Google; end-to-end test environment running with deterministic sampling to save resources at all stages of the pipeline so everything from injection through logjoining and aggregation to the final database updates were continuously running the latest code for testing before a release to prod. Thousands of prod machines devoted to it. That was in addition to the integrated tests and unit tests during the development process. At FB the statistical counter system I worked on also kept a continuous test pipeline running, again with event sampling to reduce cost, to ensure everything would work properly before a new release. My takeaway is that where it matters companies are willing to spend the money and people to have full end-to-end testing, often in realtime, and have some leeway to save on resources with subsampling, smaller datasets, etc.