Philosophically I agree, but in my experience the company isn't willing to pay for experiments except in Production due to scaling issues and quality of synthet

Philosophically I agree, but in my experience the company isn't willing to pay for experiments except in Production due to scaling issues and quality of synthetic data. So you need to instead have a series of features that are architecturally solid, have KPIs to measure success, and be willing to throw out working production code that doesn't move metrics enough to warrant their complexity.