This may or may not be helpful depending on your business/user-base but one option would be to have a different URL for your site that customers can opt-in to b

This may or may not be helpful depending on your business/user-base but one option would be to have a different URL for your site that customers can opt-in to beta-test early features, changes, etc... Give them some perk for doing this. Some free thing(s), extra features, etc... On that beta/release-candidate URL require their accounts flagged by the end-user to have double-opted-in to advanced analytics. Some people love to try out new features early. In the event a change breaks something ensure those that opted in can still go back to the main URL of your site. Tell your legal team you want the opt-in legal language to be easily readable and understandable by the average person and make the font big so nobody can say there was fine print. Advantages: - People that opt-in to such things are far less likely to make a stink when something breaks. Again, reward them for finding bugs even if its something trivial. Maybe even set up a private community forum for these people to discuss the bugs they find. - These people will also go out of their way to find bugs for you. Real-world traffic meets a free Quality Assurance team. - Your Beta/RC URL can be excluded from any service level agreements in an updated AUP/ToS. Finance people like such risk mitigations. Disadvantages: - More work and cost up front to set this all up even if it pays for itself in the long run. - Requires a bit of humility and discipline to openly share your errors with a handful of your user base.