That's fine so long as you have a full test suite to exercise the app - catch small mail server configuration changes, apparmor and MySQL (we got bitten by that

That's fine so long as you have a full test suite to exercise the app - catch small mail server configuration changes, apparmor and MySQL (we got bitten by that when upgrading to MySQL 5.5 from 5.1) et al - it's not a matter of saying "hey this looks like it's running" but verifying there's no regressions either. Mind you containers don't help with this either...