2 ms·
Not affecting real customers is kind of the point of controlled experiments.
by zzzcpan 9y ago
Not affecting real customers is kind of the point of controlled experiments.
- drdrey 9y agoYou can very much impact real customers, but the idea is that you should be able to only apply the experiment to a small subset of customers and abort it in case something goes wrong. For instance, you may want to check that the authorization service can function without access to the A/B testing service, so you cut the network connection between them. If authoritazion errors start rising, you stop the experiment and investigate why this happened. You may also find that clients did not retry properly on unexpected errors from the authorization service (e.g. http 500 error codes)