Switching when there is no effect is harmless, from the perspective of the product. It only hurts your takeaways, and if you're doing A/B, you're conceding that

Switching when there is no effect is harmless, from the perspective of the product. It only hurts your takeaways, and if you're doing A/B, you're conceding that your previous takeaways weren't the final word anyway. Now, if switching is expensive for some reason, and your A/B isn't as conclusive as you'd need, there's a pretty good chance your change resistance will catch that. So even then it's probably not a big deal.