Agreed. Even if not ran at the same time, it still seems dangerous to split by modulo. Why? Suppose some test is set up with the end result that group A visits

Agreed. Even if not ran at the same time, it still seems dangerous to split by modulo. Why? Suppose some test is set up with the end result that group A visits the site 5% less in the future, and B is unchanged. After this result is discovered, the next test is run, which is to test whether the background should be white (A) or puke green (B). Now even if in actuality people don't care one way or the other about the color, it may well be discovered that group A does worse, because there are still people among this group that were exposed to the previous test and are now coming back less (or have stopped coming at all).