I think I understand the Aharonov–Bohm effect for magnetism, but I don't understand how you can have the branches at different gravitational potentials without

I think I understand the Aharonov–Bohm effect for magnetism, but I don't understand how you can have the branches at different gravitational potentials without the packets experiencing a field somewhere when they branch/join.