Why can’t the linear flow of time and a singular instance of reality just be the narrow lense through which we view the universe? If reality is infinitely bran

Why can’t the linear flow of time and a singular instance of reality just be the narrow lense through which we view the universe? If reality is infinitely branching, then time travel is merely a way for the traveler to perceive and move between those infinite branches. If you kill your grandfather in an alternate timeline your grandfather is still alive in the timeline you came from, hence no paradox. You (an alien in the new timeline) can then watch a world develop in that alternate reality where your grandfather died. If time travel were in fact possible, this would be the only way my puny brain could conceive of it actually working. If there’s only strictly one path that reality takes even from the perspective of an outside observer (God) then I can’t see how time travel could be possible in such a construct. If it is possible it seems you must implicitly admit that there is not in fact a singular thread of reality/time. Just because we define some equations where t can flow in reverse and fit them into the standard model doesn’t seem to me to be persuasive evidence one way or another.