I just thought of what might be a silly question: In a situation where we can't prove that a probability distribution converges to some theoretical assumed limi

I just thought of what might be a silly question: In a situation where we can't prove that a probability distribution converges to some theoretical assumed limit, can we assume it even converges to anything? All we have is circumstantial evidence. Or who is to say that somewhere along the line it doesn't converge to something else? Even if tested to the billionth digit, it's still practically zero compared to testing to the billionth busy beaver number etc. etc. I guess this is more of a philosophical question - given no mathematical proof, why is any sample size of N that is tested not better in any sense than a sample of value N=1?