4D Gaussian splatting will probably get us there, there are many research papers about it for example: https://fudan-zvg.github.io/4d-gaussian-splatting/ https

4D Gaussian splatting will probably get us there, there are many research papers about it for example: https://fudan-zvg.github.io/4d-gaussian-splatting/ https://fudan-zvg.github.io/4d-gaussian-splatting/ But even with that, it would allow only limited movement because if you move to much in the reconstructed space then it would have to show you occluded areas that has been never recorded. But for that case generative AI could be used to fill in the the gaps, which means it wouldn't be true to life reconstruction anymore, but it could be still useful where that isn't that important.