There were actually a lot of discussions about this sort of thing that lead up to the BIFS specification in Mpeg4 (Part 11). But BIFS obviously never anticipate

There were actually a lot of discussions about this sort of thing that lead up to the BIFS specification in Mpeg4 (Part 11). But BIFS obviously never anticipated virtual reality and the like, let alone light field or holography.