Agreed that there is an absolutely colossal leap required. Computers can detect the shape & state of a room, but actually giving users interesting & meaningful

Agreed that there is an absolutely colossal leap required. Computers can detect the shape & state of a room, but actually giving users interesting & meaningful experiences that befit the environment is a colossal challenge. Starting with cosmetics is I think a good first step. Take a room & add some vines & flora. Create a variety of sci-fi style embellishments that can be added. Just getting the immersion right, without purpose, I think is a necessary-ish step that has to lead; it shows that the computer can really functionally play with space. From the capabilities we can start to explore what is useful & helpful. It's mostly floating text, but I think Vernor Vinge's Rainbow's End still has some of the best food-for-though, as it's so heavily networked.