Yes for any sort of complicated input i.e. computer code, real world experience can not be used. On the other hand natural language processing almost good enoug

Yes for any sort of complicated input i.e. computer code, real world experience can not be used. On the other hand natural language processing almost good enough today for inputs with medium complexity like grocery list. Real world interaction is obvious for certain fields where user does something that's comparable to a real world task. i.e. Loading ammunition, Surgery, Plumbing, Repair of various instruments. For fields that don't have comparable real world representation I would imagine that we're again at the moment in computer science history where we have an opportunity to create a new desktop environment but in 3D space with certain well known natural gestures. For example transferring a file from my laptop to a smartphone doesn't have to be done with context menu, we can have a VR smartphone object and just drag file with our hands in that object lie we would do in real world if were to move a file from one place to another. Instead of having to switch between different overlapping windows, we can have all windows laid out all around us in a comfortable zone. To go to particular window in focus we use certain gestures like pushing of pulling with hands. Even if some gestures have to be created artificially, user would probably remember that better next time because of the muscle memory. There's a really good discussion regarding this in Unity Unite 2016 (https://www.youtube.com/watch?v=HzPDTrWcjxc https://www.youtube.com/watch?v=HzPDTrWcjxc). In fact 2016 conference had a lot of good VR related discussions.