I'm reminded of the description in "Rainbow End?" where the guy doesn't have a VR setup yet and sees people reaching out and touching shapes held in the air by

I'm reminded of the description in "Rainbow End?" where the guy doesn't have a VR setup yet and sees people reaching out and touching shapes held in the air by mechanical arms. Basically, there are 'self-arranging blobs' that arrive 'just in time' to your interaction, changing their tactile surface to reflect what you expect. The user sees none of this, and only what they see in their interface. I can't imagine it working perfectly. Say, pulling a book out of a shelf should have exactly zero resistance (other than the force of gravity pulling the book down). Having near zero latency between action and the 'tactile interface' moving in exactly the right direction would be a very hard problem...