4 ms·

I noticed this while experimenting with VR and hand-tracking. Without tactile or precise auditory feedback, the experience was hollow, even with relatively high

by bsanr2 7y ago

I noticed this while experimenting with VR and hand-tracking. Without tactile or precise auditory feedback, the experience was hollow, even with relatively high-fidelity visuals and tracking. You can never feel like you are somewhere else - or something immaterial is with you - until a full sensory experience coalesces, even if it's "low resolution." Until then, the result is landing in an experiential uncanny valley. On another note, we should also be looking at accessibility when approaching this research. Contrary to that last little bit at the end, not everyone has full use of their bodies, or even of their hands. I just finished listening to an NPR segment about inner-city youth dealing with permanent disability as a result of gun violence, and the unique challenges presented to them. Perhaps with people like them in mind, the mass solutions we eventually end up with will be better for focusing farther upstream than nerve endings.

gfodor 7y ago
Hands-in-air interactions in VR I agree lack the tactile feedback necessary to avoid it feeling hollow. However, I've often advocated that the "ideal" VR input system in 2019 is: - Freely tracked dominant hand, used for fine-grained control/pointing, expressing body language, and grab/release-like mechanics (which work decently for certain kinds of virtual objects/interactions, like throwing stuff) - Tracked 6dof controller in non-dominant hand, similar to the existing controllers except with a fully capable twiddler-like button grid for doing fast symbolic entry. For anything where tactile feedback is core, move it onto the controller hand, where the hardware buttons can provide a more reasonable proxy for things like press-and-release or sustained-holding in terms of motor interactions. (Haptics are still pretty bad on all other accounts, but controllers at least give you pressure feedback under your finger tips, instantaneous response, etc.) With the above setup, I suspect you could perform most knowledge worker oriented tasks with similar efficiency to a mouse and keyboard, and also perform existing VR-centric use cases like 3d modelling/manipulation, avatar locomotion, full body social interaction, etc. The current two-controller or dual hand-tracked paradigms fall short on the former. With the introduction of hand tracking capabilities on Quest coming, it seems that we're inching up to mainstream VR devices having the capability to deliver this, except the controllers still lack anything close to keyboard-level discrete inputs. However, once Quest hand tracking launches, it may be possible to use an off-the-shelf twiddler for this if the SDK exposes the ability to track in-hand tools (similar to the Leap Motion SDK.)
Wowfunhappy 7y ago
> You can never feel like you are somewhere else - or something immaterial is with you - until a full sensory experience coalesces, even if it's "low resolution." I'm not sure I agree with that. Games like Beat Saber and the old demo of Budget Cuts have really made me feel like I'm inside another world, particularly when I was newer to VR. I say "not sure" because we're not necessarily talking about the same thing—the above are "games" or at least "experiences", not "interfaces. You don't really touch anything in Beat Saber, except for the blocks where a strong rumble is sufficient. Harder to do that in a UI.
- seanmcdirmid 7y ago
  In a fitness experience, you can imagine the resistance at least, which is what I wind up doing in beat saber to increase my heart rate and calorie burn. It is actually an interesting problem: can the mind imagine the feedback that really isn’t there, causing muscles to work harder even if they don’t have to? For some people at least, the answer is yes, though I bet it would be hard for others. But shadow boxing, katas, and so on, have shown that imagining resistance has been around for awhile. We can kind of imagine tactile feedback already in a UI, and vibration is effective in reinforcing that illusion. Perhaps the way forward is to further play around with fooling the brain (via some kind of cognitive interface) rather than somehow reproducing real feedback.
  - TeMPOraL 7y ago
    Vibration is key in current VR experience, IMO. Having spent many hours on Beat Saber over the past few weeks, I can say that the only reason my brain recognizes that I hit something with the blade is the little controller buzz that activates at the right moment. As for adapting yourself to the game, I keep having to focus on preventing myself from using my wrists too much when playing. Given that my goal is calorie burn, I do my best to use my whole arm (as if wielding a real blade). I've noticed that I do create "impact resistance" myself too.
    - seanmcdirmid 7y ago
      It took awhile to stop us g wrists, but now I imagine it as some immensely satisfying glaive work. BoxVR is another game where imagining feedback is crucial, more so actually with their fitness focus. Their inaccurate calorie counter is completely motion base, so wider faster movements will drive that up as well. Honestly, BoxVR (custom with lots of fast songs) is now more important than BeatSaber to my daily workout, because I’m able to get a lot more effective motions in. On the other hand, I can’t play it with the toddler around (if he got in my way, he could really be hurt by the gestures I’m doing).

by bsanr2 7y ago

gfodor 7y ago

Hands-in-air interactions in VR I agree lack the tactile feedback necessary to avoid it feeling hollow. However, I've often advocated that the "ideal" VR input system in 2019 is: - Freely tracked dominant hand, used for fine-grained control/pointing, expressing body language, and grab/release-like mechanics (which work decently for certain kinds of virtual objects/interactions, like throwing stuff) - Tracked 6dof controller in non-dominant hand, similar to the existing controllers except with a fully capable twiddler-like button grid for doing fast symbolic entry. For anything where tactile feedback is core, move it onto the controller hand, where the hardware buttons can provide a more reasonable proxy for things like press-and-release or sustained-holding in terms of motor interactions. (Haptics are still pretty bad on all other accounts, but controllers at least give you pressure feedback under your finger tips, instantaneous response, etc.) With the above setup, I suspect you could perform most knowledge worker oriented tasks with similar efficiency to a mouse and keyboard, and also perform existing VR-centric use cases like 3d modelling/manipulation, avatar locomotion, full body social interaction, etc. The current two-controller or dual hand-tracked paradigms fall short on the former. With the introduction of hand tracking capabilities on Quest coming, it seems that we're inching up to mainstream VR devices having the capability to deliver this, except the controllers still lack anything close to keyboard-level discrete inputs. However, once Quest hand tracking launches, it may be possible to use an off-the-shelf twiddler for this if the SDK exposes the ability to track in-hand tools (similar to the Leap Motion SDK.)

Wowfunhappy 7y ago

> You can never feel like you are somewhere else - or something immaterial is with you - until a full sensory experience coalesces, even if it's "low resolution." I'm not sure I agree with that. Games like Beat Saber and the old demo of Budget Cuts have really made me feel like I'm inside another world, particularly when I was newer to VR. I say "not sure" because we're not necessarily talking about the same thing—the above are "games" or at least "experiences", not "interfaces. You don't really touch anything in Beat Saber, except for the blocks where a strong rumble is sufficient. Harder to do that in a UI.

seanmcdirmid 7y ago
In a fitness experience, you can imagine the resistance at least, which is what I wind up doing in beat saber to increase my heart rate and calorie burn. It is actually an interesting problem: can the mind imagine the feedback that really isn’t there, causing muscles to work harder even if they don’t have to? For some people at least, the answer is yes, though I bet it would be hard for others. But shadow boxing, katas, and so on, have shown that imagining resistance has been around for awhile. We can kind of imagine tactile feedback already in a UI, and vibration is effective in reinforcing that illusion. Perhaps the way forward is to further play around with fooling the brain (via some kind of cognitive interface) rather than somehow reproducing real feedback.
- TeMPOraL 7y ago
  Vibration is key in current VR experience, IMO. Having spent many hours on Beat Saber over the past few weeks, I can say that the only reason my brain recognizes that I hit something with the blade is the little controller buzz that activates at the right moment. As for adapting yourself to the game, I keep having to focus on preventing myself from using my wrists too much when playing. Given that my goal is calorie burn, I do my best to use my whole arm (as if wielding a real blade). I've noticed that I do create "impact resistance" myself too.
  - seanmcdirmid 7y ago
    It took awhile to stop us g wrists, but now I imagine it as some immensely satisfying glaive work. BoxVR is another game where imagining feedback is crucial, more so actually with their fitness focus. Their inaccurate calorie counter is completely motion base, so wider faster movements will drive that up as well. Honestly, BoxVR (custom with lots of fast songs) is now more important than BeatSaber to my daily workout, because I’m able to get a lot more effective motions in. On the other hand, I can’t play it with the toddler around (if he got in my way, he could really be hurt by the gestures I’m doing).