5 ms·

VR hand tracking will be better as an input method when the user has little or no expectation of physical resistance. Imagine being in a wizard game where you h

VR hand tracking will be better as an input method when the user has little or no expectation of physical resistance. Imagine being in a wizard game where you have hundreds of spells memorized - water spells, fire, electricity, teleportation, telekinetic manipulation, etc.... all of which are activated by esoteric hand gestures. Make it an RPG with UE5-demo level graphics and it will be one of the most mind-blowing games ever.

airstrike 6y ago
Would love it even more if the game had hilarious consequences for not making the right gestures
- novok 6y ago
  I get that feeling already when I tried half life alyx's gesture to gravity grab objects in the air.
GuiA 6y ago
Step 1: Make an amazing game with gestures Step 2: ??? Step 3: Profit! Joke aside, such game mechanics, albeit with cursor gesturing rather than hand gesturing, have been tried many times before. See Arx Fatalis, Black & White, etc. Jonathan Blow has a pre-Braid prototype about that as well that he shows in some detail in a talk he gave. The reality is that sure, it sounds cool when you describe it like that, but there are many design issues to contend with: - how does the player remember all these obscure symbols ? - how do you deal with the frustration when a symbol isn't recognized? When it's confused by the system with another symbol? Etc. On top of that, there is the fact that drawing squiggles to cast spells is not a game mechanic that is inherently extremely fun. Why are people going to the bother of doing that vs pressing a button? It's all a lot of work to make it make sense, and there is little reason to believe that gesturing with your hands instead of your mouse would solve some of the more fundamental problems.
- keenmaster 6y ago
  The limitations you describe are rooted in a pre-VR world. VR is much more conducive to gestures. In Half Life Alyx, Valve could have technically allowed you to lob a grenade by just pressing a button "because it's easier." But they didn't, and opted for a gesture-based input. They could have allowed you to turn a wheel by mashing a button, but they didn't, and you actually have to turn the wheel in circles. These decisions would have been insane with a keyboard and mouse as inputs. They're natural for a VR game. Without a keyboard, it's cumbersome to switch between a bunch of spells. Gestures are both more immersive and functionally better. Maybe most people wouldn't memorize hundreds of spells, but a couple dozen are certainly feasible. There could be a spell "language," where the initial part of the gesture could correspond to the category of spell. Fireball, fire wave, etc...can start with the same basic gesture. All wave spells can end with the same gesture. Hundreds of spells can be constructed from permutations of 20-30 foundational gestures, kind of like how a core group of trilateral roots in Arabic and Hebrew can construct a much larger set of words (sometimes without previous knowledge of a specific word/permutation) https://en.wikipedia.org/wiki/Semitic_root https://en.wikipedia.org/wiki/Semitic_root
  - GuiA 6y ago
    The examples you mention work well because there is a real world affordance (the wheel, the act of throwing) that the gesture maps to. I agree that rotating a wheel gesturally makes more sense as an interaction in VR than the same action does with a mouse pointer. The grenade example is a good one, because throwing/flicking doesn't require much feedback, unlike say rotation or fine grained manipulation. Of course the interaction would feel nicer if you could feel the weight of what you're throwing - the physics of throwing objects always feels kind of off in VR games - but it still works well enough. Note that many games did this with the Wiimote, and there is nothing about that interaction that is singular to VR. However, the moment you lose that affordance based in a real world object, and ask your user/player to make gestures in mid air based on recollection, things really do start to fall apart. There is a host of HCI literature/projects around this - e.g. check out papers/demos from the Kinect era. An RPG game like the one you describe could be made with Kinect, and I vaguely recall a few in development - but Kinect demonstrated well that the only kind of gestural interaction that works in long term sessions is dancing. Btw, this is why if you go to e.g. the VR center in Tokyo, almost every booth has physical props for you to hold/grip/sit on while you're fishing/golfing/shooting a gun/riding a bike in VR.
deleted 6y ago
[deleted]

airstrike 6y ago

Would love it even more if the game had hilarious consequences for not making the right gestures

novok 6y ago
I get that feeling already when I tried half life alyx's gesture to gravity grab objects in the air.

GuiA 6y ago

Step 1: Make an amazing game with gestures Step 2: ??? Step 3: Profit! Joke aside, such game mechanics, albeit with cursor gesturing rather than hand gesturing, have been tried many times before. See Arx Fatalis, Black & White, etc. Jonathan Blow has a pre-Braid prototype about that as well that he shows in some detail in a talk he gave. The reality is that sure, it sounds cool when you describe it like that, but there are many design issues to contend with: - how does the player remember all these obscure symbols ? - how do you deal with the frustration when a symbol isn't recognized? When it's confused by the system with another symbol? Etc. On top of that, there is the fact that drawing squiggles to cast spells is not a game mechanic that is inherently extremely fun. Why are people going to the bother of doing that vs pressing a button? It's all a lot of work to make it make sense, and there is little reason to believe that gesturing with your hands instead of your mouse would solve some of the more fundamental problems.

keenmaster 6y ago
The limitations you describe are rooted in a pre-VR world. VR is much more conducive to gestures. In Half Life Alyx, Valve could have technically allowed you to lob a grenade by just pressing a button "because it's easier." But they didn't, and opted for a gesture-based input. They could have allowed you to turn a wheel by mashing a button, but they didn't, and you actually have to turn the wheel in circles. These decisions would have been insane with a keyboard and mouse as inputs. They're natural for a VR game. Without a keyboard, it's cumbersome to switch between a bunch of spells. Gestures are both more immersive and functionally better. Maybe most people wouldn't memorize hundreds of spells, but a couple dozen are certainly feasible. There could be a spell "language," where the initial part of the gesture could correspond to the category of spell. Fireball, fire wave, etc...can start with the same basic gesture. All wave spells can end with the same gesture. Hundreds of spells can be constructed from permutations of 20-30 foundational gestures, kind of like how a core group of trilateral roots in Arabic and Hebrew can construct a much larger set of words (sometimes without previous knowledge of a specific word/permutation) https://en.wikipedia.org/wiki/Semitic_root https://en.wikipedia.org/wiki/Semitic_root
- GuiA 6y ago
  The examples you mention work well because there is a real world affordance (the wheel, the act of throwing) that the gesture maps to. I agree that rotating a wheel gesturally makes more sense as an interaction in VR than the same action does with a mouse pointer. The grenade example is a good one, because throwing/flicking doesn't require much feedback, unlike say rotation or fine grained manipulation. Of course the interaction would feel nicer if you could feel the weight of what you're throwing - the physics of throwing objects always feels kind of off in VR games - but it still works well enough. Note that many games did this with the Wiimote, and there is nothing about that interaction that is singular to VR. However, the moment you lose that affordance based in a real world object, and ask your user/player to make gestures in mid air based on recollection, things really do start to fall apart. There is a host of HCI literature/projects around this - e.g. check out papers/demos from the Kinect era. An RPG game like the one you describe could be made with Kinect, and I vaguely recall a few in development - but Kinect demonstrated well that the only kind of gestural interaction that works in long term sessions is dancing. Btw, this is why if you go to e.g. the VR center in Tokyo, almost every booth has physical props for you to hold/grip/sit on while you're fishing/golfing/shooting a gun/riding a bike in VR.

deleted 6y ago

[deleted]