9 ms·

One commonality between mouse+keyboard and smartphone is that small movements mostly in fingers and wrists are all that's needed for input. Contrast with Xbox K

One commonality between mouse+keyboard and smartphone is that small movements mostly in fingers and wrists are all that's needed for input. Contrast with Xbox Kinect - hype then disappeared, Nintendo Wii - hype then disappeared, and now some VR input...in all of these systems many of the experiences are designed around large body movements that go against the ingrained human instinct to be lazy. See "gorilla arms". AR/VR needs micro-gesture input. Logitech K830 keyboard is tracked in VR with Oculus Quest 2 which is a step in the right direction. FB acquired CTRL-Labs whose CTRL-kit allows for some elementary BCI and Valve is investing heavily in BCI, Gabe Newell can't stop talking about how excited he is about it. And I'm sure Google and MS are trying as well. But AR will need low-effort input before it replaces smartphones (unless it focuses only on high read / low write use case as its killer apps). Hopefully Apple has an input system for its AR product that doesn't require gorilla arms. AR + smartphone where the smartphone touchscreen acts as a trackpad or controller would likely be a better system than any hand wave-y / hand tracking.

TylerE 5y ago
AKA gorilla arm syndrome.
jayd16 5y ago
The hand tracking is getting pretty good if you look at a Hololens or some of the recent Quest 2 work. Trying to type with it is trash but you get much more fidelity when trying to manipulate something in 3D space than any other input system. The UX primitives are still getting worked out. Question like "What is it like to scroll in 3D AR?", "Whats the best way to input text", "What gestures can be reliably tracked from a headset?" and many others are still open questions. It will be interesting to see what shakes out. Maybe we'll end up with a lot of inter-connectivity with phones and PCs for better inputs.
- go_elmo 5y ago
  As a quest user I have to object. Even though it is very good on first impression, the pure hand tracking gets very annoying very quickly when trying to use it as it still misinterprets too often. Eye tracking will be the future of "lazy" UI, but this is a hard one. Also, distincting actions, e.g. clicks or scrolls is non trivial
  - tsimionescu 5y ago
    I don't think an eye input keyboard will ever be ergonomic for anything beyond 'lol'. Repetitive strain injury is bad enough in hands, I shudder to think how it would feel if you were using your eyes.
    - go_elmo 5y ago
      yeah, not as keyboard, but e.g. mouse? Would even make todays VR much better / less anoying imo
nullifidian 5y ago
The AR case could probably be solved with a ring(in the jewelry meaning of the word 'ring')-like controller accessory, with a little touch pad and a couple of buttons. Hopefully it could be done with reasonable work time per battery charge. I don't believe in hand tracking too much, for the first generation of AR headsets at least. People complained about latency a lot, and many use cases are simply impossible to reconstruct accurately with headset camera tracking. For example changing direction in a laser pointer mode, by lazily shifting your controller between your thumb and your pointing finger is something neural hand tracking would be hard pressed to reproduce adequately.
CodeGlitch 5y ago
> One commonality between mouse+keyboard and smartphone is that small movements mostly in fingers and wrists are all that's needed for input How about pairing the glasses with a smart watch? And the input comes from eye movement and touching the screen on the watch? Voice can also act as "low effort input"

TylerE 5y ago

AKA gorilla arm syndrome.

jayd16 5y ago

The hand tracking is getting pretty good if you look at a Hololens or some of the recent Quest 2 work. Trying to type with it is trash but you get much more fidelity when trying to manipulate something in 3D space than any other input system. The UX primitives are still getting worked out. Question like "What is it like to scroll in 3D AR?", "Whats the best way to input text", "What gestures can be reliably tracked from a headset?" and many others are still open questions. It will be interesting to see what shakes out. Maybe we'll end up with a lot of inter-connectivity with phones and PCs for better inputs.

go_elmo 5y ago
As a quest user I have to object. Even though it is very good on first impression, the pure hand tracking gets very annoying very quickly when trying to use it as it still misinterprets too often. Eye tracking will be the future of "lazy" UI, but this is a hard one. Also, distincting actions, e.g. clicks or scrolls is non trivial
- tsimionescu 5y ago
  I don't think an eye input keyboard will ever be ergonomic for anything beyond 'lol'. Repetitive strain injury is bad enough in hands, I shudder to think how it would feel if you were using your eyes.
  - go_elmo 5y ago
    yeah, not as keyboard, but e.g. mouse? Would even make todays VR much better / less anoying imo

nullifidian 5y ago

The AR case could probably be solved with a ring(in the jewelry meaning of the word 'ring')-like controller accessory, with a little touch pad and a couple of buttons. Hopefully it could be done with reasonable work time per battery charge. I don't believe in hand tracking too much, for the first generation of AR headsets at least. People complained about latency a lot, and many use cases are simply impossible to reconstruct accurately with headset camera tracking. For example changing direction in a laser pointer mode, by lazily shifting your controller between your thumb and your pointing finger is something neural hand tracking would be hard pressed to reproduce adequately.

CodeGlitch 5y ago

> One commonality between mouse+keyboard and smartphone is that small movements mostly in fingers and wrists are all that's needed for input How about pairing the glasses with a smart watch? And the input comes from eye movement and touching the screen on the watch? Voice can also act as "low effort input"