3 ms·

As far as I'm concerned I already have a multi-touch hand pose interface, one that's seen more than a century of improvements. Not sure what 3D would add to it.

As far as I'm concerned I already have a multi-touch hand pose interface, one that's seen more than a century of improvements. Not sure what 3D would add to it. Perhaps just a larger attack vector for tennis elbow, i.e. literally crippling. When I see movies with 3D projected interfaces and people interacting by waving their hands around them I can't help laughing about the ergonomics of it. I have similar thoughts about the general applicability of VR. For as long as VR means focusing on a screen a couple of centimeters from your eyes, it's not going to be a generally useful productivity tool because of the eye strain. With a monitor I can easily turn my chair and focus on something else for a few minutes. Overall, it's my opinion that the goal of designing human interfaces should be to minimize cognitive load, eye strain and arm strain. Window management "as collaborative improv dance" with "3D hand poses" sounds like the exact opposite of that. The future of human computer interfaces IMO lies in things like replacing the mouse with something that requires less hand movement, technology like e-ink to improve legibility and minimize eye strain, mouseless window management, ergonomic keyboards...at least generally. I can still see how VR and 3D gesturing can be immensely useful for things like CAD and video games, but in general I use my computer to read and write documents (maybe a specialized use case in itself). Whether those documents fly around in a 3D space or are projected in thin air in front of me is uninteresting to that end. That said, to sort of answer your question, here is an interesting paper: https://our.oakland.edu/bitstream/handle/10323/4785/An%20Infinite-Pane%2C%20Zooming%20User%20Interface%20Window%20Manager%20and%20Survey%20of%20X%20Window%20Managers.pdf?sequence=1&isAllowed=y https://our.oakland.edu/bitstream/handle/10323/4785/An%20Inf...

mncharity 8y ago
> one that's seen more than a century of improvements Ah, I have tried quills a couple of times. And dip pens. It was interesting. ;) > movies [...] can't help laughing about the ergonomics of it Yes. Though they do seem to have an "outreach" role, raising awareness and interest that improvement over the familiar is possible. And moving while working, like shifting from sitting to standing desk and back, and walking, can be nice. > minimize cognitive load, eye strain and arm strain Nod. > focusing on a screen a couple of centimeters [...] eye strain The focus distance is usually either around 1 m, or infinity. Downsides include being fixed, creating vergence-accommodation conflict. Prototypes, and the new Microsoft Hololens 2 (which paints the retina), variously improve on this. > minimize cognitive load One thing I've played with, is taking a downward keyboard-cam of hands on keyboard, and overlaying it on a 3D monitor. Raising it slightly in Z seemed to allow focusing past it, making it merely bothersome, rather than untenable. Upside is it reminded me of an old Lisp Machine, with the screen telling you what various keystrokes would do in your current context, so I don't have to remember them all. > less hand movement Yes. I really like the idea of keyboard as multitouch surface. Stroke a keycap, rather than moving to a touchpad. Touch a modifier key keycap, rather than having to actually press and hold it. And optical finger tracking allowing subtle hand gestures to be commands. > read and write documents [...] 3D space or are projected in thin air in front of me is uninteresting to that end Hmm. Sometimes it's useful to print out a document, and put it up on a wall, so you can see the big picture, or talk with someone about it. If one has a task which fills the available monitor space, it's easy and common to remember what it was like with a smaller monitor, and not want to go back, but harder to imagine whether the same might happen again with further expansion. More generally, one thing that cheaper real estate might do is lower the barrier for tooling to provide speculative assistance. You're writing for an audience including someone from Great Britain, using a phrase that has different meaning there than in the US... checking that with a keystroke, or as part of a "I'm done, check it now" report, seems less attractive than "comments are available off to your side - glance if you care". Grammarly meets VR. For myself, writing documents is often preceded by a lot of graph drawing, as I figure out the structure of the domain, concept space, and presentation alternatives. I very much look forward to moving that into direct-manipulation 3D.

mncharity 8y ago

> one that's seen more than a century of improvements Ah, I have tried quills a couple of times. And dip pens. It was interesting. ;) > movies [...] can't help laughing about the ergonomics of it Yes. Though they do seem to have an "outreach" role, raising awareness and interest that improvement over the familiar is possible. And moving while working, like shifting from sitting to standing desk and back, and walking, can be nice. > minimize cognitive load, eye strain and arm strain Nod. > focusing on a screen a couple of centimeters [...] eye strain The focus distance is usually either around 1 m, or infinity. Downsides include being fixed, creating vergence-accommodation conflict. Prototypes, and the new Microsoft Hololens 2 (which paints the retina), variously improve on this. > minimize cognitive load One thing I've played with, is taking a downward keyboard-cam of hands on keyboard, and overlaying it on a 3D monitor. Raising it slightly in Z seemed to allow focusing past it, making it merely bothersome, rather than untenable. Upside is it reminded me of an old Lisp Machine, with the screen telling you what various keystrokes would do in your current context, so I don't have to remember them all. > less hand movement Yes. I really like the idea of keyboard as multitouch surface. Stroke a keycap, rather than moving to a touchpad. Touch a modifier key keycap, rather than having to actually press and hold it. And optical finger tracking allowing subtle hand gestures to be commands. > read and write documents [...] 3D space or are projected in thin air in front of me is uninteresting to that end Hmm. Sometimes it's useful to print out a document, and put it up on a wall, so you can see the big picture, or talk with someone about it. If one has a task which fills the available monitor space, it's easy and common to remember what it was like with a smaller monitor, and not want to go back, but harder to imagine whether the same might happen again with further expansion. More generally, one thing that cheaper real estate might do is lower the barrier for tooling to provide speculative assistance. You're writing for an audience including someone from Great Britain, using a phrase that has different meaning there than in the US... checking that with a keystroke, or as part of a "I'm done, check it now" report, seems less attractive than "comments are available off to your side - glance if you care". Grammarly meets VR. For myself, writing documents is often preceded by a lot of graph drawing, as I figure out the structure of the domain, concept space, and presentation alternatives. I very much look forward to moving that into direct-manipulation 3D.