4 ms·

The current generation of headsets is unsuited for showing large amounts of reasonably sized text. The display resolution is not there yet. If you map your desk

The current generation of headsets is unsuited for showing large amounts of reasonably sized text. The display resolution is not there yet. If you map your desktop into a plane in VR space at the proper distance, your UI text shows up heavily aliased and barely readable due to the texture resampling that happens in that process. You need to increase resolution by another factor of two or three to make that actually convenient. And then there is the issue with holding 3d pointing devices for prolonged amounts of time. This is taxing upper body strength if you do. The weight of thenheadset itself is also straining the neck muscles. All of this taken together usually means that I want out of VR after about 2 hours. I barely have sessions that are longer.

Zak 7y ago
What's the resolution of the headset you've been using? I don't think I'd like my desktop to have a lot of 3D features or require a 3D pointing device. What I'm imagining is essentially a giant 2D workspace where I can still use a Trackpoint.
- gmueckl 7y ago
  I've used or tried most of the major headsets available in the market. None of them is remotely close to a resolution where reading reasonably sized text isn't straining. The large screen concept is a bit problematic. A large planar screen distorts the portions you're not in front of because you're only seeing them at pretty extreme angles. A cylindrical screen distorts projected content, e.g. photos or 3d graphics displayed on it. A better solution would be to treat each window as a flat rectangle that can be freely positioned in 3d. So you can arrange the most important window around you in ways that you can conveniently look at them by turning your head and you'd still look at them dead on. For implementing that, a 3D pointer is by far the best option.
  - Zak 7y ago
    What about a flat rectangle that's automatically facing the user's face as if pinned to the inside of a sphere? Whatever window you're facing should have the right orientation.
  - mncharity 7y ago
    > I've used or tried most of the major headsets available in the market. None of them is remotely close to a resolution where reading reasonably sized text isn't straining. I use DJI's 1080p drone goggles. With DIY shutter glasses, because DJI didn't expose independent panel inputs. :/ It looks like a 3D TV screen, not VR, but you get readable small fonts from corner-to-corner (almost, depending on whether you also need to use reading glasses with it). But with batteries, they indeed aren't light weight. Nreal's AR glasses could be available in a couple of months. > problematic [...] angles [...] positioned in 3d [...] arrange [...] around you The skeuomorphic design phase with phones was a short-lived part of onboarding an untrained user population. For non-novice professional user interfaces in XR, I suggest it's... silly. :) But oddly reoccurring, perhaps because of game tooling's availability?
    - gmueckl 7y ago
      What I wrote has nothing to do with skeuomorphism, only with human anatomy, perception and pretty basic geometry. I am not proposing that windows get rendered with 3d modelled grips for grabbing and moving.
      - mncharity 7y ago
        When an artist draws with a display-less graphics tablet, the hand is in one place, but they watch and use an abstraction of it on a display, displaced in space, orientation, and scale. It can take some getting used to. Touchpads and cursors have similar non-trivial relationships. And similarly, there's no reason for your hands to be positioned at the end of your arms (as with "throwy hands"). Nor, with non-novice users, need any other physical correlation be maintained if it's useful to ignore it. When you use office apps, there's little attempt to maintain physical-world properties like say object permanence. Delete a word, and it "goes someplace, because it can't just vanish"? That would be silly. There's no need to emulate physical paper, or pens, or rubber erasers, or cut-and-paste with an xacto knife, paper fragments, and gooey paste from a jar. It can sometimes be useful or artistically fun to do so, but that's a UX choice. Or say you're looking at a whole-desktop "window", placed uncomfortably close and large, to be readable despite current HMD's limited angular resolution. When you turn your head, how much does it move? With what velocity profile? And with what alterations in appearance? Skeuomorphism suggests it appear as a real physical object, nailed in space. I found I preferred a quantized 2-3x displacement, to reduce head motion, and enabled it by transiently increased transparency (including "comfort mode" clipping from fov), with slightly delayed motion and added jiggle, so other visual cues could dominate wrt balance. And the shape distorted to maintain pixel alignment, because given my weak gpu, alignment was worth more than temporal antialiasing when reading small fonts. YMMV, as just what combinations of cues are fine vs intolerable varies so much among people. The common thread is aphysicallity. Skeuomorphism can be useful when onboarding novices, but is later often largely abandoned. As with phones. With the current focus on immersive games and novices, that second part tends to be neglected. I joke that while I use XR hardware, I'm not doing XR, because being focused on non-game non-novice UX, and using a custom non-gaming stack, I've no allegiance to the "R", to Reality. Resemblance to reality as UX design smell.
asark 7y ago
> And then there is the issue with holding 3d pointing devices for prolonged amounts of time. This is taxing upper body strength if you do. The weight of the headset itself is also straining the neck muscles. You could use a real mouse and keyboard, if you're presented with some kind of virtual workstation. And even for navigation outside the workstation, of your broader "environment", 3D video games have been using a mouse & keyboard to navigate complex interfaces & environments for a long time, with more than a little success. Granted that means you still need a surface of some sort, but I think non-touch-typing text input inside a VR space would be kinda hellish anyway, so you're at least gonna want a real keyboard regardless.
- cjbprime 7y ago
  A keyboard with trackpoint in the middle would probably be a good compromise, no fumbling to find where you left the mouse.
- gmueckl 7y ago
  Navigation in VR is a barely solved problem. Most games have players teleport between positions instantly because a continous motion would create a sensation of conflict between balance reported by the inner ear and the perceived motion. That means instant nausea. Positioning objects in 3D precisely and conveniently without a 3D pointing device is hard, too. The best option for that is a mouse and crutches like handles oriented along coordinate axes. But that only works because the mouse pointer has a reference plane it moves on (the projection plane for the 3D view). In VR, this plane is tracking the head precisely. You cannot reasonably put the mouse cursor on that plane. You would need a different reference plane and then you're back to all the problems that 3D input on a 2D screen has. And a positioning objects in 3D with a keyboard is even more tedious.
- mceachen 7y ago
  They were demoing with knuckles: https://store.steampowered.com/app/1059550/Valve_Index_Controllers/ https://store.steampowered.com/app/1059550/Valve_Index_Contr... They wrap around your hand, so you can type (ish) and not take them off. I'd think it would be fantastic to physically walk around to different monitor/window/server setups (in VR), though, and that'd require dragging around a rolling standup desk, or figuring out how to type based on finger positions from the controllers (which I'd imagine would be decidedly not great).
dreamcompiler 7y ago
Lighter, higher-resolution headsets + full keyboard control = 3D Emacs!
- mncharity 7y ago
  Yes. If anyone knows of any existing code to support say 3D cross/parallel-eye stereo views, with z-axis displaced words, in an emacs buffer being actively edited, I'd like to hear of it. Teaching your window manager to be 3D is fun, but the next step is to teach your apps.
  - gmueckl 7y ago
    You are welcome to try, but I can tell you right now that a UI that places words in a text at different depths is a terrible idea. Depths is simulated by varying parallax and vergence. And forcing a permanent vergence adjustment is a strain on the eye muscles. Plus you get to enjoy the vergence-focus mismatch from the fixed headset focal distance to the fullest. This is a headache-inducing cue.
    - mncharity 7y ago
      Which is why you keep your primary text plane at focus depth, and emphasize shallow 3D displacement. You can do a lot with a mere cm of depth around your laptop screen, or a similarly scaled annulus at an HMD's 2 meter focus distance.
      - gmueckl 7y ago
        This results in an interesting question: I don't know if the available headset resolutions are sufficient to provise these quite subtle vergence cues.
        mncharity 7y ago
        Hmm. For VR HMDs, with RGB subpixels (non-PenTile), you can read a pixel-aligned 5 pt character in the center, but you don't get many of them (because of lens blur, and just VR lensing), and the pixels are a bit large. I've no idea how well SteamVR/Unreal/Unity currently let you use them. The lenses of Valve's Index were praised elsewhere here. AR HMDs trade narrower FoV for angularly smaller pixels, and you can use "all" of them. But the pixels may be poorly aligned between eyes, as with Project North Star. Maybe Hololens 2 (with different a-v issues). I can't comment on Magic Leap. I look forward to trying next month's/year's Nreal glasses. Media-viewing and drone HMDs offer a screen at some distance. Some are stereo 3D. Some are 1080p. Few have head tracking. You can add it with Intel t265, Zed, Structure Core (no linux), or DIY. Many have a "diopter" adjustment. They've varied build quality, and supported nose sizes. Some months back, I failed to find a 3D 1080p HMD, and so currently use mono 1080p DJI drone goggles (discontinued cheaper white model), with anaglyph (crushing the color space to reduce eye strain) or DIY shutter (flickery 30 or 20 Hz) to recover 3D. :/ Usually with reading glasses, despite my old eyes being nearsighted, which costs some edge pixels. DIY optical tracking, so no temporal antialiasing. I'd not recommend it. My current hope is for a Q4, 1080p, 50-ish degree FoV, 3D "screen" HMD (media-viewer or AR). With either head tracking built in, or COTS. It may need a Windows box in support, especially for hand tracking. I'll likely stay with a simple custom browser-as-compositor stack, for low-cost pixel-precise control, but a mainstream VR or AR stack might work too.
erikpukinskis 7y ago
The difference between a headset and a monitor is that in VR when you lean in the text gets sharper.
- gmueckl 7y ago
  Leaning in is not ergonomic over longer periods of time and may be very undesirable in constrained spaces. I find myself actually doing that a lot when interacting with the desktop in VR and it becomes physically straining after a few minutes.
deleted 7y ago
[deleted]

Zak 7y ago

What's the resolution of the headset you've been using? I don't think I'd like my desktop to have a lot of 3D features or require a 3D pointing device. What I'm imagining is essentially a giant 2D workspace where I can still use a Trackpoint.

gmueckl 7y ago
I've used or tried most of the major headsets available in the market. None of them is remotely close to a resolution where reading reasonably sized text isn't straining. The large screen concept is a bit problematic. A large planar screen distorts the portions you're not in front of because you're only seeing them at pretty extreme angles. A cylindrical screen distorts projected content, e.g. photos or 3d graphics displayed on it. A better solution would be to treat each window as a flat rectangle that can be freely positioned in 3d. So you can arrange the most important window around you in ways that you can conveniently look at them by turning your head and you'd still look at them dead on. For implementing that, a 3D pointer is by far the best option.
- Zak 7y ago
  What about a flat rectangle that's automatically facing the user's face as if pinned to the inside of a sphere? Whatever window you're facing should have the right orientation.
- mncharity 7y ago
  > I've used or tried most of the major headsets available in the market. None of them is remotely close to a resolution where reading reasonably sized text isn't straining. I use DJI's 1080p drone goggles. With DIY shutter glasses, because DJI didn't expose independent panel inputs. :/ It looks like a 3D TV screen, not VR, but you get readable small fonts from corner-to-corner (almost, depending on whether you also need to use reading glasses with it). But with batteries, they indeed aren't light weight. Nreal's AR glasses could be available in a couple of months. > problematic [...] angles [...] positioned in 3d [...] arrange [...] around you The skeuomorphic design phase with phones was a short-lived part of onboarding an untrained user population. For non-novice professional user interfaces in XR, I suggest it's... silly. :) But oddly reoccurring, perhaps because of game tooling's availability?
  - gmueckl 7y ago
    What I wrote has nothing to do with skeuomorphism, only with human anatomy, perception and pretty basic geometry. I am not proposing that windows get rendered with 3d modelled grips for grabbing and moving.
    - mncharity 7y ago
      When an artist draws with a display-less graphics tablet, the hand is in one place, but they watch and use an abstraction of it on a display, displaced in space, orientation, and scale. It can take some getting used to. Touchpads and cursors have similar non-trivial relationships. And similarly, there's no reason for your hands to be positioned at the end of your arms (as with "throwy hands"). Nor, with non-novice users, need any other physical correlation be maintained if it's useful to ignore it. When you use office apps, there's little attempt to maintain physical-world properties like say object permanence. Delete a word, and it "goes someplace, because it can't just vanish"? That would be silly. There's no need to emulate physical paper, or pens, or rubber erasers, or cut-and-paste with an xacto knife, paper fragments, and gooey paste from a jar. It can sometimes be useful or artistically fun to do so, but that's a UX choice. Or say you're looking at a whole-desktop "window", placed uncomfortably close and large, to be readable despite current HMD's limited angular resolution. When you turn your head, how much does it move? With what velocity profile? And with what alterations in appearance? Skeuomorphism suggests it appear as a real physical object, nailed in space. I found I preferred a quantized 2-3x displacement, to reduce head motion, and enabled it by transiently increased transparency (including "comfort mode" clipping from fov), with slightly delayed motion and added jiggle, so other visual cues could dominate wrt balance. And the shape distorted to maintain pixel alignment, because given my weak gpu, alignment was worth more than temporal antialiasing when reading small fonts. YMMV, as just what combinations of cues are fine vs intolerable varies so much among people. The common thread is aphysicallity. Skeuomorphism can be useful when onboarding novices, but is later often largely abandoned. As with phones. With the current focus on immersive games and novices, that second part tends to be neglected. I joke that while I use XR hardware, I'm not doing XR, because being focused on non-game non-novice UX, and using a custom non-gaming stack, I've no allegiance to the "R", to Reality. Resemblance to reality as UX design smell.

asark 7y ago

> And then there is the issue with holding 3d pointing devices for prolonged amounts of time. This is taxing upper body strength if you do. The weight of the headset itself is also straining the neck muscles. You could use a real mouse and keyboard, if you're presented with some kind of virtual workstation. And even for navigation outside the workstation, of your broader "environment", 3D video games have been using a mouse & keyboard to navigate complex interfaces & environments for a long time, with more than a little success. Granted that means you still need a surface of some sort, but I think non-touch-typing text input inside a VR space would be kinda hellish anyway, so you're at least gonna want a real keyboard regardless.

cjbprime 7y ago
A keyboard with trackpoint in the middle would probably be a good compromise, no fumbling to find where you left the mouse.
gmueckl 7y ago
Navigation in VR is a barely solved problem. Most games have players teleport between positions instantly because a continous motion would create a sensation of conflict between balance reported by the inner ear and the perceived motion. That means instant nausea. Positioning objects in 3D precisely and conveniently without a 3D pointing device is hard, too. The best option for that is a mouse and crutches like handles oriented along coordinate axes. But that only works because the mouse pointer has a reference plane it moves on (the projection plane for the 3D view). In VR, this plane is tracking the head precisely. You cannot reasonably put the mouse cursor on that plane. You would need a different reference plane and then you're back to all the problems that 3D input on a 2D screen has. And a positioning objects in 3D with a keyboard is even more tedious.
mceachen 7y ago
They were demoing with knuckles: https://store.steampowered.com/app/1059550/Valve_Index_Controllers/ https://store.steampowered.com/app/1059550/Valve_Index_Contr... They wrap around your hand, so you can type (ish) and not take them off. I'd think it would be fantastic to physically walk around to different monitor/window/server setups (in VR), though, and that'd require dragging around a rolling standup desk, or figuring out how to type based on finger positions from the controllers (which I'd imagine would be decidedly not great).

dreamcompiler 7y ago

Lighter, higher-resolution headsets + full keyboard control = 3D Emacs!

mncharity 7y ago
Yes. If anyone knows of any existing code to support say 3D cross/parallel-eye stereo views, with z-axis displaced words, in an emacs buffer being actively edited, I'd like to hear of it. Teaching your window manager to be 3D is fun, but the next step is to teach your apps.
- gmueckl 7y ago
  You are welcome to try, but I can tell you right now that a UI that places words in a text at different depths is a terrible idea. Depths is simulated by varying parallax and vergence. And forcing a permanent vergence adjustment is a strain on the eye muscles. Plus you get to enjoy the vergence-focus mismatch from the fixed headset focal distance to the fullest. This is a headache-inducing cue.
  - mncharity 7y ago
    Which is why you keep your primary text plane at focus depth, and emphasize shallow 3D displacement. You can do a lot with a mere cm of depth around your laptop screen, or a similarly scaled annulus at an HMD's 2 meter focus distance.
    - gmueckl 7y ago
      This results in an interesting question: I don't know if the available headset resolutions are sufficient to provise these quite subtle vergence cues.
      - mncharity 7y ago
        Hmm. For VR HMDs, with RGB subpixels (non-PenTile), you can read a pixel-aligned 5 pt character in the center, but you don't get many of them (because of lens blur, and just VR lensing), and the pixels are a bit large. I've no idea how well SteamVR/Unreal/Unity currently let you use them. The lenses of Valve's Index were praised elsewhere here. AR HMDs trade narrower FoV for angularly smaller pixels, and you can use "all" of them. But the pixels may be poorly aligned between eyes, as with Project North Star. Maybe Hololens 2 (with different a-v issues). I can't comment on Magic Leap. I look forward to trying next month's/year's Nreal glasses. Media-viewing and drone HMDs offer a screen at some distance. Some are stereo 3D. Some are 1080p. Few have head tracking. You can add it with Intel t265, Zed, Structure Core (no linux), or DIY. Many have a "diopter" adjustment. They've varied build quality, and supported nose sizes. Some months back, I failed to find a 3D 1080p HMD, and so currently use mono 1080p DJI drone goggles (discontinued cheaper white model), with anaglyph (crushing the color space to reduce eye strain) or DIY shutter (flickery 30 or 20 Hz) to recover 3D. :/ Usually with reading glasses, despite my old eyes being nearsighted, which costs some edge pixels. DIY optical tracking, so no temporal antialiasing. I'd not recommend it. My current hope is for a Q4, 1080p, 50-ish degree FoV, 3D "screen" HMD (media-viewer or AR). With either head tracking built in, or COTS. It may need a Windows box in support, especially for hand tracking. I'll likely stay with a simple custom browser-as-compositor stack, for low-cost pixel-precise control, but a mainstream VR or AR stack might work too.

erikpukinskis 7y ago

The difference between a headset and a monitor is that in VR when you lean in the text gets sharper.

gmueckl 7y ago
Leaning in is not ergonomic over longer periods of time and may be very undesirable in constrained spaces. I find myself actually doing that a lot when interacting with the desktop in VR and it becomes physically straining after a few minutes.

deleted 7y ago

[deleted]