6 ms·

Well looking at the video, I mean, I can have dozens workspaces with hundreds of windows (if I need to) and navigate them with moving just my fingers, whereas y

Well looking at the video, I mean, I can have dozens workspaces with hundreds of windows (if I need to) and navigate them with moving just my fingers, whereas you are apparently breaking your neck to navigate more than 3 windows. I think what you're doing is cool and has to be done for the sake of exploration, but I would really, really focus on navigation. Also, how are you supposed to focus on text, when reading or reading or writing when it's wiggling all the time. It seems to me that VR really is for games, but for working with text I think humans are better off with stationary screens. On a second thought: if you would make it easier to "fix" window, such that one can focus on it and work. As when you are "placing" a new window in your video, that could make it a workable solution.

kanetw 7y ago
Navigation is one of the things we definitely need to improve upon. I think the wiggling is actually kind of natural -- you don't notice your head or eye movements when you do them automatically, but it might be exacerbated in VR. The fixing is an interesting idea, we'll try it out. I'm a bit worried it might feel weird in VR.
- todd8 7y ago
  Yes, there is a thing called corollary discharge in the human visual system that keeps the world steady as one glances around (in the real world). Changing the direction one is looking causes the image to shift rapidly on the retina, yet perceptually we see a world that is not wiggling. Corollary discharge[1] causes the brain to “subtract” out the motion of the image on the retina. It happens when our nervous system instructs the eyes to look in another direction. You can have someone else move your eyes or even do it yourself by pressing gently on the side of the eyeball with a fingertip on top of the lid. Wiggling one eye this way is not supported by the hardware of the eye and produces a very noticeable wiggly view of the real world. I mentioned subtracting out the motion going on in the visual system. Experiments suggest that something like this is happening. If a muscle paralyzing drug is injected into the eye’s muscles it causes a wiggly world because the intent to glance somewhere causes both an ineffective command to move the eye, producing no real movement of the image in the retina while the triggered corollary discharge subtracts the intended motion from the still image and causes a perceived motion (in the opposite direction). [1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2807735/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2807735/
- ggreer 7y ago
  YouTube has support for VR videos. If you could record & render such that people with headsets could watch a session, that might help with interest and adoption. I think navigation needs eye tracking, which sadly no headset currently supports. Focus-follows-gaze would be a game changer.
  - georgewsinger 7y ago
    Uploading a YouTube VR video of a work session is a potentially fantastic idea. Thanks for the suggestion. Right now, Simula uses "dumb" eye tracking, in the sense that windows receive keyboard and cursor focus when the user's forward eye gaze intersects a window. We also have it so that users can control the cursor focus with their forward gaze (presently binded to `Super + Apostrophe`); similarly, users can drag windows around by holding `Super + Alt` and looking around. The experience adds up to something quite productive once you learn all the keyboard shortcuts (your fingers don't need to leave the keyboard).
  - anp 7y ago
    I bought a Vive Pro Eye specifically to try and develop eye tracking navigation for a system like this. I haven’t done any actual digging yet bc it turns out the NDAs for their SDK are...unfortunate. Definitely an opportunity for a free/open alternative.
TooKool4This 7y ago
I haven't used VR devices too much but I believe the wiggling that you see in VR demos is not present when actually using the device. The device is simply adjusting the perspective to match your head motion and so the actual user is not seeing the wiggling but in the demo it appears that the view is wiggling to the watcher. But I agree that wiggling is off putting. I think VR demos should low-pass filter the output to some degree to give a more realistic perception of what VR is like.
- kanetw 7y ago
  I wonder if a separate camera for pancake with a larger FOV would help, with the translation (e: and rotation) matrix lowpass-filtered from user orientation.
  - TooKool4This 7y ago
    Not sure what you mean by pancake. But I feel its mostly rotational motion that is perceptible in VR demos rather than translational motion. I suspect that's because a small rotational motion results in a large shift in perspective for objects in the world that are some distance away from the camera while small translational movements don't really result in a significant perspective shift. Would be interested in seeing what research into human head motion while sitting/standing/walking/etc. shows though.
    - kanetw 7y ago
      Pancake is the 2D view observers see when they're looking at the monitor (i.e. that what gets recorded). Yeah, both rotation and translation would be filtered in the idea I had, so that small wiggling motions aren't as visible.
  - gpm 7y ago
    I've seen "motion smoothing" options for VR spectators in some games - not entirely sure that's what they do, but I assume it is.
techopoly 7y ago
Virtual Desktop already has many customizable features that allow you to work the way you're describing. I've worked in it before, and it was very immersive, though there was of course some fatigue. You can have just one window at a time, and with a good headset, the wiggle is not significant. You can even adjust it so that your workspace follows you when your head turns.
- georgewsinger 7y ago
  RE eyestrain: here are a couple of links showing off Simula's text quality: https://github.com/SimulaVR/Simula/blob/gdwlroots-xwayland/doc/TextQuality.gif https://github.com/SimulaVR/Simula/blob/gdwlroots-xwayland/d... https://github.com/SimulaVR/Simula/blob/gdwlroots-xwayland/doc/TextQuality2.gif https://github.com/SimulaVR/Simula/blob/gdwlroots-xwayland/d... I haven't used Windows' Virtual Desktop since the early days, but I suspect our text enhancements have improved upon the situation dramatically (when keeping hardware constant).
nsxwolf 7y ago
There is no wiggling. That wiggling you see in videos is your head movements not being in sync with the wearer’s head movements. You’re doing it right now, your brain is just anticipating and compensating for it.
deleted 7y ago
[deleted]
Kiro 7y ago
> It seems to me that VR really is for games, but for working with text I think humans are better off with stationary screens. Have you actually tried VR? The fact that you think there's wiggling makes me think you haven't. Yes, videos make it look that way but I can guarantee you that's not reflected in the actual experience. VR is notoriously bad to watch on video and doesn't capture the experience at all.
numpad0 7y ago
Actually the problem is low resolution and FOV of VR headsets. VR require at least 72Hz framerate at 4Kish which result in 2K 72Hz 90° ish stereo and that’s hard already even for moderate gaming PCs. As for pinned windows: fixed HUD is a no-no in VR games because it burns in same spot of your eyesight. When a HUD is desired, players are usually given a virtual or symbolic helmet that allow display to lag behind head movement.

kanetw 7y ago

Navigation is one of the things we definitely need to improve upon. I think the wiggling is actually kind of natural -- you don't notice your head or eye movements when you do them automatically, but it might be exacerbated in VR. The fixing is an interesting idea, we'll try it out. I'm a bit worried it might feel weird in VR.

todd8 7y ago
Yes, there is a thing called corollary discharge in the human visual system that keeps the world steady as one glances around (in the real world). Changing the direction one is looking causes the image to shift rapidly on the retina, yet perceptually we see a world that is not wiggling. Corollary discharge[1] causes the brain to “subtract” out the motion of the image on the retina. It happens when our nervous system instructs the eyes to look in another direction. You can have someone else move your eyes or even do it yourself by pressing gently on the side of the eyeball with a fingertip on top of the lid. Wiggling one eye this way is not supported by the hardware of the eye and produces a very noticeable wiggly view of the real world. I mentioned subtracting out the motion going on in the visual system. Experiments suggest that something like this is happening. If a muscle paralyzing drug is injected into the eye’s muscles it causes a wiggly world because the intent to glance somewhere causes both an ineffective command to move the eye, producing no real movement of the image in the retina while the triggered corollary discharge subtracts the intended motion from the still image and causes a perceived motion (in the opposite direction). [1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2807735/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2807735/
ggreer 7y ago
YouTube has support for VR videos. If you could record & render such that people with headsets could watch a session, that might help with interest and adoption. I think navigation needs eye tracking, which sadly no headset currently supports. Focus-follows-gaze would be a game changer.
- georgewsinger 7y ago
  Uploading a YouTube VR video of a work session is a potentially fantastic idea. Thanks for the suggestion. Right now, Simula uses "dumb" eye tracking, in the sense that windows receive keyboard and cursor focus when the user's forward eye gaze intersects a window. We also have it so that users can control the cursor focus with their forward gaze (presently binded to `Super + Apostrophe`); similarly, users can drag windows around by holding `Super + Alt` and looking around. The experience adds up to something quite productive once you learn all the keyboard shortcuts (your fingers don't need to leave the keyboard).
- anp 7y ago
  I bought a Vive Pro Eye specifically to try and develop eye tracking navigation for a system like this. I haven’t done any actual digging yet bc it turns out the NDAs for their SDK are...unfortunate. Definitely an opportunity for a free/open alternative.

TooKool4This 7y ago

I haven't used VR devices too much but I believe the wiggling that you see in VR demos is not present when actually using the device. The device is simply adjusting the perspective to match your head motion and so the actual user is not seeing the wiggling but in the demo it appears that the view is wiggling to the watcher. But I agree that wiggling is off putting. I think VR demos should low-pass filter the output to some degree to give a more realistic perception of what VR is like.

kanetw 7y ago
I wonder if a separate camera for pancake with a larger FOV would help, with the translation (e: and rotation) matrix lowpass-filtered from user orientation.
- TooKool4This 7y ago
  Not sure what you mean by pancake. But I feel its mostly rotational motion that is perceptible in VR demos rather than translational motion. I suspect that's because a small rotational motion results in a large shift in perspective for objects in the world that are some distance away from the camera while small translational movements don't really result in a significant perspective shift. Would be interested in seeing what research into human head motion while sitting/standing/walking/etc. shows though.
  - kanetw 7y ago
    Pancake is the 2D view observers see when they're looking at the monitor (i.e. that what gets recorded). Yeah, both rotation and translation would be filtered in the idea I had, so that small wiggling motions aren't as visible.
- gpm 7y ago
  I've seen "motion smoothing" options for VR spectators in some games - not entirely sure that's what they do, but I assume it is.

techopoly 7y ago

Virtual Desktop already has many customizable features that allow you to work the way you're describing. I've worked in it before, and it was very immersive, though there was of course some fatigue. You can have just one window at a time, and with a good headset, the wiggle is not significant. You can even adjust it so that your workspace follows you when your head turns.

georgewsinger 7y ago
RE eyestrain: here are a couple of links showing off Simula's text quality: https://github.com/SimulaVR/Simula/blob/gdwlroots-xwayland/doc/TextQuality.gif https://github.com/SimulaVR/Simula/blob/gdwlroots-xwayland/d... https://github.com/SimulaVR/Simula/blob/gdwlroots-xwayland/doc/TextQuality2.gif https://github.com/SimulaVR/Simula/blob/gdwlroots-xwayland/d... I haven't used Windows' Virtual Desktop since the early days, but I suspect our text enhancements have improved upon the situation dramatically (when keeping hardware constant).

nsxwolf 7y ago

There is no wiggling. That wiggling you see in videos is your head movements not being in sync with the wearer’s head movements. You’re doing it right now, your brain is just anticipating and compensating for it.

deleted 7y ago

[deleted]

Kiro 7y ago

> It seems to me that VR really is for games, but for working with text I think humans are better off with stationary screens. Have you actually tried VR? The fact that you think there's wiggling makes me think you haven't. Yes, videos make it look that way but I can guarantee you that's not reflected in the actual experience. VR is notoriously bad to watch on video and doesn't capture the experience at all.

numpad0 7y ago

Actually the problem is low resolution and FOV of VR headsets. VR require at least 72Hz framerate at 4Kish which result in 2K 72Hz 90° ish stereo and that’s hard already even for moderate gaming PCs. As for pinned windows: fixed HUD is a no-no in VR games because it burns in same spot of your eyesight. When a HUD is desired, players are usually given a virtual or symbolic helmet that allow display to lag behind head movement.