6 ms·

In case people are curious what it's like to work in VR, here's a demo of me using Simula: https://youtu.be/FWLuwG91HnI https://youtu.be/FWLuwG91HnI I would ar

In case people are curious what it's like to work in VR, here's a demo of me using Simula: https://youtu.be/FWLuwG91HnI https://youtu.be/FWLuwG91HnI I would argue that working in VR is fundamentally superior to working in PCs/laptops. It's basically 10x the screens, focus, and work immersion. Simula's text quality is also very good (getting around the eye fatigue present in older VR Desktops).

outworlder 7y ago
Is it possible to have a giant curved "screen"? I feel we are just scratching the surface of what's possible. Yes, multiple (arbitrarily sized) windows are neat. But we can probably do more. Even without any changes though, I just wish my VR headset was lighter. I can go more than 8h straight in VR... playing games. Not sure about working. Will try this :)
- kanetw 7y ago
  > Is it possible to have a giant curved "screen"? SteamVR does this, but I've found it less useful than movable/resizable windows. > I feel we are just scratching the surface of what's possible. Definitely. This is one of the first viable iterations, but there's lots and lots of improvement potential everywhere. > Even without any changes though, I just wish my VR headset was lighter. Yeah, agreed. But I think future hardware will improve on that.
- zachrose 7y ago
  Non-Euclidean UI
skratlo 7y ago
Well looking at the video, I mean, I can have dozens workspaces with hundreds of windows (if I need to) and navigate them with moving just my fingers, whereas you are apparently breaking your neck to navigate more than 3 windows. I think what you're doing is cool and has to be done for the sake of exploration, but I would really, really focus on navigation. Also, how are you supposed to focus on text, when reading or reading or writing when it's wiggling all the time. It seems to me that VR really is for games, but for working with text I think humans are better off with stationary screens. On a second thought: if you would make it easier to "fix" window, such that one can focus on it and work. As when you are "placing" a new window in your video, that could make it a workable solution.
- kanetw 7y ago
  Navigation is one of the things we definitely need to improve upon. I think the wiggling is actually kind of natural -- you don't notice your head or eye movements when you do them automatically, but it might be exacerbated in VR. The fixing is an interesting idea, we'll try it out. I'm a bit worried it might feel weird in VR.
  - todd8 7y ago
    Yes, there is a thing called corollary discharge in the human visual system that keeps the world steady as one glances around (in the real world). Changing the direction one is looking causes the image to shift rapidly on the retina, yet perceptually we see a world that is not wiggling. Corollary discharge[1] causes the brain to “subtract” out the motion of the image on the retina. It happens when our nervous system instructs the eyes to look in another direction. You can have someone else move your eyes or even do it yourself by pressing gently on the side of the eyeball with a fingertip on top of the lid. Wiggling one eye this way is not supported by the hardware of the eye and produces a very noticeable wiggly view of the real world. I mentioned subtracting out the motion going on in the visual system. Experiments suggest that something like this is happening. If a muscle paralyzing drug is injected into the eye’s muscles it causes a wiggly world because the intent to glance somewhere causes both an ineffective command to move the eye, producing no real movement of the image in the retina while the triggered corollary discharge subtracts the intended motion from the still image and causes a perceived motion (in the opposite direction). [1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2807735/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2807735/
  - ggreer 7y ago
    YouTube has support for VR videos. If you could record & render such that people with headsets could watch a session, that might help with interest and adoption. I think navigation needs eye tracking, which sadly no headset currently supports. Focus-follows-gaze would be a game changer.
    - georgewsinger 7y ago
lukifer 7y ago
How is the eye fatigue in general? How long can you go in a VR work session? For VR gaming (Oculus Rift), I start to experience mild discomfort after about 30min, which grows severe by 60min. It's hard to envision using VR for a full work day. (My eyes are fairly light-sensitive from laser-eye surgery; not sure how much is saccading, vs. bright light directly in front of the eyes.)
- georgewsinger 7y ago
  RE eye strain, here are a couple of links showing off Simula's text quality: https://github.com/SimulaVR/Simula/blob/gdwlroots-xwayland/doc/TextQuality.gif https://github.com/SimulaVR/Simula/blob/gdwlroots-xwayland/d... https://github.com/SimulaVR/Simula/blob/gdwlroots-xwayland/doc/TextQuality2.gif https://github.com/SimulaVR/Simula/blob/gdwlroots-xwayland/d... Many early VR Desktops (and VR games) didn't go to special effort to optimize text quality, and I think this has given people a lower than warranted impression of what is possible in VR today. There's no doubt that VR hardware is going to get exponentially lighter/sharper over time, and that these improvements will be welcome, but things are good enough now to put in 1hr+ long sessions without eye strain (at least in my experience as a daily user).
- mhh__ 7y ago
  It really depends on the task - I (currently) have an OG Vive, and its absolutely fine for games with good anti-aliasing (no discomfort for me for basically any length of time, although I mainly play beatsaber and you don't really focus on the blocks beyond blinks of light and muscle memory), but it's hopeless if you want to watch a movie let alone program. The kind of programming I tend to do is generally fiddly enough that I've got a book open or some kind of datasheet etc. (I think - even good - VR would be a hindrance here) I think the Index might be OK but I couldn't comment.
- driverdan 7y ago
  I can't speak to working with text but I could easily play games all day on a Vive with no eye strain.
emj 7y ago
I would not be able to work like that, it would lead to neck strain or wrist pain, if you think about it comes naturally to reposition your body when you are looking in a new direction there is a reason for this. This hurts me because the keyboard locks the shoulders and it's only my heavy head that moves, this is not ergonomical. So being able to fetch up windows so it's perfectly centered above the keyboard would make it easier for me to use.
- fractal618 7y ago
  Maybe there should be a harness to suspend us by one ankle while we work
Lambdanaut 7y ago
I saw that you were hacking on... Haskell..? In that video? Where does Haskell fit in with Godot and Simula?
- kanetw 7y ago
  SimulaVR (unrelated to Simula) is written mostly in Haskell, with Godot as the engine.
cesarb 7y ago
An interesting potential for VR would be alternative portable form factors. The laptop form factor is mostly dictated by the shape and size of the screen; since with VR the screen is moved to the HMD, this would no longer be a restriction. The webcam view is also interesting; if it were "inverted" so that, instead of being confined to a window, it was outside of all windows (that is, the background "wallpaper"), it would work even better as a laptop replacement (that is, the screen doesn't cover your whole view, allowing you to keep aware of your surroundings).
- kanetw 7y ago
  We do actually plan to design standalone portable devices as a longer-term business plan. The AR idea is really interesting. We'll try replacing the background with "reality" as seen from the HMD cameras.
  - netsharc 7y ago
    If could contribute an idea, I wonder if one can make the camera watch your surroundings but with people recognizer that would notify you if someone is approaching with the intent to interact with you, and maybe show a notification or an approaching avatar. It would be neat in an office scenario. The next step would be a screen in the real world telling your visitor whether you're busy or interruptible, and a button the visitor can push to "knock on your door". So that's the future, maybe: cubicle dwellers get their own VR offices...
    - gpm 7y ago
      This seems like the job for another program that you run simultaneously, not the window manager. Unless there are problems with having multiple programs make use of the camera or something.
      - kanetw 7y ago
        Yeah, this should be a separate tool. There's some issues with multiple access I bet that's solvable.
airstrike 7y ago
Have you considered using a mounted Leap Motion controller for interacting with the UI? Would allow you to spin the windows around without having to spin yourself, for prolonged periods of "switching over to the right" for instance. Could use a different gesture to move around the 3D space too allowing you to navigate to different pockets of work
- kroltan 7y ago
  Good idea! You could do that with a spare controller or a Vive tracker too.
ravenstine 7y ago
> It's basically 10x the screens, Holy crap, I never thought of it that way! Anyone know if this is possible with macOS?
- month13 7y ago
  There is [ImmersedVR](https://immersedvr.com/ https://immersedvr.com/), but it's expensive and only compatible with the Oculus Go/Quest. It can create extra displays, but you have to buy an Elite license for that.
ianai 7y ago
Productivity use for VR is way more compelling to me than any game I’ve ever seen. Maybe that’s why it’s never taken off. We probably need to have an email client people want to use in VR and an IDE before we worry about gamers.
- georgewsinger 7y ago
  Totally agreed. Our project's "secret": most people think that the future of VR is in games & entertainment, but it's actually in office work. I never would have even purchased a VR headset had it not been for the promise to start working on a productivity environment. But before we can start building the killer VR office apps, we have to get the basic 2D apps working crisply/perfectly, IMO.
  - gavinray 7y ago
    I came here to say this. Never in my life have I wanted a VR headset. I've used others' to play games (Beatsaber in particular is really fun) but I don't have time in my daily life to allow for a regular gaming habit or indulging that kind of money on it. In 2017, I had a conversation with a co-worker, where we talked about how badly we wanted a VR desktop environment with infinite screen-space. There's VRDesktop, but that isn't this. And then again the same conversation last year with another friend. After seeing the demo video, I was compelled to go look at Vive prices (~$1,000, ouch). I feel like my productivity would skyrocket with this, mostly due to lack of background distractions. Imagine wearing noise-cancelling headphones. I want this so bad.
Robotbeat 7y ago
It’d be awesome if you could do this natively on the Quest (sideloaded). Native gesture control, inside-out tracking, wireless, almost twice the resolution, and wouldn’t even need to be connected to a PC at all so could be used anywhere (with Bluetooth keyboard and mouse). Have you thought about an Android/Quest version?
- kanetw 7y ago
  IIRC the Quest is too different, but our medium to long-term goal is to design a portable computer in HMD format, if that's feasible. I'll take a look at Quest docs and see if it's doable, but I don't have high hopes for it.
  - zmmmmm 7y ago
    Quest would be amazing because it would truly demonstrate a completely untethered, self contained portable full desktop system that you could take anywhere. Having said that, getting even a standard desktop setup on Android to work well is iffy so doing it in VR does seem like a pretty immense challenge. But it would be a breakthrough in what it could demonstrate in terms of the concept.
the_duke 7y ago
I love the idea of a VR workspace. There are so many limitations in our current workflows and tools that mainly just arise from being restricted to a small flat surface. Even just having your perfect setup anywhere you go would be amazing. All you need is a headset and a small portable cube computer - basically a laptop without a screen. --- But after watching your video, I feel like this really needs some tiling window manager (i3 etc) inspired way of easily creating workspaces with tiles, and effortlessly navigating between them. With relatively minimal head movement. Also, does Simula support curved displays? It would seem very natural to arrange different curved screens around you instead of flat ones, which also potentially waste quite a bit of useful space.
- kanetw 7y ago
  Yes, a tiling functionality is something we're considering as a future feature. In fact, the shortcut system is inspired by it. I'll put curved windows onto our todo list. Not sure if they're a good idea for all applications but at least some would benefit.
  - the_duke 7y ago
    After using curved ultrawide monitors like the Philips 499P9H, I think a gentle curve is not an issue for most applications, and feels very natural. I won't get to try out Simula until Sunday, but based on the video, my intuition says the apps being curved would look more natural / immersive.
    - kanetw 7y ago
      I use a curved monitor (Samsung CRG9) myself as my daily driver, and it appears curved when I use desktop view in SteamVR, so I definitely see the appeal. Just not sure how nice it looks on smaller windows.
llarsson 7y ago
I love the little joke that you ran "top" in the window that you had to look toward the ceiling to see. :)
stewbrew 7y ago
Cool demo. But wouldn't it be better (for your neck) to move the screen instead of moving your head? Or both? I guess sooner or later it's going to hurt the way it is implemented now.
xtiansimon 7y ago
Very good demo. Now you need something like a knob or dial to rotate the background, so you don't need to move your chair or turn 360.

outworlder 7y ago

Is it possible to have a giant curved "screen"? I feel we are just scratching the surface of what's possible. Yes, multiple (arbitrarily sized) windows are neat. But we can probably do more. Even without any changes though, I just wish my VR headset was lighter. I can go more than 8h straight in VR... playing games. Not sure about working. Will try this :)

kanetw 7y ago
> Is it possible to have a giant curved "screen"? SteamVR does this, but I've found it less useful than movable/resizable windows. > I feel we are just scratching the surface of what's possible. Definitely. This is one of the first viable iterations, but there's lots and lots of improvement potential everywhere. > Even without any changes though, I just wish my VR headset was lighter. Yeah, agreed. But I think future hardware will improve on that.
zachrose 7y ago
Non-Euclidean UI

skratlo 7y ago

Well looking at the video, I mean, I can have dozens workspaces with hundreds of windows (if I need to) and navigate them with moving just my fingers, whereas you are apparently breaking your neck to navigate more than 3 windows. I think what you're doing is cool and has to be done for the sake of exploration, but I would really, really focus on navigation. Also, how are you supposed to focus on text, when reading or reading or writing when it's wiggling all the time. It seems to me that VR really is for games, but for working with text I think humans are better off with stationary screens. On a second thought: if you would make it easier to "fix" window, such that one can focus on it and work. As when you are "placing" a new window in your video, that could make it a workable solution.

kanetw 7y ago
Navigation is one of the things we definitely need to improve upon. I think the wiggling is actually kind of natural -- you don't notice your head or eye movements when you do them automatically, but it might be exacerbated in VR. The fixing is an interesting idea, we'll try it out. I'm a bit worried it might feel weird in VR.
- todd8 7y ago
  Yes, there is a thing called corollary discharge in the human visual system that keeps the world steady as one glances around (in the real world). Changing the direction one is looking causes the image to shift rapidly on the retina, yet perceptually we see a world that is not wiggling. Corollary discharge[1] causes the brain to “subtract” out the motion of the image on the retina. It happens when our nervous system instructs the eyes to look in another direction. You can have someone else move your eyes or even do it yourself by pressing gently on the side of the eyeball with a fingertip on top of the lid. Wiggling one eye this way is not supported by the hardware of the eye and produces a very noticeable wiggly view of the real world. I mentioned subtracting out the motion going on in the visual system. Experiments suggest that something like this is happening. If a muscle paralyzing drug is injected into the eye’s muscles it causes a wiggly world because the intent to glance somewhere causes both an ineffective command to move the eye, producing no real movement of the image in the retina while the triggered corollary discharge subtracts the intended motion from the still image and causes a perceived motion (in the opposite direction). [1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2807735/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2807735/
- ggreer 7y ago
  YouTube has support for VR videos. If you could record & render such that people with headsets could watch a session, that might help with interest and adoption. I think navigation needs eye tracking, which sadly no headset currently supports. Focus-follows-gaze would be a game changer.
  - georgewsinger 7y ago

lukifer 7y ago

How is the eye fatigue in general? How long can you go in a VR work session? For VR gaming (Oculus Rift), I start to experience mild discomfort after about 30min, which grows severe by 60min. It's hard to envision using VR for a full work day. (My eyes are fairly light-sensitive from laser-eye surgery; not sure how much is saccading, vs. bright light directly in front of the eyes.)

georgewsinger 7y ago
RE eye strain, here are a couple of links showing off Simula's text quality: https://github.com/SimulaVR/Simula/blob/gdwlroots-xwayland/doc/TextQuality.gif https://github.com/SimulaVR/Simula/blob/gdwlroots-xwayland/d... https://github.com/SimulaVR/Simula/blob/gdwlroots-xwayland/doc/TextQuality2.gif https://github.com/SimulaVR/Simula/blob/gdwlroots-xwayland/d... Many early VR Desktops (and VR games) didn't go to special effort to optimize text quality, and I think this has given people a lower than warranted impression of what is possible in VR today. There's no doubt that VR hardware is going to get exponentially lighter/sharper over time, and that these improvements will be welcome, but things are good enough now to put in 1hr+ long sessions without eye strain (at least in my experience as a daily user).
mhh__ 7y ago
It really depends on the task - I (currently) have an OG Vive, and its absolutely fine for games with good anti-aliasing (no discomfort for me for basically any length of time, although I mainly play beatsaber and you don't really focus on the blocks beyond blinks of light and muscle memory), but it's hopeless if you want to watch a movie let alone program. The kind of programming I tend to do is generally fiddly enough that I've got a book open or some kind of datasheet etc. (I think - even good - VR would be a hindrance here) I think the Index might be OK but I couldn't comment.
driverdan 7y ago
I can't speak to working with text but I could easily play games all day on a Vive with no eye strain.

emj 7y ago

I would not be able to work like that, it would lead to neck strain or wrist pain, if you think about it comes naturally to reposition your body when you are looking in a new direction there is a reason for this. This hurts me because the keyboard locks the shoulders and it's only my heavy head that moves, this is not ergonomical. So being able to fetch up windows so it's perfectly centered above the keyboard would make it easier for me to use.

fractal618 7y ago
Maybe there should be a harness to suspend us by one ankle while we work

Lambdanaut 7y ago

I saw that you were hacking on... Haskell..? In that video? Where does Haskell fit in with Godot and Simula?

kanetw 7y ago
SimulaVR (unrelated to Simula) is written mostly in Haskell, with Godot as the engine.

cesarb 7y ago

An interesting potential for VR would be alternative portable form factors. The laptop form factor is mostly dictated by the shape and size of the screen; since with VR the screen is moved to the HMD, this would no longer be a restriction. The webcam view is also interesting; if it were "inverted" so that, instead of being confined to a window, it was outside of all windows (that is, the background "wallpaper"), it would work even better as a laptop replacement (that is, the screen doesn't cover your whole view, allowing you to keep aware of your surroundings).

kanetw 7y ago
We do actually plan to design standalone portable devices as a longer-term business plan. The AR idea is really interesting. We'll try replacing the background with "reality" as seen from the HMD cameras.
- netsharc 7y ago
  If could contribute an idea, I wonder if one can make the camera watch your surroundings but with people recognizer that would notify you if someone is approaching with the intent to interact with you, and maybe show a notification or an approaching avatar. It would be neat in an office scenario. The next step would be a screen in the real world telling your visitor whether you're busy or interruptible, and a button the visitor can push to "knock on your door". So that's the future, maybe: cubicle dwellers get their own VR offices...
  - gpm 7y ago
    This seems like the job for another program that you run simultaneously, not the window manager. Unless there are problems with having multiple programs make use of the camera or something.
    - kanetw 7y ago
      Yeah, this should be a separate tool. There's some issues with multiple access I bet that's solvable.

airstrike 7y ago

Have you considered using a mounted Leap Motion controller for interacting with the UI? Would allow you to spin the windows around without having to spin yourself, for prolonged periods of "switching over to the right" for instance. Could use a different gesture to move around the 3D space too allowing you to navigate to different pockets of work

kroltan 7y ago
Good idea! You could do that with a spare controller or a Vive tracker too.

ravenstine 7y ago

> It's basically 10x the screens, Holy crap, I never thought of it that way! Anyone know if this is possible with macOS?

month13 7y ago
There is [ImmersedVR](https://immersedvr.com/ https://immersedvr.com/), but it's expensive and only compatible with the Oculus Go/Quest. It can create extra displays, but you have to buy an Elite license for that.

ianai 7y ago

Productivity use for VR is way more compelling to me than any game I’ve ever seen. Maybe that’s why it’s never taken off. We probably need to have an email client people want to use in VR and an IDE before we worry about gamers.

georgewsinger 7y ago
Totally agreed. Our project's "secret": most people think that the future of VR is in games & entertainment, but it's actually in office work. I never would have even purchased a VR headset had it not been for the promise to start working on a productivity environment. But before we can start building the killer VR office apps, we have to get the basic 2D apps working crisply/perfectly, IMO.
- gavinray 7y ago
  I came here to say this. Never in my life have I wanted a VR headset. I've used others' to play games (Beatsaber in particular is really fun) but I don't have time in my daily life to allow for a regular gaming habit or indulging that kind of money on it. In 2017, I had a conversation with a co-worker, where we talked about how badly we wanted a VR desktop environment with infinite screen-space. There's VRDesktop, but that isn't this. And then again the same conversation last year with another friend. After seeing the demo video, I was compelled to go look at Vive prices (~$1,000, ouch). I feel like my productivity would skyrocket with this, mostly due to lack of background distractions. Imagine wearing noise-cancelling headphones. I want this so bad.

Robotbeat 7y ago

It’d be awesome if you could do this natively on the Quest (sideloaded). Native gesture control, inside-out tracking, wireless, almost twice the resolution, and wouldn’t even need to be connected to a PC at all so could be used anywhere (with Bluetooth keyboard and mouse). Have you thought about an Android/Quest version?

kanetw 7y ago
IIRC the Quest is too different, but our medium to long-term goal is to design a portable computer in HMD format, if that's feasible. I'll take a look at Quest docs and see if it's doable, but I don't have high hopes for it.
- zmmmmm 7y ago
  Quest would be amazing because it would truly demonstrate a completely untethered, self contained portable full desktop system that you could take anywhere. Having said that, getting even a standard desktop setup on Android to work well is iffy so doing it in VR does seem like a pretty immense challenge. But it would be a breakthrough in what it could demonstrate in terms of the concept.

the_duke 7y ago

I love the idea of a VR workspace. There are so many limitations in our current workflows and tools that mainly just arise from being restricted to a small flat surface. Even just having your perfect setup anywhere you go would be amazing. All you need is a headset and a small portable cube computer - basically a laptop without a screen. --- But after watching your video, I feel like this really needs some tiling window manager (i3 etc) inspired way of easily creating workspaces with tiles, and effortlessly navigating between them. With relatively minimal head movement. Also, does Simula support curved displays? It would seem very natural to arrange different curved screens around you instead of flat ones, which also potentially waste quite a bit of useful space.

kanetw 7y ago
Yes, a tiling functionality is something we're considering as a future feature. In fact, the shortcut system is inspired by it. I'll put curved windows onto our todo list. Not sure if they're a good idea for all applications but at least some would benefit.
- the_duke 7y ago
  After using curved ultrawide monitors like the Philips 499P9H, I think a gentle curve is not an issue for most applications, and feels very natural. I won't get to try out Simula until Sunday, but based on the video, my intuition says the apps being curved would look more natural / immersive.
  - kanetw 7y ago
    I use a curved monitor (Samsung CRG9) myself as my daily driver, and it appears curved when I use desktop view in SteamVR, so I definitely see the appeal. Just not sure how nice it looks on smaller windows.

llarsson 7y ago

I love the little joke that you ran "top" in the window that you had to look toward the ceiling to see. :)

stewbrew 7y ago

Cool demo. But wouldn't it be better (for your neck) to move the screen instead of moving your head? Or both? I guess sooner or later it's going to hurt the way it is implemented now.

xtiansimon 7y ago

Very good demo. Now you need something like a knob or dial to rotate the background, so you don't need to move your chair or turn 360.