7 ms·

Resolution is the main concern. Is oculus detailed enough to do programming on a virtual monitor?

lifeisstillgood 11y ago
Probably - if you are like me and have terrible eyes and program in emacs at 18+ pt and 80 chars across. The vrdesktop demo is impressive and gives you an idea of what is likely possible. I think if you want it enough you will put up with it
pavlov 11y ago
Not even close. The screens for each eye probably have about 1440 pixels of vertical resolution (I'm guessing they are Samsung OLED screens, similar to the display on the Galaxy S6). Imagine that resolution stretched out to fill your field of view and distorted by the lenses -- the perceptual resolution available for a virtual monitor would probably be less than VGA. VR environments that can do high-quality typography will require something like 10k pixels vertical resolution. It's going to be more than a decade until we have the screens and GPUs that can push those pixels. (I have a Samsung Gear VR with the Galaxy Note 4 which is a 2560*1440 screen split for two eyes, so I have a pretty good idea of what Oculus VR looks like on a 1440px screen.)
- ethbro 11y ago
  > the perceptual resolution available for a virtual monitor would probably be less than VGA It's a shame we weren't able to program on VGA monitors. :(
  - brianorwhatever 11y ago
    programming wasn't really possible until HD came around
  - pavlov 11y ago
    VGA had 480 scanlines. When was the last time anyone seriously programmed on a display worse than that? My guess would be Commodore 64 with an NTSC TV for display. That's pretty close to the text fidelity you can reproduce on a "virtual monitor" in a current-generation VR environment. (I think a lot of people underestimate the amount of resolution lost due to lens correction -- the rendered screen image is heavily distorted to account for the wide-angle lenses.) Well, I suppose it would be sort of fun to have a virtual C64, as long as the virtual surroundings are well executed: a complete high school kid's bedroom from 1986.
    - derekp7 11y ago
      I recall being my most productive on an old IBM 3151 terminal (probably because of the keyboard, but also fewer distractions). It was only 80x24 characters. And 80x24 is easily represented in a 640x480 screen. Now add in head tracking, and you can have the screen move up and down and scroll the text in the opposite direction, making it appear as if you are moving a virtual picture frame over a bigger sheet of paper. Even today, most of my (C / Bash) programming is done in an 80x24 xterm, although I'll typically have documentation open in another window. However web and GUI app development requires enough room to see the resulting product.
    - erikpukinskis 11y ago
      For one of my projects, I am writing code Thet is meant to be read by the 99%, i.e. formatted to be readable on a phone. 40 chars wide, and I rarely look at more than 10 lines at a time. It's generally fine. Every now and then I feel like I want to see the broader context of the code, but it's rare. In general it forces me to write much cleaner, more readable code, and to structure things better so that concerns are truly isolated and I don't need to look at a lot of code to see how things work. Overall I think it's a positive experience.
    - AHHspiders 11y ago
      So long as it has that sweet sweet lamborghini countach poster on the wall, i'm sold.
    - mikekchar 11y ago
      Although I have much higher resolution, lately some occular migranes have forced me to up the font size on my monitor. Having really big type seems to stave off the migranes for some reason. Anyway, I think I've been going for about a month and a half with 80x27 text displays (full screen with tmux). I'm doing web development and my browser is similarly scaled (thank god we're building something with a scalable UI). I have not missed the screen real estate at all. Of course, I'm old and I used to always work on 80x25, so the extra 2 lines are luxury ;-)
  - Avshalom 11y ago
    other than "can't fall off a seat tray" there isn't much point in having 4 virtual monitors if it gives you less readable real-estate than a real monitor.
    - ethbro 11y ago
      Really? Even if the visible, readable real-estate is limited, I think there's be a huge usability boost from using... "virtual monitors" is a terribly ambiguous phrase... virtually co-located monitors? The key metric for me is how much information I can access with input under a minimal threshold. From personal experience, a hotkey to swap virtual desktops (e.g. Alt+Up) still isn't the same as having multiple physical monitors to reference. However, I'd expect VR head orientation changes to look at different monitors to be fairly similar perceptually to what I do now, since it's the same physical action. The bigger problem for the seat-tray problem is, afaik, both Oculus and the Vive use externally located tracking devices. Would be curious whether a fuzzier, internal-sensor-only, limited "intent" tracking mode (e.g. flick head to switch monitor) would make people hurl or not.
      - Tuna-Fish 11y ago
        > Would be curious whether a fuzzier, internal-sensor-only, limited "intent" tracking mode (e.g. flick head to switch monitor) would make people hurl or not. Yes, it would. This has been studied pretty extensively, the head movements need to be very precisely matched by rendering. The absolute worst you can do is any kind of non-linear response -- acceleration + lag can make people who are very tolerant of VR nausea literally throw up.
        moron4hire 11y ago
        Oh, it's not that bad. Jesus, people make more out of this stuff than is at all necessary. How did people ever adopt cars or boats, talking like that? And use it on any sort of system that can show the outside world (AR, or pass through camera in VR) and it goes away completely. Gesture interfaces could work.
  - 72deluxe 11y ago
    I used to program on a BBC Micro and that was low resolution.
- 0xdeadbeefbabe 11y ago
  Well you could change the interface to not need so much resolution. Maybe you could use a giant card and punch out holes in it?
- MaulingMonkey 11y ago
  I've seen the idea of multi-resolution rendering floating around, which I think has some promise when combined with eye tracking, would help with the GPU side of things. Not so much on the "raw display hardware" side of things though. I could still see it being a decade+ out on that alone.
  - Otik 11y ago
    Yep, foveated rendering. http://research.microsoft.com/en-us/projects/foveateddisplay/ http://research.microsoft.com/en-us/projects/foveateddisplay... Cool stuff, but it needs extremely fast and low latency eye tracking to work effectively. It's not insurmountable, but a significant engineering problem.
- dharma1 11y ago
  More than a decade sounds like a long time. Oculus has been riding the smartphone screen density wave but I'm not sure that will continue much longer since for phone use going beyond 500ppi doesn't make much difference. Maybe some kind of projected light field will be the next solution? GPUs will keep getting faster and smaller. Have you seen NVIDIA X1?
  - MattRix 11y ago
    "Maybe some kind of projected light field will be the next solution?" already being done! http://en.m.wikipedia.org/wiki/Magic_Leap http://en.m.wikipedia.org/wiki/Magic_Leap
    - Otik 11y ago
      Magic Leap is more of an AR solution, like HoloLens, as opposed to VR (as far as anyone can tell, seeing as they haven't demoed anything publicly). Nvidia's light-field stuff seems to be more of a VR solution, but requires even higher pixel densities. https://research.nvidia.com/publication/near-eye-light-field-displays https://research.nvidia.com/publication/near-eye-light-field...
- moron4hire 11y ago
  >> not even close Huh. What the fuck have I been doing for the last 6 months, then?
  - jboggan 11y ago
    Please elaborate! I am curious what your experience and setup is. I have zero interest in consuming media or games through VR but I am really wondering how it can be productive. Do you get more work done compared to regular monitors, or is it the same work while feeling 199% cooler?
    - corysama 11y ago
      Check the url in his profile.
  - pavlov 11y ago
    I'm not criticising your project at all -- just saying that my experience with current VR display tech is such that I can't imagine reading text for more than 30 seconds. The aliasing and distortion would make my eyes bleed pretty quickly. The time will come for textual VR environments though! I can't wait to have "newspaper resolution" for textures (hold up a virtual newspaper in a VR world and you can actually read the text like it were a printed page). But I suspect it's not going to happen before the mid-2020s at least.
    - moron4hire 11y ago
      It's nowhere near as bad as you've described. Even on the 960x1080 DK2, the readability of text has more to do with texture quality (both total resolution and good filtering for mipmaps). On a 2d display, the pixels are static on the display and text is mapped essentially 1-to-1 (or a fixed n-to-m when using subpixel antialiasing). But on an HMD, antialiasing is basically 'free', and the constant, minuscule bob of the head creates a dynamic, temporal aliasing that improves the readability of the text. We've known about the effect since the 80s or 90s, when psychologists figured out the dynamic environment of real flight improved the apparent eyesight of pilots over static images. There are a lot of people who are doing text wrong in VR. You don't render at 10pt and expect it to look right. You have to pay attention and not just take the default settings. I've spent much time reading and writing text in VR and it hasn't been a problem. If building a real, live code editing environment were my goal, I could have it done in a week. But I have different goals.
paulmd 11y ago
Resolution of the final hardware will be 1080x1200 per eye (2160x1200 total) over a 90* field of view. I'm not quite sure how to translate that into pixel density properly given the resolution is spread between your eyes, but I'd guess at least as good as a 1080p screen that fills a 90* angle of view? The user experience would be quite different, as you could simply rotate+tilt the viewport to see sections of a larger virtual monitor screen. So it wouldn't necessarily be as restricting as low resolution on a fixed screen.
- thethimble 11y ago
  1080p that fills a 90* angle of view would be like looking at a 40" 1080p TV from ~28 inches away. Certainly not the same experience that you're used to with a retina display, unfortunately.
  - sjs382 11y ago
    That's if you're looking at the full screen. Imagine walking closer, and having the resolution "increase" as you get closer (because you have the same number of pixels, looking at less "stuff" per pixel).
    - socceroos 11y ago
      I'm not sure people would see the benefit of using it if you had to peer so close that each character took up a third of your displays just so it was at a comfortable resolution.
spudlyo 11y ago
A readable 80x24 text mode is the bare minimum for programming IMHO. It's probably detailed enough for that, but it's not going rival your laptop/desktop experience any time soon.
corysama 11y ago
A few people are experimenting with that over at http://www.reddit.com/r/HMDprogramming/ http://www.reddit.com/r/HMDprogramming/ Right now, the resolution is clearly not high enough for a straightforward implementation. I've got some ideas I'd like to play with around green-on-black text rendered directly in the post-lens-warp buffer, but I haven't had time to try it out yet.
- kibibu 11y ago
  Didn't SDK 0.6.0 remove some of the ability to do that stuff? > Removed support for application-based distortion rendering. Removed functions include ovrHmd_CreateDistortionMesh, ovrHmd_GetRenderScaleAndOffset, and so on. If you feel that you require application-based distortion rendering, please contact Oculus Developer Relations. https://developer.oculus.com/history/ https://developer.oculus.com/history/
amalcon 11y ago
I'd think the best approach for something like this would be to have higher pixel density in the middle of the display -- that is, where your head is looking -- and lower pixel density in the perhipheral. That's tough to do with an LCD, but it could be done with a specially designed projector or two different LCDs that you combine optically.
jfoster 11y ago
It doesn't (yet) seem particularly comfortable, but this was an impressive demo of someone coding their environment in the Rift. https://www.youtube.com/watch?v=db-7J5OaSag https://www.youtube.com/watch?v=db-7J5OaSag

Probably - if you are like me and have terrible eyes and program in emacs at 18+ pt and 80 chars across. The vrdesktop demo is impressive and gives you an idea of what is likely possible. I think if you want it enough you will put up with it

pavlov 11y ago

Not even close. The screens for each eye probably have about 1440 pixels of vertical resolution (I'm guessing they are Samsung OLED screens, similar to the display on the Galaxy S6). Imagine that resolution stretched out to fill your field of view and distorted by the lenses -- the perceptual resolution available for a virtual monitor would probably be less than VGA. VR environments that can do high-quality typography will require something like 10k pixels vertical resolution. It's going to be more than a decade until we have the screens and GPUs that can push those pixels. (I have a Samsung Gear VR with the Galaxy Note 4 which is a 2560*1440 screen split for two eyes, so I have a pretty good idea of what Oculus VR looks like on a 1440px screen.)

ethbro 11y ago
> the perceptual resolution available for a virtual monitor would probably be less than VGA It's a shame we weren't able to program on VGA monitors. :(
- brianorwhatever 11y ago
  programming wasn't really possible until HD came around
- pavlov 11y ago
  VGA had 480 scanlines. When was the last time anyone seriously programmed on a display worse than that? My guess would be Commodore 64 with an NTSC TV for display. That's pretty close to the text fidelity you can reproduce on a "virtual monitor" in a current-generation VR environment. (I think a lot of people underestimate the amount of resolution lost due to lens correction -- the rendered screen image is heavily distorted to account for the wide-angle lenses.) Well, I suppose it would be sort of fun to have a virtual C64, as long as the virtual surroundings are well executed: a complete high school kid's bedroom from 1986.
  - derekp7 11y ago
    I recall being my most productive on an old IBM 3151 terminal (probably because of the keyboard, but also fewer distractions). It was only 80x24 characters. And 80x24 is easily represented in a 640x480 screen. Now add in head tracking, and you can have the screen move up and down and scroll the text in the opposite direction, making it appear as if you are moving a virtual picture frame over a bigger sheet of paper. Even today, most of my (C / Bash) programming is done in an 80x24 xterm, although I'll typically have documentation open in another window. However web and GUI app development requires enough room to see the resulting product.
  - erikpukinskis 11y ago
    For one of my projects, I am writing code Thet is meant to be read by the 99%, i.e. formatted to be readable on a phone. 40 chars wide, and I rarely look at more than 10 lines at a time. It's generally fine. Every now and then I feel like I want to see the broader context of the code, but it's rare. In general it forces me to write much cleaner, more readable code, and to structure things better so that concerns are truly isolated and I don't need to look at a lot of code to see how things work. Overall I think it's a positive experience.
  - AHHspiders 11y ago
    So long as it has that sweet sweet lamborghini countach poster on the wall, i'm sold.
  - mikekchar 11y ago
    Although I have much higher resolution, lately some occular migranes have forced me to up the font size on my monitor. Having really big type seems to stave off the migranes for some reason. Anyway, I think I've been going for about a month and a half with 80x27 text displays (full screen with tmux). I'm doing web development and my browser is similarly scaled (thank god we're building something with a scalable UI). I have not missed the screen real estate at all. Of course, I'm old and I used to always work on 80x25, so the extra 2 lines are luxury ;-)
- Avshalom 11y ago
  other than "can't fall off a seat tray" there isn't much point in having 4 virtual monitors if it gives you less readable real-estate than a real monitor.
  - ethbro 11y ago
    Really? Even if the visible, readable real-estate is limited, I think there's be a huge usability boost from using... "virtual monitors" is a terribly ambiguous phrase... virtually co-located monitors? The key metric for me is how much information I can access with input under a minimal threshold. From personal experience, a hotkey to swap virtual desktops (e.g. Alt+Up) still isn't the same as having multiple physical monitors to reference. However, I'd expect VR head orientation changes to look at different monitors to be fairly similar perceptually to what I do now, since it's the same physical action. The bigger problem for the seat-tray problem is, afaik, both Oculus and the Vive use externally located tracking devices. Would be curious whether a fuzzier, internal-sensor-only, limited "intent" tracking mode (e.g. flick head to switch monitor) would make people hurl or not.
    - Tuna-Fish 11y ago
      > Would be curious whether a fuzzier, internal-sensor-only, limited "intent" tracking mode (e.g. flick head to switch monitor) would make people hurl or not. Yes, it would. This has been studied pretty extensively, the head movements need to be very precisely matched by rendering. The absolute worst you can do is any kind of non-linear response -- acceleration + lag can make people who are very tolerant of VR nausea literally throw up.
      - moron4hire 11y ago
        Oh, it's not that bad. Jesus, people make more out of this stuff than is at all necessary. How did people ever adopt cars or boats, talking like that? And use it on any sort of system that can show the outside world (AR, or pass through camera in VR) and it goes away completely. Gesture interfaces could work.
- 72deluxe 11y ago
  I used to program on a BBC Micro and that was low resolution.
0xdeadbeefbabe 11y ago
Well you could change the interface to not need so much resolution. Maybe you could use a giant card and punch out holes in it?
MaulingMonkey 11y ago
I've seen the idea of multi-resolution rendering floating around, which I think has some promise when combined with eye tracking, would help with the GPU side of things. Not so much on the "raw display hardware" side of things though. I could still see it being a decade+ out on that alone.
- Otik 11y ago
  Yep, foveated rendering. http://research.microsoft.com/en-us/projects/foveateddisplay/ http://research.microsoft.com/en-us/projects/foveateddisplay... Cool stuff, but it needs extremely fast and low latency eye tracking to work effectively. It's not insurmountable, but a significant engineering problem.
dharma1 11y ago
More than a decade sounds like a long time. Oculus has been riding the smartphone screen density wave but I'm not sure that will continue much longer since for phone use going beyond 500ppi doesn't make much difference. Maybe some kind of projected light field will be the next solution? GPUs will keep getting faster and smaller. Have you seen NVIDIA X1?
- MattRix 11y ago
  "Maybe some kind of projected light field will be the next solution?" already being done! http://en.m.wikipedia.org/wiki/Magic_Leap http://en.m.wikipedia.org/wiki/Magic_Leap
  - Otik 11y ago
    Magic Leap is more of an AR solution, like HoloLens, as opposed to VR (as far as anyone can tell, seeing as they haven't demoed anything publicly). Nvidia's light-field stuff seems to be more of a VR solution, but requires even higher pixel densities. https://research.nvidia.com/publication/near-eye-light-field-displays https://research.nvidia.com/publication/near-eye-light-field...
moron4hire 11y ago
>> not even close Huh. What the fuck have I been doing for the last 6 months, then?
- jboggan 11y ago
  Please elaborate! I am curious what your experience and setup is. I have zero interest in consuming media or games through VR but I am really wondering how it can be productive. Do you get more work done compared to regular monitors, or is it the same work while feeling 199% cooler?
  - corysama 11y ago
    Check the url in his profile.
- pavlov 11y ago
  I'm not criticising your project at all -- just saying that my experience with current VR display tech is such that I can't imagine reading text for more than 30 seconds. The aliasing and distortion would make my eyes bleed pretty quickly. The time will come for textual VR environments though! I can't wait to have "newspaper resolution" for textures (hold up a virtual newspaper in a VR world and you can actually read the text like it were a printed page). But I suspect it's not going to happen before the mid-2020s at least.
  - moron4hire 11y ago
    It's nowhere near as bad as you've described. Even on the 960x1080 DK2, the readability of text has more to do with texture quality (both total resolution and good filtering for mipmaps). On a 2d display, the pixels are static on the display and text is mapped essentially 1-to-1 (or a fixed n-to-m when using subpixel antialiasing). But on an HMD, antialiasing is basically 'free', and the constant, minuscule bob of the head creates a dynamic, temporal aliasing that improves the readability of the text. We've known about the effect since the 80s or 90s, when psychologists figured out the dynamic environment of real flight improved the apparent eyesight of pilots over static images. There are a lot of people who are doing text wrong in VR. You don't render at 10pt and expect it to look right. You have to pay attention and not just take the default settings. I've spent much time reading and writing text in VR and it hasn't been a problem. If building a real, live code editing environment were my goal, I could have it done in a week. But I have different goals.

paulmd 11y ago

Resolution of the final hardware will be 1080x1200 per eye (2160x1200 total) over a 90* field of view. I'm not quite sure how to translate that into pixel density properly given the resolution is spread between your eyes, but I'd guess at least as good as a 1080p screen that fills a 90* angle of view? The user experience would be quite different, as you could simply rotate+tilt the viewport to see sections of a larger virtual monitor screen. So it wouldn't necessarily be as restricting as low resolution on a fixed screen.

thethimble 11y ago
1080p that fills a 90* angle of view would be like looking at a 40" 1080p TV from ~28 inches away. Certainly not the same experience that you're used to with a retina display, unfortunately.
- sjs382 11y ago
  That's if you're looking at the full screen. Imagine walking closer, and having the resolution "increase" as you get closer (because you have the same number of pixels, looking at less "stuff" per pixel).
  - socceroos 11y ago
    I'm not sure people would see the benefit of using it if you had to peer so close that each character took up a third of your displays just so it was at a comfortable resolution.

spudlyo 11y ago

A readable 80x24 text mode is the bare minimum for programming IMHO. It's probably detailed enough for that, but it's not going rival your laptop/desktop experience any time soon.

corysama 11y ago

A few people are experimenting with that over at http://www.reddit.com/r/HMDprogramming/ http://www.reddit.com/r/HMDprogramming/ Right now, the resolution is clearly not high enough for a straightforward implementation. I've got some ideas I'd like to play with around green-on-black text rendered directly in the post-lens-warp buffer, but I haven't had time to try it out yet.

kibibu 11y ago
Didn't SDK 0.6.0 remove some of the ability to do that stuff? > Removed support for application-based distortion rendering. Removed functions include ovrHmd_CreateDistortionMesh, ovrHmd_GetRenderScaleAndOffset, and so on. If you feel that you require application-based distortion rendering, please contact Oculus Developer Relations. https://developer.oculus.com/history/ https://developer.oculus.com/history/

amalcon 11y ago

I'd think the best approach for something like this would be to have higher pixel density in the middle of the display -- that is, where your head is looking -- and lower pixel density in the perhipheral. That's tough to do with an LCD, but it could be done with a specially designed projector or two different LCDs that you combine optically.

jfoster 11y ago

It doesn't (yet) seem particularly comfortable, but this was an impressive demo of someone coding their environment in the Rift. https://www.youtube.com/watch?v=db-7J5OaSag https://www.youtube.com/watch?v=db-7J5OaSag