3 ms·

Yes. If anyone knows of any existing code to support say 3D cross/parallel-eye stereo views, with z-axis displaced words, in an emacs buffer being actively edi

Yes. If anyone knows of any existing code to support say 3D cross/parallel-eye stereo views, with z-axis displaced words, in an emacs buffer being actively edited, I'd like to hear of it. Teaching your window manager to be 3D is fun, but the next step is to teach your apps.

gmueckl 7y ago
You are welcome to try, but I can tell you right now that a UI that places words in a text at different depths is a terrible idea. Depths is simulated by varying parallax and vergence. And forcing a permanent vergence adjustment is a strain on the eye muscles. Plus you get to enjoy the vergence-focus mismatch from the fixed headset focal distance to the fullest. This is a headache-inducing cue.
- mncharity 7y ago
  Which is why you keep your primary text plane at focus depth, and emphasize shallow 3D displacement. You can do a lot with a mere cm of depth around your laptop screen, or a similarly scaled annulus at an HMD's 2 meter focus distance.
  - gmueckl 7y ago
    This results in an interesting question: I don't know if the available headset resolutions are sufficient to provise these quite subtle vergence cues.
    - mncharity 7y ago
      Hmm. For VR HMDs, with RGB subpixels (non-PenTile), you can read a pixel-aligned 5 pt character in the center, but you don't get many of them (because of lens blur, and just VR lensing), and the pixels are a bit large. I've no idea how well SteamVR/Unreal/Unity currently let you use them. The lenses of Valve's Index were praised elsewhere here. AR HMDs trade narrower FoV for angularly smaller pixels, and you can use "all" of them. But the pixels may be poorly aligned between eyes, as with Project North Star. Maybe Hololens 2 (with different a-v issues). I can't comment on Magic Leap. I look forward to trying next month's/year's Nreal glasses. Media-viewing and drone HMDs offer a screen at some distance. Some are stereo 3D. Some are 1080p. Few have head tracking. You can add it with Intel t265, Zed, Structure Core (no linux), or DIY. Many have a "diopter" adjustment. They've varied build quality, and supported nose sizes. Some months back, I failed to find a 3D 1080p HMD, and so currently use mono 1080p DJI drone goggles (discontinued cheaper white model), with anaglyph (crushing the color space to reduce eye strain) or DIY shutter (flickery 30 or 20 Hz) to recover 3D. :/ Usually with reading glasses, despite my old eyes being nearsighted, which costs some edge pixels. DIY optical tracking, so no temporal antialiasing. I'd not recommend it. My current hope is for a Q4, 1080p, 50-ish degree FoV, 3D "screen" HMD (media-viewer or AR). With either head tracking built in, or COTS. It may need a Windows box in support, especially for hand tracking. I'll likely stay with a simple custom browser-as-compositor stack, for low-cost pixel-precise control, but a mainstream VR or AR stack might work too.
dreamcompiler 7y ago
Interesting idea. What would you use this for?
- mncharity 7y ago
  When one does software development, one might use a variety of media. Technical pen on paper for design-space and architecture graphs. Whiteboard. Whiteboard with colleague. Notes. Exploratory code. And so on. So one question to ask is, here comes consumer VR and AR, and what might these new media offer to justify their place in this tool collection? That takes exploration. Another question to ask is, given hardware with 3D tracking and display, how might you usefully tweak your existing tools to leverage that? Rather than reinventing wheels, languages, keyboards, etc, I like the approach of augmenting what exists. And I use emacs a lot. TensorFlow posenet-based head tracking is easy.[1] And wiggle 3D, and anaglyph shaders, aren't hard. And making DIY shutter glasses (arduino, phototransistor, and Adafruit light valves) was surprisingly simple. So my generic thinkpad screen can do jittery, washed-out, or flickery 3D. And electron/node.js/v8 can serve up low-latency video of the desktop, to be sliced and diced. So an app can provide two or three eye views, in separate windows or time multiplexed, and it can be shown as 3D. When last I tried to do that separation in emacs, I timed out, still unsure if there was a viable path. Thus my query. Another thing one might try is to overlay graphics. An emacs could have an arbitrary backchannel chat with the compositor, and draw 3D lines and such on itself. And of course control its own window positions and orientations, fragmentation and coalesce. ;) If you have red-cyan anaglyph 3D glasses, you might enjoy toying with https://atom.io/themes/anaglyph-syntax https://atom.io/themes/anaglyph-syntax - but trading color space for depth is a steep price. > What would you use this for? So basically for exploration. What might our tooling now look like, if we'd had 3D display and tracking in widespread use for the last two decades? [1] https://storage.googleapis.com/tfjs-models/demos/posenet/camera.html https://storage.googleapis.com/tfjs-models/demos/posenet/cam...

gmueckl 7y ago

You are welcome to try, but I can tell you right now that a UI that places words in a text at different depths is a terrible idea. Depths is simulated by varying parallax and vergence. And forcing a permanent vergence adjustment is a strain on the eye muscles. Plus you get to enjoy the vergence-focus mismatch from the fixed headset focal distance to the fullest. This is a headache-inducing cue.

mncharity 7y ago
Which is why you keep your primary text plane at focus depth, and emphasize shallow 3D displacement. You can do a lot with a mere cm of depth around your laptop screen, or a similarly scaled annulus at an HMD's 2 meter focus distance.
- gmueckl 7y ago
  This results in an interesting question: I don't know if the available headset resolutions are sufficient to provise these quite subtle vergence cues.
  - mncharity 7y ago
    Hmm. For VR HMDs, with RGB subpixels (non-PenTile), you can read a pixel-aligned 5 pt character in the center, but you don't get many of them (because of lens blur, and just VR lensing), and the pixels are a bit large. I've no idea how well SteamVR/Unreal/Unity currently let you use them. The lenses of Valve's Index were praised elsewhere here. AR HMDs trade narrower FoV for angularly smaller pixels, and you can use "all" of them. But the pixels may be poorly aligned between eyes, as with Project North Star. Maybe Hololens 2 (with different a-v issues). I can't comment on Magic Leap. I look forward to trying next month's/year's Nreal glasses. Media-viewing and drone HMDs offer a screen at some distance. Some are stereo 3D. Some are 1080p. Few have head tracking. You can add it with Intel t265, Zed, Structure Core (no linux), or DIY. Many have a "diopter" adjustment. They've varied build quality, and supported nose sizes. Some months back, I failed to find a 3D 1080p HMD, and so currently use mono 1080p DJI drone goggles (discontinued cheaper white model), with anaglyph (crushing the color space to reduce eye strain) or DIY shutter (flickery 30 or 20 Hz) to recover 3D. :/ Usually with reading glasses, despite my old eyes being nearsighted, which costs some edge pixels. DIY optical tracking, so no temporal antialiasing. I'd not recommend it. My current hope is for a Q4, 1080p, 50-ish degree FoV, 3D "screen" HMD (media-viewer or AR). With either head tracking built in, or COTS. It may need a Windows box in support, especially for hand tracking. I'll likely stay with a simple custom browser-as-compositor stack, for low-cost pixel-precise control, but a mainstream VR or AR stack might work too.

dreamcompiler 7y ago

Interesting idea. What would you use this for?

mncharity 7y ago
When one does software development, one might use a variety of media. Technical pen on paper for design-space and architecture graphs. Whiteboard. Whiteboard with colleague. Notes. Exploratory code. And so on. So one question to ask is, here comes consumer VR and AR, and what might these new media offer to justify their place in this tool collection? That takes exploration. Another question to ask is, given hardware with 3D tracking and display, how might you usefully tweak your existing tools to leverage that? Rather than reinventing wheels, languages, keyboards, etc, I like the approach of augmenting what exists. And I use emacs a lot. TensorFlow posenet-based head tracking is easy.[1] And wiggle 3D, and anaglyph shaders, aren't hard. And making DIY shutter glasses (arduino, phototransistor, and Adafruit light valves) was surprisingly simple. So my generic thinkpad screen can do jittery, washed-out, or flickery 3D. And electron/node.js/v8 can serve up low-latency video of the desktop, to be sliced and diced. So an app can provide two or three eye views, in separate windows or time multiplexed, and it can be shown as 3D. When last I tried to do that separation in emacs, I timed out, still unsure if there was a viable path. Thus my query. Another thing one might try is to overlay graphics. An emacs could have an arbitrary backchannel chat with the compositor, and draw 3D lines and such on itself. And of course control its own window positions and orientations, fragmentation and coalesce. ;) If you have red-cyan anaglyph 3D glasses, you might enjoy toying with https://atom.io/themes/anaglyph-syntax https://atom.io/themes/anaglyph-syntax - but trading color space for depth is a steep price. > What would you use this for? So basically for exploration. What might our tooling now look like, if we'd had 3D display and tracking in widespread use for the last two decades? [1] https://storage.googleapis.com/tfjs-models/demos/posenet/camera.html https://storage.googleapis.com/tfjs-models/demos/posenet/cam...