4 ms·

For the VFX industry, the tracking had already been solved for ages, with those reflective little balls on suits etc. in a mocap system. The Wii sensor bar's

For the VFX industry, the tracking had already been solved for ages, with those reflective little balls on suits etc. in a mocap system. The Wii sensor bar's thing was that it was really cheap. But yes, damn close to a holodeck. But you can't see depth in this setup, right?

whatshisface 7y ago
They have polarized 3D screens, and with head tracking, you have it.
- ragebol 7y ago
  Was thinking the same, but that'll also work for just 1 PoV
  - jacobush 7y ago
    Or VERY high framerates.
  - andybak 7y ago
    You could multiplex more images if the glasses were synced and actively polarized. Every person gets a timeslice of the total image.
rebuilder 7y ago
Mocap systems haven't really been able to produce deliverable results without human intervention very long. I'd argue they're still not there, some filtering and cleanup of the data is usually required. A lot of VFX is still about throwing human labour at problems. Edit: I should note I'm talking about motion capture for characters etc. Capturing the motion of a rigid object like a camera in a controlled environment is very doable.
- ChuckNorris89 7y ago
  You're right. The mocap data for the actor who played Thanos in Avengers had to be post processed by artist by hand since the motion of a regular man doesn't correspond to the motion of a larger and heavier Titan. I guess in a few years all that could be automated with ML.
  - estebank 7y ago
    You don't need ML to solve that problem, you can use inverse kinematics: https://youtu.be/KLjTU0yKS00 https://youtu.be/KLjTU0yKS00
nocut12 7y ago
If it's perspective corrected for the camera, it would probably look very distorted for anyone else on set -- whether there's depth or not And that's certainly not the goal with this. Something along those lines has been around for a while (https://en.wikipedia.org/wiki/Cave_automatic_virtual_environment https://en.wikipedia.org/wiki/Cave_automatic_virtual_environ...). This system seems specifically targeted for solving challenges for film production, as it probably should be. I am pretty impressed that real time rendering has gotten good enough to use for these purposes. I certainly wouldn't have expected that those backgrounds in the show were coming out of a video game engine.
- RaptorJ 7y ago
  From the video posted upthread, around 3:40 it looks like they're doing just that: https://youtu.be/gUnxzVOs3rk?t=220 https://youtu.be/gUnxzVOs3rk?t=220
  - mlrtime 7y ago
    Look at the ceiling, you can see it moving.
  - vernie 7y ago
    The most interesting aspect to me is that the system is pulling double duty by displaying both a dynamic, perspective-correct backdrop for the camera's POV and a static view for environment lighting and reflections outside of the camera's view frustum. I wonder if they had to take care to mitigate artifacts caused by the dynamic view bouncing off of surfaces facing away from the camera.
    - BubRoss 7y ago
      What artifacts?
      - rangibaby 7y ago
        I guess he is thinking about situations where you would use a negative fill
babypuncher 7y ago
It won't trick anyone with stereo vision, but the perspective correction provides all the depth cues needed to create a convincing 3D visual for pirates and non-stereo cameras.
- ragebol 7y ago
  pirates? What arrrr you talking about?
  - frostburg 7y ago
    Eyepatches.
  - deleted 7y ago
    [deleted]
- lsaferite 7y ago
  Interestingly I would imagine pulling the non-CG part out of the frame would be possible as they have the ability to generate the exact image without the real-world aspects. Basically a virtual green-screen. Combine that with a stereo camera and the fact that the source CG is actually 3D and you could get very convincing 3D movies I'd think. Yeah, that simplified a lot, but I'd say it's possible. And they still get the benefits of the lighting aspects as well as the immersion for the actors.

whatshisface 7y ago

They have polarized 3D screens, and with head tracking, you have it.

ragebol 7y ago
Was thinking the same, but that'll also work for just 1 PoV
- jacobush 7y ago
  Or VERY high framerates.
- andybak 7y ago
  You could multiplex more images if the glasses were synced and actively polarized. Every person gets a timeslice of the total image.

rebuilder 7y ago

Mocap systems haven't really been able to produce deliverable results without human intervention very long. I'd argue they're still not there, some filtering and cleanup of the data is usually required. A lot of VFX is still about throwing human labour at problems. Edit: I should note I'm talking about motion capture for characters etc. Capturing the motion of a rigid object like a camera in a controlled environment is very doable.

ChuckNorris89 7y ago
You're right. The mocap data for the actor who played Thanos in Avengers had to be post processed by artist by hand since the motion of a regular man doesn't correspond to the motion of a larger and heavier Titan. I guess in a few years all that could be automated with ML.
- estebank 7y ago
  You don't need ML to solve that problem, you can use inverse kinematics: https://youtu.be/KLjTU0yKS00 https://youtu.be/KLjTU0yKS00

nocut12 7y ago

If it's perspective corrected for the camera, it would probably look very distorted for anyone else on set -- whether there's depth or not And that's certainly not the goal with this. Something along those lines has been around for a while (https://en.wikipedia.org/wiki/Cave_automatic_virtual_environment https://en.wikipedia.org/wiki/Cave_automatic_virtual_environ...). This system seems specifically targeted for solving challenges for film production, as it probably should be. I am pretty impressed that real time rendering has gotten good enough to use for these purposes. I certainly wouldn't have expected that those backgrounds in the show were coming out of a video game engine.

RaptorJ 7y ago
From the video posted upthread, around 3:40 it looks like they're doing just that: https://youtu.be/gUnxzVOs3rk?t=220 https://youtu.be/gUnxzVOs3rk?t=220
- mlrtime 7y ago
  Look at the ceiling, you can see it moving.
- vernie 7y ago
  The most interesting aspect to me is that the system is pulling double duty by displaying both a dynamic, perspective-correct backdrop for the camera's POV and a static view for environment lighting and reflections outside of the camera's view frustum. I wonder if they had to take care to mitigate artifacts caused by the dynamic view bouncing off of surfaces facing away from the camera.
  - BubRoss 7y ago
    What artifacts?
    - rangibaby 7y ago
      I guess he is thinking about situations where you would use a negative fill

babypuncher 7y ago

It won't trick anyone with stereo vision, but the perspective correction provides all the depth cues needed to create a convincing 3D visual for pirates and non-stereo cameras.

ragebol 7y ago
pirates? What arrrr you talking about?
- frostburg 7y ago
  Eyepatches.
- deleted 7y ago
  [deleted]
lsaferite 7y ago
Interestingly I would imagine pulling the non-CG part out of the frame would be possible as they have the ability to generate the exact image without the real-world aspects. Basically a virtual green-screen. Combine that with a stereo camera and the fact that the source CG is actually 3D and you could get very convincing 3D movies I'd think. Yeah, that simplified a lot, but I'd say it's possible. And they still get the benefits of the lighting aspects as well as the immersion for the actors.