7 ms·

Unreal is now starting to be used in place of green screens in some movies. (there's a lot more that goes into it, but essentially it seems like unreal is one

by swalsh 6y ago

Unreal is now starting to be used in place of green screens in some movies. (there's a lot more that goes into it, but essentially it seems like unreal is one of the core software pieces) Here is an example: https://www.youtube.com/watch?v=Hjb-AqMD-a4 https://www.youtube.com/watch?v=Hjb-AqMD-a4

jotm 6y ago
Amazing, the future of this tech is just exciting
centimeter 6y ago
The Mandalorian filmed a lot of scenes with UE4 doing real-time rendering onto large LED screens like in that demo. Looked pretty good! Probably more fun and easier for the actors too.
- miohtama 6y ago
  Here is how the production looked https://youtu.be/gUnxzVOs3rk https://youtu.be/gUnxzVOs3rk
  - diimdeep 6y ago
    Incredible. Replace green screen with real time rendered scene. Truman Show IRL.
    - ngold 6y ago
      We are getting closer to wallpaper and screen meaning the same thing.
    - m3at 6y ago
      If you're interested, this was a good article going into details of what's happening behind the scene for the real time rendering: https://ascmag.com/articles/the-mandalorian https://ascmag.com/articles/the-mandalorian
- nabla9 6y ago
  If you can afford to put one or more GPU's per LED screen, there is little limit what you can render real-time, especially when the camera is so far away. I wonder how many GPU's that system had.
  - penagwin 6y ago
    > If you can afford to put one or more GPU's per LED screen, there is little limit what you can render real-time Umm, Ray tracing would love to have a word :P In all seriousness, typical animated frames for big budget films easily take hours or longer _per frame_. It really depends on what looks you're trying to achieve. Game engines have come a long way in terms of realistic graphics with realtime rendering, but it's worth noting that it's still not the same quality as a fully ray traced scene (whether that matters depends on the content I guess)
    - nabla9 6y ago
      Yes, of course. But fully animated CGI traces everything back to camera and single screen in movie quality. This setup has 1,326 individual screens, 123,904 px/m² filmed from several meters away for 180 degree view. None of those screens were rendered even close to movie quality. btw. Only about 50% of the scenes were made with this setup. rest was traditional ray-tracing.
    - basch 6y ago
      A lot of what/why they are doing it this way, is realistic lighting and reflections. Im not sure the difference between a realtime game engine and ray tracing matters that much when you are using it as faux ambient light. After it is filmed, they can still go back and touchup the backgrounds. Someday with ray tracing they can do real time finished products, but for now the tech works great at what its intended to do.
      - jl6 6y ago
        If a scene has been filmed in this setup, how easy is it to separate the physical foreground from the background screen if they want to re-composite the foreground with a more detailed rey traced background?
- mkl 6y ago
  That was done for lighting. The lower quality real time imagery was replaced with offline rendered versions in post-production. "The final animated VFX would be added in later; the screens were merely to provide interactive lighting to match the animations." -- https://ascmag.com/articles/the-mandalorian https://ascmag.com/articles/the-mandalorian, discussed here: https://news.ycombinator.com/item?id=22378679 https://news.ycombinator.com/item?id=22378679
  - theptip 6y ago
    > That was done for lighting. I don't think that's correct, the ILM team presented this technology in a video: https://youtu.be/gUnxzVOs3rk?t=104 https://youtu.be/gUnxzVOs3rk?t=104 (paraphrased) "it was designed to both light the actors and be a background that we can photograph, so that we end up with real-time live pixels in-camera" Also from the article you linked: > If the content was created in advance of the shoot, then photographing actors, props and set pieces in front of this wall could create final in-camera visual effects — or “near” finals, with only technical fixes required, and with complete creative confidence in the composition and look of the shots. I'm not an expert in film terminology but it does not sound by any stretch that this was primarily about lighting.
    - mkl 6y ago
      I hadn't seen that video. I wonder how much was just left as the background screens (out of focus backgrounds in closeup shots seem likely). It does make it clear that the ceiling screens were mostly just for lighting, as they don't photograph well from the low angles. It was for was shot composition as well; the cinematographer could see how it would end up while shooting. The screens updated to match the camera position in real time (e.g. to show the rest of a spaceship whose door was a physical prop).
      - ghayes 6y ago
        As mentioned a few times in this comment section, this article covers the subject pretty well: https://ascmag.com/articles/the-mandalorian https://ascmag.com/articles/the-mandalorian. All in all, it looks like many shots were truly captured at or near final, without significant post-production modifications.
alufers 6y ago
This is really cool, looks like HDR environments in real life. Makes me wonder if this could be used in conjunction with cameras and some clever image processing algorithms as a camouflage method.
bilekas 6y ago
This is really cool, I just wonder for this particular demo, would it be cheaper to film in a real world location! :)
- ksec 6y ago
  For movies industry, a lot of the cost are actors and time involved. Having everything filmed with pre made scenes saves you lots of time comparing to green screen back and forth with VFX. Real world location also means lots of travelling.
  - atonse 6y ago
    Traveling with lots of heavy, expensive equipment, setting up, tearing down, then all the unpredictability of outdoor anything (weather, etc). Also one amazing thing was the guy talking about "shooting a 10-hour dawn" – I can imagine a lot of time is wasted trying to recreate a certain time of day. With this, you don't waste that time. You can do as many takes as you need. The sun stays right where it is. Absolutely mind-boggling.
    - bilekas 6y ago
      > Also one amazing thing was the guy talking about "shooting a 10-hour dawn" I watched the whole video on the MAnchurian example, and it sold it obviously for me, but I was only initially thinking of the first demo. Clearly its hugely beneficial. What I find interesting is, naughty dog had a bit of a staff turnover for designers and hired film animators (in this case it was a crunch), cgi etc.. Funny how they're now almost cross skillset now. Have to admit, makes me feel older every day. REF: https://www.eurogamer.net/articles/2020-03-14-crunch-once-again-in-the-spotlight-after-damning-report-on-the-last-of-us-2-developer-naughty-dog https://www.eurogamer.net/articles/2020-03-14-crunch-once-ag...
ACow_Adonis 6y ago
I wonder if this has anything to do with my wife's complaint (and I agree with her on this) that modern productions are beginning to look more and more like computer games. She's been watching the third season of westworld, and it's kinda scary to me how almost every episode feels more like a game trailer than an actual story...
- ksec 6y ago
  Does she watch Game of Thrones? What does she think about those VFX? Or which drama does she think that is not Computer Gaming Graphics like? It is an interesting complaint. Need some time to think about that in details.
  - ACow_Adonis 6y ago
    We actually binge watched game of thrones for the first time while we were both on mat leave. Hadn't seen a single episode before then. The difficulty there is that I think everyone universally acknowledges (?) GOT just fell apart like a car crash in slow motion in the later seasons. It's hard to separate that from the knowledge that they started to depart from the books and were clearly undergoing irreparable problems/pressures by the end of season 5. Don't think we can blame that solely on tech... And it's hard to think about the cgi specifically when the rest is falling apart...

by swalsh 6y ago

jotm 6y ago

Amazing, the future of this tech is just exciting

centimeter 6y ago

The Mandalorian filmed a lot of scenes with UE4 doing real-time rendering onto large LED screens like in that demo. Looked pretty good! Probably more fun and easier for the actors too.

miohtama 6y ago
Here is how the production looked https://youtu.be/gUnxzVOs3rk https://youtu.be/gUnxzVOs3rk
- diimdeep 6y ago
  Incredible. Replace green screen with real time rendered scene. Truman Show IRL.
  - ngold 6y ago
    We are getting closer to wallpaper and screen meaning the same thing.
  - m3at 6y ago
    If you're interested, this was a good article going into details of what's happening behind the scene for the real time rendering: https://ascmag.com/articles/the-mandalorian https://ascmag.com/articles/the-mandalorian
nabla9 6y ago
If you can afford to put one or more GPU's per LED screen, there is little limit what you can render real-time, especially when the camera is so far away. I wonder how many GPU's that system had.
- penagwin 6y ago
  > If you can afford to put one or more GPU's per LED screen, there is little limit what you can render real-time Umm, Ray tracing would love to have a word :P In all seriousness, typical animated frames for big budget films easily take hours or longer _per frame_. It really depends on what looks you're trying to achieve. Game engines have come a long way in terms of realistic graphics with realtime rendering, but it's worth noting that it's still not the same quality as a fully ray traced scene (whether that matters depends on the content I guess)
  - nabla9 6y ago
    Yes, of course. But fully animated CGI traces everything back to camera and single screen in movie quality. This setup has 1,326 individual screens, 123,904 px/m² filmed from several meters away for 180 degree view. None of those screens were rendered even close to movie quality. btw. Only about 50% of the scenes were made with this setup. rest was traditional ray-tracing.
  - basch 6y ago
    A lot of what/why they are doing it this way, is realistic lighting and reflections. Im not sure the difference between a realtime game engine and ray tracing matters that much when you are using it as faux ambient light. After it is filmed, they can still go back and touchup the backgrounds. Someday with ray tracing they can do real time finished products, but for now the tech works great at what its intended to do.
    - jl6 6y ago
      If a scene has been filmed in this setup, how easy is it to separate the physical foreground from the background screen if they want to re-composite the foreground with a more detailed rey traced background?
mkl 6y ago
That was done for lighting. The lower quality real time imagery was replaced with offline rendered versions in post-production. "The final animated VFX would be added in later; the screens were merely to provide interactive lighting to match the animations." -- https://ascmag.com/articles/the-mandalorian https://ascmag.com/articles/the-mandalorian, discussed here: https://news.ycombinator.com/item?id=22378679 https://news.ycombinator.com/item?id=22378679
- theptip 6y ago
  > That was done for lighting. I don't think that's correct, the ILM team presented this technology in a video: https://youtu.be/gUnxzVOs3rk?t=104 https://youtu.be/gUnxzVOs3rk?t=104 (paraphrased) "it was designed to both light the actors and be a background that we can photograph, so that we end up with real-time live pixels in-camera" Also from the article you linked: > If the content was created in advance of the shoot, then photographing actors, props and set pieces in front of this wall could create final in-camera visual effects — or “near” finals, with only technical fixes required, and with complete creative confidence in the composition and look of the shots. I'm not an expert in film terminology but it does not sound by any stretch that this was primarily about lighting.
  - mkl 6y ago
    I hadn't seen that video. I wonder how much was just left as the background screens (out of focus backgrounds in closeup shots seem likely). It does make it clear that the ceiling screens were mostly just for lighting, as they don't photograph well from the low angles. It was for was shot composition as well; the cinematographer could see how it would end up while shooting. The screens updated to match the camera position in real time (e.g. to show the rest of a spaceship whose door was a physical prop).
    - ghayes 6y ago
      As mentioned a few times in this comment section, this article covers the subject pretty well: https://ascmag.com/articles/the-mandalorian https://ascmag.com/articles/the-mandalorian. All in all, it looks like many shots were truly captured at or near final, without significant post-production modifications.

alufers 6y ago

This is really cool, looks like HDR environments in real life. Makes me wonder if this could be used in conjunction with cameras and some clever image processing algorithms as a camouflage method.

bilekas 6y ago

This is really cool, I just wonder for this particular demo, would it be cheaper to film in a real world location! :)

ksec 6y ago
For movies industry, a lot of the cost are actors and time involved. Having everything filmed with pre made scenes saves you lots of time comparing to green screen back and forth with VFX. Real world location also means lots of travelling.
- atonse 6y ago
  Traveling with lots of heavy, expensive equipment, setting up, tearing down, then all the unpredictability of outdoor anything (weather, etc). Also one amazing thing was the guy talking about "shooting a 10-hour dawn" – I can imagine a lot of time is wasted trying to recreate a certain time of day. With this, you don't waste that time. You can do as many takes as you need. The sun stays right where it is. Absolutely mind-boggling.
  - bilekas 6y ago
    > Also one amazing thing was the guy talking about "shooting a 10-hour dawn" I watched the whole video on the MAnchurian example, and it sold it obviously for me, but I was only initially thinking of the first demo. Clearly its hugely beneficial. What I find interesting is, naughty dog had a bit of a staff turnover for designers and hired film animators (in this case it was a crunch), cgi etc.. Funny how they're now almost cross skillset now. Have to admit, makes me feel older every day. REF: https://www.eurogamer.net/articles/2020-03-14-crunch-once-again-in-the-spotlight-after-damning-report-on-the-last-of-us-2-developer-naughty-dog https://www.eurogamer.net/articles/2020-03-14-crunch-once-ag...

ACow_Adonis 6y ago

I wonder if this has anything to do with my wife's complaint (and I agree with her on this) that modern productions are beginning to look more and more like computer games. She's been watching the third season of westworld, and it's kinda scary to me how almost every episode feels more like a game trailer than an actual story...

ksec 6y ago
Does she watch Game of Thrones? What does she think about those VFX? Or which drama does she think that is not Computer Gaming Graphics like? It is an interesting complaint. Need some time to think about that in details.
- ACow_Adonis 6y ago
  We actually binge watched game of thrones for the first time while we were both on mat leave. Hadn't seen a single episode before then. The difficulty there is that I think everyone universally acknowledges (?) GOT just fell apart like a car crash in slow motion in the later seasons. It's hard to separate that from the knowledge that they started to depart from the books and were clearly undergoing irreparable problems/pressures by the end of season 5. Don't think we can blame that solely on tech... And it's hard to think about the cgi specifically when the rest is falling apart...