5 ms·

Since there's some confusion in a few threads, here a little bit of explanation. I spent the last ~18 months speeding up a production renderer for an upcoming a

Since there's some confusion in a few threads, here a little bit of explanation. I spent the last ~18 months speeding up a production renderer for an upcoming animated feature. 29 hours for a frame is high but not unusual. Referred to as "Blinn's Law", the tendency is that scene assets grow with computing power, eating up any performance gains that you may have gained from newer hardware. Render time for an animated movie today is still in the same order of magnitude as it was in 1995, using state of the art hardware. Now, how can they do that at 25fps all of a sudden? They're not. They're using the same method, but not even remotely the same scene complexity or image fidelity. Ray tracing, and its extension path tracing, scales very well. The same algorithm that works for 500 polygons works for 500 million polygons, as long as you can fit them in memory - it's just going to be slower (building quality acceleration structures is often O(n log(n)). If 10 rays per pixel don't give you the quality you want, you can go up to 100 or 1000 rays per pixel, with render time going up respectively. For film, we're rendering hundreds of primary rays per pixel (tens of secondary rays for each primary ray), with hundreds of millions of polygons per scene. I have no insight about the scene of Nvidia's demo, but it is claimed to run at 5 rays per pixel. So even if scenes were identical, it's already 20-500 times faster just by doing 20-500 times less work. Shading complexity is also a huge difference. Production rendering uses tens of GBs up to the TB range just for textures, on top of procedural shading. Not only does the computing part of that often dominate render times (instead of the ray tracing part), but just paging texture data in and out of memory on demand can become a bottleneck by itself. That Nvidia demo is a simple scene with no I/O overhead, rendered using a fraction of the rays used for film production, on 18 billion transistors of specialised hardware. Of course it's several orders of magnitudes faster, how could it not be?

gammateam 8y ago
> That Nvidia demo is a simple scene with no I/O overhead, rendered using a fraction of the rays used for film production. But if the consumer can't tell the difference, does it matter? Many films - in this example because we are talking about films - use a variety of visual techniques to reduce the complexity. Horror films for example rarely show the antagonist. The creature is shown in the dark, for split seconds, until a final expensive moment, leaving your brain to fill the rest in. Although YOU can't unsee it, the audience, nor the studio, should care that the stormtroopers are rendered on a simple set with the snowstorm being used solely to avoid the necessity of rendering. The sound of the creature gave the effect of a complex monster lurking in the midst and I don't need a raytracing demo to show that.
- swerner 8y ago
  > But if the consumer can't tell the difference, does it matter? The consumer gets used to it. The CG effects in Toy Story 1 look dated, and if you were to watch the Incredibles and the Incredibles 2 back to back, you would also notice immediately. Or compare the movie "Gravity" to "Babylon 5". Plus I've not even mentioned hair, fur and volumetrics. A staple in pretty much any feature film FX, they were completely absent in the real time ray tracing demos. Hair and fur are commonly not ray traced as triangles but use custom intersection routines. Those are an extra layer of complexity on top. I don't know yet how Nvidia's own API for the RTX card looks like, but Microsoft DXR and Apple's Metal ray tracing both are restricted to triangle geometry only.
  - gammateam 8y ago
    Toy Story 1 still had a budget of 30 million and a box office of 370 million, in 1995 dollars. Lets skip past the discipline and lets focus on the consequences. What are the consequences here? Why is Nvidia's new chip potentially failing product market fit?
    - swerner 8y ago
      Who mentioned failure? I'm merely explaining why and how Nvidia can show 5 rays/pixel @ 25pfs while we're at the same time budgeting several hours per frame for state of the art production rendering. Sure, more and better hardware will help film rendering. It always has.
  - agumonkey 8y ago
    This effect gives interesting perspective on pre CGI movies. You see how movie makers conveyed emotions, effect without very limited toolset and built great movies without rubbing the visual organ constantly. I find a lot of movies today are falling in the Lucas trap: too much CGI (cost reign supreme I guess). It's a bad soil for movies...
    - swerner 8y ago
      Just compare "South Park" to "Final Fantasy: The Spirits Within". CGI can't replace a good script.
swerner 8y ago
The special issue of the ACM Transactions on Graphics has several articles describing the inner workings of some current production renderers: https://dl.acm.org/citation.cfm?id=3243123&picked=prox https://dl.acm.org/citation.cfm?id=3243123&picked=prox
danmaz74 8y ago
Do you have a link to a video of the demo? I'd love to check how much I notice a difference with movie scenes.
- swerner 8y ago
  https://www.ustream.tv/NVIDIA https://www.ustream.tv/NVIDIA Real-time demo at 19m30s.
cmroanirgo 8y ago
Thanks for this heads up. I remember years ago, us developers would spin up slave program overnight so that the graphics guys could eke out a few frames of ray traced images. It's hilarious to me that Blinn's Law is alive and well (although the state of the web itself shows it). It was pre 3d card (and up until 3fdx/ early nvidia days), and drove me nuts that every time we managed to get another fps, the graphics guys would add a few hundred polys to be rendered by the realtime engine.
- erikpukinskis 8y ago
  To be fair, it’s completely different now. Artists can previsualize almost everything that’s happening in the renderer, often in real-time or close to it. It’s just the final frames that go to production that take hours to render.
skgoa 8y ago
So this is a toy example used for marketing. They are pulling the same shit with their automated driving "research".
- Keyframe 8y ago
  Yes and definitely no. RT rendering (or close to it, well more like RT rendering techniques) are making it into production. Not as much as film, but TV definitely and it's only a matter of time when two worlds converge. ZAFARI is a recent example - https://www.unrealengine.com/en-US/blog/animated-children-s-series-zafari-springs-to-life-with-unreal-engine https://www.unrealengine.com/en-US/blog/animated-children-s-...
master-litty 8y ago
How did you get into this industry? I'm fascinated by the nonchalant discussion that goes into developing animated features, it's been a dream of mine to penetrate the space. I'm a senior engineer, made my own renderer and software "suite" to go with it as a hobby project for the resume, but I'm not sure what else I can make to get my foot in the door. Do you have any general advice?
- erikpukinskis 8y ago
  Not OP, but look at the credits after films and look for special effect companies to apply to. There are so many, if you are free to move and you can write a renderer, you should be able to get an interview somewhere. Companies like that are constantly staffing up and laying people off with the film cycles so it shouldn’t be too hard to get in. Once in you just have to keep your eyes peeled for a good job in an abusive industry.
- swerner 8y ago
  15 years of experience in 3D application development outside of the of film industry, two years of contributions to Blender and a lot of luck of being at the right place at the right time. In my opinion, don't write your own renderer unless you can really invest a lot of time into it. There are countless "spheres in a Cornell box" path tracers out there, you'll not excite anyone with the basics. Contribute to existing software like Blender or Appleseed instead, that will give you a lot more visibility and demonstrate that you can write robust code.
dharma1 8y ago
Nvidia definitely knows how to massage perf figures for press releases. But I guess the tensor cores can help alleviate lower rays per pixel/more noise with machine learning based noise reduction - https://youtu.be/pp7HdI0-MIo https://youtu.be/pp7HdI0-MIo
sytelus 8y ago
Probably a stupid question: At what point there is a real perceptible difference for common user who is watching movie 10ft away from their 40” TV? I get that doing 100 Ray per pixel gives you quality that an expert need to look hard to find defects given expensive calibrated displays in well lighted controlled environment. But a normal user isn’t doing that. Their room lighting is already washing out on several artifacts. Their TV sets are not even properly calibrated or even 4K. For those common people, what is good enough?
- dragontamer 8y ago
  Its way more complicated than that. In the old days, artists would change the scenes to match the technology. If human skin couldn't be rendered well, you just showed off plastic and metals. Or, they'd make a "cartoony style" and hope that the audience would get used to the plastic barbie-looking skin. Or in the case of Toy Story: you just make a story revolve around plastic toys. Plastic looks good, remember, in 90s CGI. As long as actually humans are rare, the movie will look "realistic". But you can't just keep making movies about Toys with plastic-like skin. And improving the algorithms to have realistic skin requires harder calculations such as "subsurface scattering". Today, artists want to show off more materials, and they also want all those materials to be more believable. Pixar is insane with their details: they spent two years on creating the "Hank the Octopus" in "Finding Dory". And it shows: good reflections off the skin highlight the wetness of the Octopus, Hank walks and moves very much like a real octopus, etc. etc. ----------------- In short: there are some materials which don't require much processor time. You probably can get away with a toy-story like plastic look and feel with 10 samples per pixel (talking 90s era here, where things were still relatively flat and primitive). But if you want to create a CGI Grand Moff Tarkin in an otherwise live-action movie (Star Wars: Rogue One)... you'll need to do better than that. These scenes will naturally require 100s or 10,000 even samples before the human eye is tricked.

gammateam 8y ago

> That Nvidia demo is a simple scene with no I/O overhead, rendered using a fraction of the rays used for film production. But if the consumer can't tell the difference, does it matter? Many films - in this example because we are talking about films - use a variety of visual techniques to reduce the complexity. Horror films for example rarely show the antagonist. The creature is shown in the dark, for split seconds, until a final expensive moment, leaving your brain to fill the rest in. Although YOU can't unsee it, the audience, nor the studio, should care that the stormtroopers are rendered on a simple set with the snowstorm being used solely to avoid the necessity of rendering. The sound of the creature gave the effect of a complex monster lurking in the midst and I don't need a raytracing demo to show that.

swerner 8y ago
> But if the consumer can't tell the difference, does it matter? The consumer gets used to it. The CG effects in Toy Story 1 look dated, and if you were to watch the Incredibles and the Incredibles 2 back to back, you would also notice immediately. Or compare the movie "Gravity" to "Babylon 5". Plus I've not even mentioned hair, fur and volumetrics. A staple in pretty much any feature film FX, they were completely absent in the real time ray tracing demos. Hair and fur are commonly not ray traced as triangles but use custom intersection routines. Those are an extra layer of complexity on top. I don't know yet how Nvidia's own API for the RTX card looks like, but Microsoft DXR and Apple's Metal ray tracing both are restricted to triangle geometry only.
- gammateam 8y ago
  Toy Story 1 still had a budget of 30 million and a box office of 370 million, in 1995 dollars. Lets skip past the discipline and lets focus on the consequences. What are the consequences here? Why is Nvidia's new chip potentially failing product market fit?
  - swerner 8y ago
    Who mentioned failure? I'm merely explaining why and how Nvidia can show 5 rays/pixel @ 25pfs while we're at the same time budgeting several hours per frame for state of the art production rendering. Sure, more and better hardware will help film rendering. It always has.
- agumonkey 8y ago
  This effect gives interesting perspective on pre CGI movies. You see how movie makers conveyed emotions, effect without very limited toolset and built great movies without rubbing the visual organ constantly. I find a lot of movies today are falling in the Lucas trap: too much CGI (cost reign supreme I guess). It's a bad soil for movies...
  - swerner 8y ago
    Just compare "South Park" to "Final Fantasy: The Spirits Within". CGI can't replace a good script.

swerner 8y ago

The special issue of the ACM Transactions on Graphics has several articles describing the inner workings of some current production renderers: https://dl.acm.org/citation.cfm?id=3243123&picked=prox https://dl.acm.org/citation.cfm?id=3243123&picked=prox

danmaz74 8y ago

Do you have a link to a video of the demo? I'd love to check how much I notice a difference with movie scenes.

swerner 8y ago
https://www.ustream.tv/NVIDIA https://www.ustream.tv/NVIDIA Real-time demo at 19m30s.

cmroanirgo 8y ago

Thanks for this heads up. I remember years ago, us developers would spin up slave program overnight so that the graphics guys could eke out a few frames of ray traced images. It's hilarious to me that Blinn's Law is alive and well (although the state of the web itself shows it). It was pre 3d card (and up until 3fdx/ early nvidia days), and drove me nuts that every time we managed to get another fps, the graphics guys would add a few hundred polys to be rendered by the realtime engine.

erikpukinskis 8y ago
To be fair, it’s completely different now. Artists can previsualize almost everything that’s happening in the renderer, often in real-time or close to it. It’s just the final frames that go to production that take hours to render.

skgoa 8y ago

So this is a toy example used for marketing. They are pulling the same shit with their automated driving "research".

Keyframe 8y ago
Yes and definitely no. RT rendering (or close to it, well more like RT rendering techniques) are making it into production. Not as much as film, but TV definitely and it's only a matter of time when two worlds converge. ZAFARI is a recent example - https://www.unrealengine.com/en-US/blog/animated-children-s-series-zafari-springs-to-life-with-unreal-engine https://www.unrealengine.com/en-US/blog/animated-children-s-...

master-litty 8y ago

How did you get into this industry? I'm fascinated by the nonchalant discussion that goes into developing animated features, it's been a dream of mine to penetrate the space. I'm a senior engineer, made my own renderer and software "suite" to go with it as a hobby project for the resume, but I'm not sure what else I can make to get my foot in the door. Do you have any general advice?

erikpukinskis 8y ago
Not OP, but look at the credits after films and look for special effect companies to apply to. There are so many, if you are free to move and you can write a renderer, you should be able to get an interview somewhere. Companies like that are constantly staffing up and laying people off with the film cycles so it shouldn’t be too hard to get in. Once in you just have to keep your eyes peeled for a good job in an abusive industry.
swerner 8y ago
15 years of experience in 3D application development outside of the of film industry, two years of contributions to Blender and a lot of luck of being at the right place at the right time. In my opinion, don't write your own renderer unless you can really invest a lot of time into it. There are countless "spheres in a Cornell box" path tracers out there, you'll not excite anyone with the basics. Contribute to existing software like Blender or Appleseed instead, that will give you a lot more visibility and demonstrate that you can write robust code.

dharma1 8y ago

Nvidia definitely knows how to massage perf figures for press releases. But I guess the tensor cores can help alleviate lower rays per pixel/more noise with machine learning based noise reduction - https://youtu.be/pp7HdI0-MIo https://youtu.be/pp7HdI0-MIo

sytelus 8y ago

Probably a stupid question: At what point there is a real perceptible difference for common user who is watching movie 10ft away from their 40” TV? I get that doing 100 Ray per pixel gives you quality that an expert need to look hard to find defects given expensive calibrated displays in well lighted controlled environment. But a normal user isn’t doing that. Their room lighting is already washing out on several artifacts. Their TV sets are not even properly calibrated or even 4K. For those common people, what is good enough?

dragontamer 8y ago
Its way more complicated than that. In the old days, artists would change the scenes to match the technology. If human skin couldn't be rendered well, you just showed off plastic and metals. Or, they'd make a "cartoony style" and hope that the audience would get used to the plastic barbie-looking skin. Or in the case of Toy Story: you just make a story revolve around plastic toys. Plastic looks good, remember, in 90s CGI. As long as actually humans are rare, the movie will look "realistic". But you can't just keep making movies about Toys with plastic-like skin. And improving the algorithms to have realistic skin requires harder calculations such as "subsurface scattering". Today, artists want to show off more materials, and they also want all those materials to be more believable. Pixar is insane with their details: they spent two years on creating the "Hank the Octopus" in "Finding Dory". And it shows: good reflections off the skin highlight the wetness of the Octopus, Hank walks and moves very much like a real octopus, etc. etc. ----------------- In short: there are some materials which don't require much processor time. You probably can get away with a toy-story like plastic look and feel with 10 samples per pixel (talking 90s era here, where things were still relatively flat and primitive). But if you want to create a CGI Grand Moff Tarkin in an otherwise live-action movie (Star Wars: Rogue One)... you'll need to do better than that. These scenes will naturally require 100s or 10,000 even samples before the human eye is tricked.