15 ms·
Racing for Realism
- kakarot 9y agoThat was a good read. I'm excited to see these gains trickle down into video game rendering. A 35% gain on normal mapping performance without any user intervention would be huge. Next step is to explore a similar method with parallax mapping so that we can see these some of these gains transfer to VR titles.
- taw55 9y agoParallax mapping will not benefit from this, since the limiting factor there is performing a raymarch on a heightfield to find an exact intersection. At every step along the ray you need to test wether you are inside or outside of the heightfield. This means offsetting the uv coordinate (xy position inside the texture) by the ray vector, and then using that coordinate to sample the texture again, to check whether or not you penetrated the heightfield. The amount of texture lookups quickly becomes the bottleneck, especially on large textures since they incur cache misses. To give you an idea: for every parallaxed pixel on the screen the heightmap texture might be looked up several dozens of times. You don’t nearly get to subpixel accuracy before performance grinds to a halt. Parallax mapping is view dependent, so roughness mapping, even if somehow applicable would need to be highly anisotropic for it to work, which means a huge storage cost.
- kakarot 9y ago> for every parallaxed pixel on the screen the heightmap texture might be looked up several dozens of times I didn't realize it was on the order of dozens. Just to clarify, we're talking pixels and not texels, right? This isn't dependent on the resolution of the map? > roughness mapping, even if somehow applicable would need to be highly anisotropic for it to work I didn't think about that, I guess you're right. How huge, exactly? Seems like something you could compress very well if you combine textures into a larger megatexture.
- mc32 9y agoNice, I like the wear and aging element it affords the renderings. This makes me wonder if at some point realism will extend to movement whereupon a studio begins to shoot (render) feature films involving digital "people" as costs undercut actors. Will they be able to cultivate favorite "actors"? So you see an "actor" in different and disparate roles? In other words create interest and following for a constructed "actor".
- sewer_bird 9y agoCould be: a lot of the 'branding' of the actor these days is about curating what films they show up in. Like, if Johnny Depp is in a movie, you know generally what kind of film it'll be. The AA (Artificial Actor, haha) would benefit from having an agent, much like real actors, to keep consumers' meta-dissonance down. That said, actors swap between wildly disparate roles all the time and no one seems to care!
- sdrothrock 9y agoAki Ross from Final Fantasy: The Spirits Within (2001, https://en.wikipedia.org/wiki/Final_Fantasy:_The_Spirits_Within https://en.wikipedia.org/wiki/Final_Fantasy:_The_Spirits_Wit...) was actually supposed to be Square's first CG actress for exactly this kind of thing.
- ksec 9y agoWhich turns out to be not anywhere close to what audience were expecting. I still remember When Toy Story was shown in 1995 created on Silicon Graphics, they said everything was so crazy good, that at the rate of Moore's law, in 10 years time realism in animation is finally achievable. 2005 and The Spirit Within came and we are still no where close. Fast forward another 10 years in 2015 we are definitely better, but realism and Digital Actor is still pretty much a fantasy. At least the case the audience knew it was not real actor. That is 20 years with Moore's law, now Moore's law is gone and we are slowing down, what other kind of innovation will lead us to realism?
- taneq 9y ago
- mozumder 9y agoOne thing about 3-D animated movies is that can be rerendered later on with improved rendering tech without being reshot. Disney should go ahead and rerelease Toy Story with its latest rendering tech (at maybe 8k), to show these kinds of rendering improvements.
- danjc 9y agoA tremendous amount of effort would still be needed to provide detail to the models
- kayoone 9y agoyou would probably only improve the resolution and minor details, but the assets (models, textures, shaders etc.) created for a CG movie from the 90ies certainly won't offer many of the details that CG movies nowadays have.
- pavlov 9y agoRe-rendering Toy Story with a global illumination lighting model would make a huge difference though, even if the geometry and textures remain the same.
- tbabb 9y agoIt would look horrible, because the lighting was done assuming no global illumination. Besides destroying artistic intent, the no-longer-missing indirect lighting effects would double up with all the "cheats" and clever techniques the artists had placed to make up for them.
- pavlov 9y agoObviously all shader code and lighting would have to be redesigned. I wasn't trying to suggest that they could just multiply a GI render pass on top of the old shader output, or something like that.
- im3w1l 9y ago> So, problem solved, right? Not entirely. Though these microfacet approaches take surface orientation into account and give us realistic results while shading, the problem is that they don’t take into account the geometric microfacet details contributed by Bump or Displacement mapping which can end up inside a single pixel and get filtered out Could someone explain this in more detail?
- LoSboccacc 9y agoCheck the bump and bump+roughness disk platter image Basically the scratches done via displacement mapping aren’t filtered trough sourface roughness using the previous approach. The new approach appies the roughness filtering on top of the displacement map, so even displaced geometry partecipates into the roughness map. physical based rendering pipeline is basically the swap between the bump and roughness filters.
- Animats 9y agoNow someone has to implement this for graphics card shaders in real time.
- danieltillett 9y agoIt is coming.
- Cthulhu_ 9y agoEhhh. I don't think modern computers could even render Toy Story still, maybe an approximation but that's what realtime graphics are about - using tricks to make approximations, to make things look "good enough". Non-realtime graphics feel easier in some ways because you can use e.g. raycasting to get "free" realistic reflections and lighting and whatnot. Then you can focus more on what makes a material look the way it does, like they do in this article.
- crummy 9y agoYou're probably right: https://www.quora.com/How-much-faster-would-it-be-to-render-Toy-Story-in-2011-compared-to-how-long-it-took-in-1995 https://www.quora.com/How-much-faster-would-it-be-to-render-... Going by the vague dates in the article it's probably ~1 min per frame rendering Toy Story today.
- AboutTheWhisles 9y agoI think the original Toy Story could be done in real time with modern graphics cards. The Quora answer below was talking about CPUs in 2011, and it seems that plenty of added time (relatively speaking) was due to network and disk infrastructure not having the same 1000x speedup. A more subtle aspect to consider is that CPUs might have increased in raw computation by huge orders of magnitude, but memory latency means that the same programs from 20 years ago will run much faster, but not at the same multiple of the raw flops. This implies that to get the full effect of a cpu or gpu, the software would have to be rearchitected to an extent to get the full benefit of modern hardware. Toy Story didn't use any ray tracing, Renderman at the time was reliant on shadow maps. Because of the lack of global access to the scene from shaders, the local illumination Reyes architecture should map extremely well to GPUs. I think it is possible that someone will do similar things with Vulkan or openGL compute buffers if they haven't already.
- LoSboccacc 9y agoVery interesting article. I remember beikg mesmerized by the old track pannings in cars and thinking how far cgi has come, reading this afterward provides an excellent background to that feeling
- contravariant 9y agoWhy is this page hijacking the scroll wheel? As far as I can tell there's literally no point in doing so.
- kwcto 9y agoAt a glance, there appears to be almost no rendering performance difference vs standard normal mapping. Seems like it would be feasible in realtime engines. Baked normal maps are commonplace now in leading game titles. AR/VR stereoscopic views would make anisotropic surface effects, diffraction, etc even more obvious for those micro details and really help sell the illusion. Very interested in seeing Cars 3 in 3D to test this hypothesis...
- taw55 9y agoIt’s an additional texture fetch on the gpu, it’s virtualy free if you discount storage costs. It’s been in games for years.
- jayd16 9y agoAdditional texture fetch is by no means free but when the alternative is 100x SSAA you can get away with a lot.
- cma 9y agoValve paper on it from 2015: http://media.steampowered.com/apps/valve/2015/Alex_Vlachos_Advanced_VR_Rendering_GDC2015.pdf http://media.steampowered.com/apps/valve/2015/Alex_Vlachos_A... Scroll to the part about normal maps.
- icc97 9y agoIt was all super impressive until the last photo of the sunset shot over the bridge, there seems something artificial about that. Possibly the blur of the car on the far side, but it's more than that. Just not sure what.
- TheRealPomax 9y agoOne problem is that if you don't live at the same longitude as where this shot is simulated, the colours are literally going to look wrong, giving you an overall feeling of "there's something off about this and I don't know what". It's the same reason why you can regret buying a beautiful jacket in Norway, only to discover it looks "completely different" when you get back to your homestead in Wisconsin.
- menacingly 9y agobut would we look at a photograph taken in another part of the world and suspect it of being computer generated?
- Leszek 9y agoYou would if it were in an article about computer generated imagery.
- menacingly 9y agoThat's true. I guess the context is key here!
- rmcpherson 9y agoWhy would longitude affect an image like this? Can you elaborate?
- dmd 9y agoThey meant latitude.
- geekamongus 9y agoWhy do we try to recreate reality instead of maintaining some amount of fantasy/imagination/unrealism? When does it make more sense to simply photograph a car than to try and make a digital image of a car look as real as possible?
- RubenSandwich 9y agoWe wouldn't have VR/AR today unless we pushed our GPUs till they cried. We wouldn't even have deep neural networks today if we didn't push hyper-realism because most deep neural network training without today's GPUs would be too slow to be useful. The truth of the matter is that just because it seems trivial in one domain doesn't mean it won't benefit other domains, the problem is that we can't see the future so who knows what is going to pay dividends.
- behringer 9y agoThere's plenty of things you can't photograph in reality, but are no problem digitally.
- gpm 9y agoIf you're making a digital image of a car being crushed... only when it is, and will remain in future shots, cheaper to crush a physical car. (Note that in some domains, such as video games, the cost of actually crushing the car is effectively infinite. In others, such as movies with animated talking cars, it may only be ridiculously high.)
- azernik 9y ago> When does it make more sense to simply photograph a car than to try and make a digital image of a car look as real as possible? If you're making a movie? Not as often as you might think. With real cars you have to acquire them, transport them, close off streets to film them, in many shots rent out a helicopter to get a shot in, you're limited to certain times of day to get the right lighting... Animation became part of film and (especially) TV quite early, and it wasn't about some stylized unrealistic aesthetic; it's just plain cheaper. And then of course you're going to want to get around the limitations of the form, and put your anthropomorphizing eyes onto objects that are as realistic as possible.
- WhitneyLand 9y agoGoing to work at a competitor to Pixar in ‘96, it was the first time I was exposed to real life computer science problems that needed more compute power than could be imagined. Between then and 2017, think about the staggering performance increases available, more so than in many fields because of the embarrassing (parallel nature) thing inherent to many of the problems. And yet there is still no end in sight to the compute power that could be leveraged. I wonder what would change if we had say, 3 orders of magnitude performance improvements across the board starting tomorrow. What might be some of the first practical advancements to be exploited and seen? Let’s assume the amount of compute power that could be leveraged effectively and practically for computer graphics is finite, however large. How much less compute power would that be, compared to what it’s going to take achieve general AI? Or even a Turing effective chat bot?
- frik 9y agoI like the 3D animation movies of Pixir with non-realistic rendering (especially from 1990s, 2000s). Though like many I dislike the CGI movies that started with the first Transformer movie (with blue screen action) and now with the dozens of super hero boring fad. Action movies of the 1980s and 1990s and early 2000s were so great. Nowadays thankfully they use more real stunts again, though this mass of boring super hero and hundreds of TV series that no one has time to see. All we want is new high value Hollywood movies.
- dahart 9y agoThat's an interesting question, with an interesting premise. Will realism (graphics + physics simulation) take less compute than AI for sure? If we look at the limit case, the reality I can see out my window has more atoms than are in my brain. So, in theory perhaps, human-equivalent AI will similarly require less compute than realistic graphics & physics simulation/animation. OTOH, evolution took billions of years and presumably depends on all the animals and all the physics that has ever happened, so maybe inference will always be cheaper than graphics, and maybe AI training and evolution will always be more expensive? I have no idea, but it's a fun thought experiment. FWIW, I also worked for a Pixar competitor, and the renderfarm we had was indeed more compute than I had been exposed to before. That said, we didn't make a huge amount of effort to optimize the system, we only made sure really stupid things didn't cause overnight renders to take longer than required for dailies the next day. I'm absolutely certain that at least 1 order of magnitude is available to software render farms, if the primary focus shifts from production to optimization. I'm halfway sure that using today's and tomorrow's GPUs could get you at least another order of magnitude, but that comes with a second level of expensive engineering effort. In short, I think if you really really wanted it, and had the money & time to commit, you could probably have your three orders of magnitude right now.
- pecg 9y agoPersonally, I think one of the main problems with current use of CGI (specifically in non-animated movies) is color treatment. Comparisons between the original non-altered version of Jurassic Park (JP) against Jurassic World (JW) shows my point. There is an increased tendency to change hue and saturation of color in post-production, in order to transmit ideas or set moods on the film itself, but this makes CGI look faker than what it is and the brain is able to identify it. EDIT: I prefer the look of JP over JW, not because I think it has better CGI, but because colors look more closer to reality.
- niceperson 9y agoThat's correct. You can compare Snyder's work in Batman v Superman to Joss Whedon's last-minute changes in Justice League and how the CGI is actually worse off in a film that has a higher budget and lower runtime than the former.
- pecg 9y agoJustice League is in Star Wars: The Attack of the Clones' level of horrendous abuse of CGI.
- Balgair 9y agoI mean, complaining about the color realism of a film with gigantic monster dinosaurs (whose coloration we may never know to begin with) seems a bit silly. Movies are not documentaries. I'd buy in a bit more for things like the BBC's 'Planet Earth' needing color realism, but even then it's ok to dramatize things. Every Frame a Painting goes over Michael Bay's work and includes some color content he uses and WHY: https://www.youtube.com/watch?v=2THVvshvq0Q https://www.youtube.com/watch?v=2THVvshvq0Q
- Danihan 9y agoIt's not silly when we're specifically discussing REALISM
- 9y ago
- dmos62 9y agoI swear for about 3 minutes I thought the title was "Racism for Realism". I said to myself, of course, Pixar. Can't have a believable fairytale without it.
- acd 9y agoMaybe there will be artificial intelligence painters for cgi movies? What I mean is AI that learns from real photos of similar objects and paints the computer generated images in a similar style. Deep learning AI can already colorize black and white images quite good. Maybe the cgi artificial look can be seen as an art form in itself and 100% realism is not the end goal for children movies.
- fwilliams 9y agoSince the article doesn't go into a whole lot of technical detail about the shading technique, here's the pixar whitepaper describing it: https://graphics.pixar.com/library/BumpRoughness/paper.pdf https://graphics.pixar.com/library/BumpRoughness/paper.pdf
- debt 9y agoIt's crazy to think that someday soon. We will have assembled actors. People that don't actually exist, appear to be human but are CGI, and that the audience prefer over real, living humans.
- glitcher 9y agoThis idea has been explored at least a few times in movies: S1m0ne (2002) http://www.imdb.com/title/tt0258153/ http://www.imdb.com/title/tt0258153/
- tmoravec 9y agoHyper realistic talking cars.