3 ms·
We're at a cusp in technology where "reverse rendering" is rapidly progressing from a research curiosity to a mainstream product. As soon as high-quality model
by jiggawatts 4y ago
We're at a cusp in technology where "reverse rendering" is rapidly progressing from a research curiosity to a mainstream product. As soon as high-quality model generation becomes possible on a typical workstation, there will be a dramatic shift in how computer game art assets are produced. Many artists will be replaced by this mechanical, automated method of 3D model production.
No longer will you need a team of 100 artists for every 2-3 developers to manually model and texture objects, instead you'll have 10-20 photographers running around taking pictures of real-life objects and scenes. These will then be reverse-rendered into models, which can then be directly placed into the game world by just a handful of map designer artists.
You can see this in Unreal Engine 5, where they combined their "infinite detail" polygon engine Nanite with their acquisition of Quixel's Megascans library of models captured via photogrammetry.
However, photogrammetry is not the same as reverse rendering: it's only really suitable for diffuse objects captured in good conditions (foggy or overcast days). It can't handle transparency, reflections, or complex/dynamic self-shadowing. Currently it's mainly used for backdrops like rocks, logs, and road surfaces.
Reverse rendering can in principle capture anything a renderer can do, including specular materials. It can "undo" self-shadowing and produce objects that can be re-lit in different scenes.
For gamers, these are going to be exciting times!
For graphic artists... perhaps also exciting, but not necessarily in a good way.
- paisawalla 4y agoIt seems like this means artists would mostly be needed for creative art, e.g. spaceships, monsters, etc. That is, things that don't exist, so can't be photographed. But there are already commercial and free libraries of commonplace items like furniture, cars, and buildings. So isn't what I described pretty close to the same situation today already?
- taylorius 4y agoBut spaceships and monsters do exist. There are thousands of them, just look on google images. An AI inverse rendering system, could just as easily ingest a ton of space sci-fi images, as real world photos.
- esperent 4y ago> For graphic artists... perhaps also exciting, but not necessarily in a good way. I think you're missing an important aspect of art - style. The CG industry has been chasing photorealism for decades and we've been slowly inching closer. Photogrammetry is an important step that will provide a big boost in that goal. But what happens when we get there? Once it's (relatively) cheap to make a AAA game that's virtually indistinguishable from reality, what happens next? Everyone will love it for a few years but then they'll start to demand something new, and that might be non-photorealistic styles. Or it might be off world or impossible environments. Photogrammetry won't help much with a game set in space in the distant future, populated with non Earth creatures, plants, and architecture. Although it will help a bit, of course. There will still be a place for ultra realistic environments (war games, spy games, romance games etc.) but it will become just one style amongst many. If anything this might lead to a revival in making real world models and then scanning them. Imagine a Dark Crystal style world except you are the player. All the characters and models were crafted by hand and then scanned into a 3d world where they are indistinguishable from their real world counterparts when viewed in VR/AR or on screen. That would be cool. As for artists, perhaps they should be worried about AI stealing their jobs. But not photogrammetry, that'll just remove some of the tedious stuff like modeling furniture.
- _carbyau_ 4y agoI second the "style" aspect. For a games example, Call Of Duty vs Borderlands - both FPS but the visual styles are worlds apart. This could provide a new tool, and maybe remove some jobs. But it will make new jobs related to this new tool.
- jiggawatts 4y agoNeural nets can do style transfer. It might be as simple as making photorealistic models and then transferring a style from a handful of non-realistic samples.
- hypertele-Xii 4y agoAnd artist are still needed to create new styles.
- themerone 4y agoI think you misjudge how much artistry goes into placing those photographed assets. You are probably way off on your estimates in how this will change the workforce. Small teams of artists can now produce television VFX that rival what took hundreds of film artists to do decades ago, but if you pay attention to movie credits, the number of artists working on big films seems to be increasing. Also, it seems to me that AI and procedural generated assets are more likely to be the future than real world scans.
- naillo 4y agoIt'll be interesting which of these win; scanning real life objects as you describe or just generating 3d assets directly through a DALL-E style model.