13 ms·

The Problem With Photorealism

163 points by blackhole 13y ago

PhasmaFelis 13y ago
Minecraft is an interesting case study here, in that it deliberately throws away realistic graphics in favor of a comprehensively mutable environment. It's interesting how that seemed like such a totally unprecedented idea at the time.
- danso 13y ago
  I'm not sure where Minecraft came in the recent era of gaming...since it was built in 2009, that was well after games started being big on iOS, and some of those big games went for 8-bit as a way to maintain a modern sense of style and artistic quirk with the limited dev and system resources. Also, notch got some inspiration from Dwarf Fortress, which made an even more extreme sacrifice of graphics (the precursor to Dwarf Fortress was a 3D game that was too difficult to develop for). Either way, who could've guessed back in 2004ish that pixelated retro games (including FTL, TowerFall, and countless iOS hits) would be what gaming would look like 10 years later?
  - lazugod 13y ago
    > Either way, who could've guessed back in 2004ish that pixelated retro games (including FTL, TowerFall, and countless iOS hits) would be what gaming would look like 10 years later? Flash games were probably the 2004-era precedent.
- AJ007 13y ago
  It wasn't unprecedented, just look at Dwarf Fortress. Tarn threw out all graphics in exchange for focusing on an incredibly complex and deeply layered game -- with an environment that is fully constructable and has both a geological and cultural history. Once the 3D accelerator become an item all gamers had, (perceptually) it appeared all major game develop shifted to producing "photorealistic" graphics. We ended up with an endless trail of games that basically produced the same thing. I wonder, what would happen if the budget of something like Call of Duty was spent on a game with Minecraft or even Ascii graphics? Perhaps a total disaster, but what could be produced if it was done really well?
  - dasil003 13y ago
    The current state of Dwarf Fortress probably is within an order of magnitude of the number man hours in it as an early Call of Duty. The difference is that the work on Call of Duty is mostly art which is trivially parallelizable, whereas the work on Dwarf Fortress is the engine which is not parallelizable to anywhere near the same degree.
- mnw21cam 13y ago
  Once upon a time there was an open source game called "Cube" which was a first person shooter. It allowed in-game multi-player level editing, with all the occlusion and lighting recomputed on the spot. It was superseded by Sauerbraten, which improved the level structure to allow multiple floors, but didn't run as quickly due to the calculations being a bit harder.
- andrey-p 13y ago
  A game with very stylised graphics will always win over a game with very realistic graphics as far as I'm concerned. If you've got a game with unassuming graphics it's a lot easier to focus on the gameplay. Alternatively, stylising things _just right_ can get you a lot further in creating a specific atmosphere - games like Limbo work because everything fits together so perfectly.
coldcode 13y ago
Having worked on a FPS MMO, I can't agree with this enough. People always complained it didn't look real enough. I always wanted to ask them if would mind buying 1,000 video cards up front and then wait 10 years buying 1000 new ones every year until I got it right. I figured by then I could retire and duck the problem. Reality is ridiculously hard to do right. Now try doing it over the internet with a half second ping time and you realize that not only reality bites, but physics bites harder.
- chiph 13y ago
  While WoW looks the way it does partly because of it's 10 year old origins, it's artwork also communicates to the player: "It's just a game". And once you're far enough away from an imitation of reality, the brain stops trying to fit it into it's model of the world and complaining when it can't.
zeidrich 13y ago
Just simulate the entire universe. It requires a computer the size of the universe. There is one freely available, however, you will have to share it with a few other people. It's barely noticeable though because it's highly parallelizable.
- guizzy 13y ago
  It's rather poorly documented, though.
  - mikeash 13y ago
    And extremely buggy.
    - Houshalter 13y ago
      It's pretty bad. The other day an entire plane just disappeared. And I have yet to see someone respawn properly.
      - mikeash 13y ago
        I saw the greatest fake release notes entry the other day: "Fixed bug where matter would sometimes spontaneously become self aware."
      - z3phyr 13y ago
        Its a bug, and the 'AI' is trying to fix it.
  - resu_nimda 13y ago
    Proprietary assholes. Maybe if they just made the source available we wouldn't have to spend countless man-millennia trying to reverse-engineer this thing.
  - mamcx 13y ago
    No poorly, is very well documented. The problem is that we run inside the exe, no in the source code!
robmcm 13y ago
This guy isn't thinking outside the box. He's trying to solve the problem using techniques from the past. Obviously the solution isn't to manually make levels, it would be to automate them. Where are EA on flying scanning drones around football stadia in order to save artists thousands of man hours? They already use these techniques to scan actors faces, I can imagine someone from 10 years ago predicting it would be impossible to script all the movements of lips and eyelids to produce realistic characters. Disclaimer: I have no idea what I am talking about, but thought my off the cuff theory would have more weight if I wrote like I did.
- msandford 13y ago
  That kind of tech is definitely coming, but the game studios aren't developing it. They'll buy it once it comes out though. Game studios aren't VCs. A VC buys pieces of HIGHLY speculative companies trying to maximize his/her odds of a 1000:1 exit. Game studios fund a small portfolio of games and try and make a profit on every single one. The culture in a studio is so completely different it's very unlikely that anyone high enough up at a game studio to have the budget and other resources to try and develop it in-house won't have the risk appetite to actually do so. And the people who have the risk appetite won't have the budget.
- chasing 13y ago
  > Disclaimer: I have no idea what I am talking about, but thought my off the cuff theory would have more weight if I wrote like I did. Ha. I like the cut of your jib.
- Gracana 13y ago
  iRacing scans racetracks, but that's for physical accuracy; it likely increases the time and money involved rather than decreases it. http://www.iracing.com/track-technology/ http://www.iracing.com/track-technology/
- pizza234 13y ago
  The author's point is more abstract. Yes, you can certainly work out a few problems, scaling up the computations in some way. Some problems though, are inherently hard (eg. animations), and it's hard to imagine some breakthrough even in the next decade.
DanielBMarkham 13y ago
Takeaway: the problem isn't photorealism in games, although that's hard enough. The problem is actually having somebody create the zillions of points of detail required to have something to photorealistically render. So, seems like the technical challenge is to take a small portal, say 64x64 pixels, and then create a procedurally-based photorealistic (including damage, real-world illumination, and a damage model) engine to power it, right?
- ZenoArrow 13y ago
  > So, seems like the technical challenge is to take a small portal, say 64x64 pixels, and then create a procedurally-based photorealistic (including damage, real-world illumination, and a damage model) engine to power it, right? You may be interested in .kkrieger as a proof of concept for this type of engine. The most incredible part is the whole game is less than 100 KiB in size, but then we're talking about coders who are used to working with in 4KiB limits: http://en.wikipedia.org/wiki/.kkrieger http://en.wikipedia.org/wiki/.kkrieger Talking of the demo scene, you may be interested in this 4K demo, another great demonstration of what procedurally generated content can do: http://www.pouet.net/prod.php?which=52938 http://www.pouet.net/prod.php?which=52938
benched 13y ago
I am a long-time 3D graphicist, and have written numerous software and accelerated renderers. I am really struggling to see the point of this article (it seems trivial and tautological to me), although I'll be glad if it generates some discussion. This is an area where I feel very strongly that perception is reality, or at least so close to it as to make no difference. The game world is virtual, so the fact that say a rock is not made of zillions of atoms computing quantum chromodynamics doesn't come as a surprise to anybody. Well, that's a shortcut right there. Immediately, the game world is not arbitrarily "realistic". But I don't think that's what the word "photorealism" usually means. I know I've never used it that way. I use it to mean "feels subjectively real". By that definition, as the OP alludes to, today's most advanced game engines come very close. In my opinion, they come so close, that the rest is gravy. That's what will be coming in the next decade. More gravy, more cake, more icing. In the next year or so, somebody (Sony? Everquest Landmark?) will start the inevitable development toward Minecraft-style procedural/cellular automata based worlds, combined with increasingly photorealistic rendering.
- rntz 13y ago
  As I see it, the OP is frustrated that people mistake mere graphics (and in particular, realtime raytracing) as the gold standard for "realism" in games, when there are open problems both more challenging and more important that are still unsolved (truly dynamic behavior, scalable content creation). I don't know about industry, but in academic CS graphics is a huge field; by contrast, my impression the other problems mentioned are rather neglected (though certainly not unknown). However, graphics and games aren't my specialty; I would love to be mistaken.
  - ilaksh 13y ago
    No, actually he dismissed realtime raytracing out of hand on the false premise that it still requires manual artist texture creation. We have solutions to "truly dynamic behavior", its called a physics engine. Scalable content creation has solutions also, i.e. 3d scanning and procedural generation. Real time path tracing is where its at these days, not ray tracing anymore. All of this stuff can go in hardware and it will within a few years. Main thing holding it back is a lack of resources. I.e., you have billions of dollars being spent on tweaking hardware and software for cheaty approximations and manual artistic endeavours rather than on advancing the technologies I mentioned.
Geee 13y ago
I don't know if you guys know how fast path-tracing is becoming reality. It just needs a few more years (of GPU advancement) to get rid of the noise, watch: https://www.youtube.com/watch?v=aKqxonOrl4Q https://www.youtube.com/watch?v=aKqxonOrl4Q For art, next-generation photorealistic graphics will be 3D scanned and/or procedurally generated. Lots of work still (you can't scan cars or anything mechanical), but graphics could also be reused more often.
- ilaksh 13y ago
  Yeah this is exactly what I was going to say and I was even about to post a Brigade engine video. LOL. McClure kind of dismisses "ray tracing" offhand (and by implication/context, similar techniques like path tracing) with the premise that every texture would need to be manual developed by an artist. The answer to that is as you mention 3d scanning and procedural generation. There are quite a few path-tracing software (and even one or maybe a few hardware) efforts out there, and many 3d scanning companies. I believe that the main thing that is holding these technologies back now is just people not knowing that they are realistic technologies, which keeps them from being mainstream. But once things enter the mainstream consciousness of engineers, you get an order of magnitude increase in the number of people working on them, starting with some of the existing working ideas and you start getting much more practical and inexpensive solutions. I believe that within say 7 years Nvidia and ATI will either acquihire or build hardware themselves that makes real-time path-tracing, procedural generation, and real-time physics, convenient and efficient.
  - Geee 13y ago
    By the way, if you are interested in (realtime) ray tracing technologies, check out this forum: http://ompf2.com/ http://ompf2.com/ By procedural/scanned I meant mainly procedural that is based on 3D scanning (scans used for learning set), such as FaceGen http://www.facegen.com/ http://www.facegen.com/ Lots of other stuff could be made this way, too.
- JetSpiegel 13y ago
  You don't understand. The article is arguing in a different level. It says that reality is inherently artistically irrelevant, and this race to photorealism is a wild goose chase. You can't 3D scan Skyrim!
eslachance 13y ago
About the last point about generating content, I think it's extremely likely that this leads us down the path of Voxels and Procedural Generation. Both those techniques together would help us at least get a starting point for an environment that's generated from certain chosen parameters, and then the "least effort" becomes modifying that environment for your game needs. I mean, if you can have a procedurally generated jungle with a huge mountain in the middle and a river cutting through the jungle, all you need to do really is to cut through the mountain for your secret underground base, create a couple of roads, camps and checkpoints and you've got a better remake of FarCry 1 (yeah I'm extrapolating of course but you get the idea).
aw3c2 13y ago
Wasn't there something about people actually prefering non-photorealistic renderings more, at least in Google Earth?
bitwize 13y ago
And that's a big part of why video games suck these days. Creating a photorealistic or cinematic experience is extremely difficult and requires a huge amount of artistic assets, and even then the illusion only holds up under certain tight constraints. So game developers do as much as possible to enforce these constraints all throughout the game.
- lttlrck 13y ago
  Or it's because the games just suck and no amount of photorealism is going to cover the cracks.
  - wmeredith 13y ago
    100 times this. If you want to know how important photorealism is to getting your subject involved with the medium, go read a paperback copy of Lord of The Rings. It's pretty lo-fi.
    - lotyrin 13y ago
      And this is why I point to Dwarf Fortress as the state of the art in games.
      - darkmighty 13y ago
        I agree, it is a proof that content >> presentation, but there's no doubt a better interface and graphics wouldn't benefit DF greatly -- it currently has a huge cost barrier before the fun part, which most people just aren't willing to try and break.
      - coldpie 13y ago
        I would love to play a DF with a decent UI, and preferably with simple 2D graphics. And I'm an ASCII DCSS player. Are there any hacks out there that make the UI less memorization-focused? I know there are graphics hacks, but I was never able to get them to work...
mattgreenrocks 13y ago
I noticed the most recent crop of engines (thinking of Frostbite 3 here mostly) do a lot of cheating to run well. This is most apparent on the awful draw distance of terrain decoration. Draw-in is very apparent at even the highest graphics setting. Worse, despite games like Battlefield having beautiful art assets, it takes very little for the illusion of photorealism to be destroyed; it could be as simple as getting too close to a piece of tall grass and realizing it's actually just a sprite. Or noticing that a flag is always flapping one way, and smoke is drifting the opposite way. It's unfortunate, but the brain is really good at picking these inconsistencies out. More stylized games seem to age much better; TF2 still looks great.
olouv 13y ago
Reminded me this "What Is Real" trailer (https://www.youtube.com/watch?v=4dnw9dWISvw https://www.youtube.com/watch?v=4dnw9dWISvw), I welcome any trick that can help the player's immersion.
marcusmolchany 13y ago
I've always wondered, given all the issues presented in this article and the expertise required to solve them does $50-$60 dollars properly pay the teams working on these games? Not even just engineers, there are artists and even actors capturing motion of characters and authors writing up scripts it doesn't seem like $60 would cover everyone involved in creating a video game.
- SixSigma 13y ago
  You can buy LOTR on Blu-Ray for about $20.
beloch 13y ago
It's often been said that the difference between Steven Spielberg and Ed Wood is merely thousands of subtleties. Tiny things add up to either make or break created worlds. Clipping is one of my biggest pet-peeves with current video game engines. It really destroys the illusion when stuff that looks realistic and solid starts magically clipping through something else that looks realistic and solid. How about when the camera passes too close to something and you see the hollow insides? This is especially jarring when it happens to be a character's head and you see the backside of his/her eyeballs and teeth! Pick any AAA title randomly, make a character, and then look at that character closely while standing in his/her default start-of-the-game gear. The odds are good that you'll see dozens of little bits of his/her gear clipping into itself while that character is engaged in the default standing animation. It's telling that even excellent artists can't solve this problem for the one combination of gear and poses they know the player will see with 100% certainty. The next big step for computer game realism is going to be material physics. Simulated materials instead of textured polygons and bounding boxes. This is going to be computationally intensive, but the boost to realism is probably going to be far greater than we'll gain from ray-tracing.
- angersock 13y ago
  "Simulated materials instead of textured polygons and bounding boxes." What does this mean? Like, what do you envision this changing?
  - nuclear_eclipse 13y ago
    I would guess that it would imply that meshes are no longer static, and that they could bend, stretch, and fold based on their properties. In some ways, we already get this in modern games that try their best to simulate cloth in flags and such, but it's mostly still smoke and mirrors.
  - beloch 13y ago
    Think about how a game treats a block of stone right now. It's a hollow cube with textures pasted on it. It's bounding box is pretty easy to calculate, but a complex object that doesn't match it's own bounding box closely will still clip right through it. Now, imagine if we replaced that single block of stone with a matrix of tiny blocks. Also, imagine that we've made the bounding box of the complex object fit its geometry much more closely. If the same motion that caused clipping before is carried out, we'll detect it. It's going to take a lot more computation, but we will detect it. Not only will we detect collisions that we wouldn't have before, we'll be able to handle them in a variety of gameplay enhancing ways. If the stone is meant to be strong stuff, we make it's mass high and the bonds between sub-blocks strong so that the complex object will come to a jarring halt when it makes contact instead of passing into the stone. If the stone is meant to be softer, we dial down the bond strength between the sub-blocks a bit so that they can be broken. The object might dig in a bit and leave a noticeable gouge as some of the sub-blocks are separated from the main block. If the stone is meant to be more like dirt, we make the bond strength very low so that a blow will cause a chain-reaction that will cause the whole block to tumble apart. We can change the probability of blocks sticking together to simulate either wet, clumpy earth or dust that's ready to fly off in all directions when it's touched. Deforming the complex object is also an option. It could be similarly be composed of sub-blocks, but with tweakable levels of stickiness/stretchiness holding them together that would allow us to capture things like flexing or bending. In short, we really oversimplify the physical nature of objects in video games at present. This greatly reduces computation, but it makes the world seem unrealistic in how objects interact. By splitting things into smaller chunks that more closely resemble real matter we can start to simulate real materials, but at the cost of greater computation that isn't devoted directly towards making things pretty. Prying resources away from the graphics engine has long been something developers just don't want to do.
vacri 13y ago
One of the things missing from 'realistic' games is specular reflection and friends. Little subtleties like that help. One of my favourite moments in the realism wars was that phase where everyone had to have 'realistic' water, and I remember logging into Morrowind for the first time. "The water does look nice, shimmery and shiny"... but it drew attention to the problem that nothing else in the game shined at all, it was all dead flat.
radiorental 13y ago
I look at the loading scenes in GTAV and I wish the game looked like that, rather than the 'uncanny valley' that it is. Borderlands II is a great example of not trying to do something you cant, instead it has great art that you can lose yourself in.
drakaal 13y ago
It should be "the problem OF photorealism" not "WITH". We all enjoy photorealism, but it is hard to make. We have a problem OF 3 times 6. Not a problem with 3 times 16. Any how.... I worked on Flight sims in the mid 90's. We had a problem of Physics back then. So we didn't do real physics, we precalculated the physics based on a number of factors and loaded them in to tables, and if you were between point A and Point B in a table we took the weighted average of the two. It worked well enough. Animations today do the same thing. Anti-aliasing is now done based on point of contrast rather than the whole image. This works well enough, you notice the jaggies on white and black borders more than 45% grey next to 65% grey. This doesn't strike me as a well informed post. Others have pointed out that it is directed at those new to the space, fine, but if that's the audience then explain "uncanny valley" in there somewhere. Where you can't be "right" be far enough off people accept it as fake because it is. We see this all the time in Pixar. We also see it in racing games with "arcade physics".
ANTSANTS 13y ago
I prefer comics and cartoons to live-action movies. It's ironic, but somehow, a primitive smile or frown on a cartoon character's face feels more genuine and "real" to me than a professional actor pretending to be happy or sad. I can take a drawing as the unfiltered expression of the artist, but when I watch a live-action film and see real people carrying out the trappings of fiction, I have a hard time taking it seriously. It feels silly, kinda like how the so called "uncanny valley" effect makes bad CG creepy and not just unconvincing. I don't think that's a common opinion, given the lasting appeal of television and Hollywood, and the lack of respect comics receive in comparison, but in my mind, if a drawing feels more "real" than an actual human acting in front of a camera, then photorealistic CG doesn't stand a chance.
- Xixi 13y ago
  On the other hand it might just be a matter of getting used to it. Japanese live-action movies are a textbook definition of overplayed, and few years ago I couldn't stand watching them. It was making me physically uncomfortable. Fast forward a few years and I enjoy watching them...
z3phyr 13y ago
Can someone list the possible technical advancements in gamedev yet to happen? -> AI programmers are negative towards AI, saying gamers want 'fun' not Intelligence. -> Graphics programmers say, the graphics will be photorealastic soon. What will graphics programmers do after that? -> Gameplay, they are focusing more on creating the same style?

163 points by blackhole 13y ago

Minecraft is an interesting case study here, in that it deliberately throws away realistic graphics in favor of a comprehensively mutable environment. It's interesting how that seemed like such a totally unprecedented idea at the time.

danso 13y ago
I'm not sure where Minecraft came in the recent era of gaming...since it was built in 2009, that was well after games started being big on iOS, and some of those big games went for 8-bit as a way to maintain a modern sense of style and artistic quirk with the limited dev and system resources. Also, notch got some inspiration from Dwarf Fortress, which made an even more extreme sacrifice of graphics (the precursor to Dwarf Fortress was a 3D game that was too difficult to develop for). Either way, who could've guessed back in 2004ish that pixelated retro games (including FTL, TowerFall, and countless iOS hits) would be what gaming would look like 10 years later?
- lazugod 13y ago
  > Either way, who could've guessed back in 2004ish that pixelated retro games (including FTL, TowerFall, and countless iOS hits) would be what gaming would look like 10 years later? Flash games were probably the 2004-era precedent.
AJ007 13y ago
It wasn't unprecedented, just look at Dwarf Fortress. Tarn threw out all graphics in exchange for focusing on an incredibly complex and deeply layered game -- with an environment that is fully constructable and has both a geological and cultural history. Once the 3D accelerator become an item all gamers had, (perceptually) it appeared all major game develop shifted to producing "photorealistic" graphics. We ended up with an endless trail of games that basically produced the same thing. I wonder, what would happen if the budget of something like Call of Duty was spent on a game with Minecraft or even Ascii graphics? Perhaps a total disaster, but what could be produced if it was done really well?
- dasil003 13y ago
  The current state of Dwarf Fortress probably is within an order of magnitude of the number man hours in it as an early Call of Duty. The difference is that the work on Call of Duty is mostly art which is trivially parallelizable, whereas the work on Dwarf Fortress is the engine which is not parallelizable to anywhere near the same degree.
mnw21cam 13y ago
Once upon a time there was an open source game called "Cube" which was a first person shooter. It allowed in-game multi-player level editing, with all the occlusion and lighting recomputed on the spot. It was superseded by Sauerbraten, which improved the level structure to allow multiple floors, but didn't run as quickly due to the calculations being a bit harder.
andrey-p 13y ago
A game with very stylised graphics will always win over a game with very realistic graphics as far as I'm concerned. If you've got a game with unassuming graphics it's a lot easier to focus on the gameplay. Alternatively, stylising things _just right_ can get you a lot further in creating a specific atmosphere - games like Limbo work because everything fits together so perfectly.

coldcode 13y ago

Having worked on a FPS MMO, I can't agree with this enough. People always complained it didn't look real enough. I always wanted to ask them if would mind buying 1,000 video cards up front and then wait 10 years buying 1000 new ones every year until I got it right. I figured by then I could retire and duck the problem. Reality is ridiculously hard to do right. Now try doing it over the internet with a half second ping time and you realize that not only reality bites, but physics bites harder.

chiph 13y ago
While WoW looks the way it does partly because of it's 10 year old origins, it's artwork also communicates to the player: "It's just a game". And once you're far enough away from an imitation of reality, the brain stops trying to fit it into it's model of the world and complaining when it can't.

zeidrich 13y ago

Just simulate the entire universe. It requires a computer the size of the universe. There is one freely available, however, you will have to share it with a few other people. It's barely noticeable though because it's highly parallelizable.

guizzy 13y ago
It's rather poorly documented, though.
- mikeash 13y ago
  And extremely buggy.
  - Houshalter 13y ago
    It's pretty bad. The other day an entire plane just disappeared. And I have yet to see someone respawn properly.
    - mikeash 13y ago
      I saw the greatest fake release notes entry the other day: "Fixed bug where matter would sometimes spontaneously become self aware."
    - z3phyr 13y ago
      Its a bug, and the 'AI' is trying to fix it.
- resu_nimda 13y ago
  Proprietary assholes. Maybe if they just made the source available we wouldn't have to spend countless man-millennia trying to reverse-engineer this thing.
- mamcx 13y ago
  No poorly, is very well documented. The problem is that we run inside the exe, no in the source code!

robmcm 13y ago

This guy isn't thinking outside the box. He's trying to solve the problem using techniques from the past. Obviously the solution isn't to manually make levels, it would be to automate them. Where are EA on flying scanning drones around football stadia in order to save artists thousands of man hours? They already use these techniques to scan actors faces, I can imagine someone from 10 years ago predicting it would be impossible to script all the movements of lips and eyelids to produce realistic characters. Disclaimer: I have no idea what I am talking about, but thought my off the cuff theory would have more weight if I wrote like I did.

msandford 13y ago
That kind of tech is definitely coming, but the game studios aren't developing it. They'll buy it once it comes out though. Game studios aren't VCs. A VC buys pieces of HIGHLY speculative companies trying to maximize his/her odds of a 1000:1 exit. Game studios fund a small portfolio of games and try and make a profit on every single one. The culture in a studio is so completely different it's very unlikely that anyone high enough up at a game studio to have the budget and other resources to try and develop it in-house won't have the risk appetite to actually do so. And the people who have the risk appetite won't have the budget.
chasing 13y ago
> Disclaimer: I have no idea what I am talking about, but thought my off the cuff theory would have more weight if I wrote like I did. Ha. I like the cut of your jib.
Gracana 13y ago
iRacing scans racetracks, but that's for physical accuracy; it likely increases the time and money involved rather than decreases it. http://www.iracing.com/track-technology/ http://www.iracing.com/track-technology/
pizza234 13y ago
The author's point is more abstract. Yes, you can certainly work out a few problems, scaling up the computations in some way. Some problems though, are inherently hard (eg. animations), and it's hard to imagine some breakthrough even in the next decade.

DanielBMarkham 13y ago

Takeaway: the problem isn't photorealism in games, although that's hard enough. The problem is actually having somebody create the zillions of points of detail required to have something to photorealistically render. So, seems like the technical challenge is to take a small portal, say 64x64 pixels, and then create a procedurally-based photorealistic (including damage, real-world illumination, and a damage model) engine to power it, right?

ZenoArrow 13y ago
> So, seems like the technical challenge is to take a small portal, say 64x64 pixels, and then create a procedurally-based photorealistic (including damage, real-world illumination, and a damage model) engine to power it, right? You may be interested in .kkrieger as a proof of concept for this type of engine. The most incredible part is the whole game is less than 100 KiB in size, but then we're talking about coders who are used to working with in 4KiB limits: http://en.wikipedia.org/wiki/.kkrieger http://en.wikipedia.org/wiki/.kkrieger Talking of the demo scene, you may be interested in this 4K demo, another great demonstration of what procedurally generated content can do: http://www.pouet.net/prod.php?which=52938 http://www.pouet.net/prod.php?which=52938

benched 13y ago

I am a long-time 3D graphicist, and have written numerous software and accelerated renderers. I am really struggling to see the point of this article (it seems trivial and tautological to me), although I'll be glad if it generates some discussion. This is an area where I feel very strongly that perception is reality, or at least so close to it as to make no difference. The game world is virtual, so the fact that say a rock is not made of zillions of atoms computing quantum chromodynamics doesn't come as a surprise to anybody. Well, that's a shortcut right there. Immediately, the game world is not arbitrarily "realistic". But I don't think that's what the word "photorealism" usually means. I know I've never used it that way. I use it to mean "feels subjectively real". By that definition, as the OP alludes to, today's most advanced game engines come very close. In my opinion, they come so close, that the rest is gravy. That's what will be coming in the next decade. More gravy, more cake, more icing. In the next year or so, somebody (Sony? Everquest Landmark?) will start the inevitable development toward Minecraft-style procedural/cellular automata based worlds, combined with increasingly photorealistic rendering.

rntz 13y ago
As I see it, the OP is frustrated that people mistake mere graphics (and in particular, realtime raytracing) as the gold standard for "realism" in games, when there are open problems both more challenging and more important that are still unsolved (truly dynamic behavior, scalable content creation). I don't know about industry, but in academic CS graphics is a huge field; by contrast, my impression the other problems mentioned are rather neglected (though certainly not unknown). However, graphics and games aren't my specialty; I would love to be mistaken.
- ilaksh 13y ago
  No, actually he dismissed realtime raytracing out of hand on the false premise that it still requires manual artist texture creation. We have solutions to "truly dynamic behavior", its called a physics engine. Scalable content creation has solutions also, i.e. 3d scanning and procedural generation. Real time path tracing is where its at these days, not ray tracing anymore. All of this stuff can go in hardware and it will within a few years. Main thing holding it back is a lack of resources. I.e., you have billions of dollars being spent on tweaking hardware and software for cheaty approximations and manual artistic endeavours rather than on advancing the technologies I mentioned.

Geee 13y ago

I don't know if you guys know how fast path-tracing is becoming reality. It just needs a few more years (of GPU advancement) to get rid of the noise, watch: https://www.youtube.com/watch?v=aKqxonOrl4Q https://www.youtube.com/watch?v=aKqxonOrl4Q For art, next-generation photorealistic graphics will be 3D scanned and/or procedurally generated. Lots of work still (you can't scan cars or anything mechanical), but graphics could also be reused more often.

ilaksh 13y ago
Yeah this is exactly what I was going to say and I was even about to post a Brigade engine video. LOL. McClure kind of dismisses "ray tracing" offhand (and by implication/context, similar techniques like path tracing) with the premise that every texture would need to be manual developed by an artist. The answer to that is as you mention 3d scanning and procedural generation. There are quite a few path-tracing software (and even one or maybe a few hardware) efforts out there, and many 3d scanning companies. I believe that the main thing that is holding these technologies back now is just people not knowing that they are realistic technologies, which keeps them from being mainstream. But once things enter the mainstream consciousness of engineers, you get an order of magnitude increase in the number of people working on them, starting with some of the existing working ideas and you start getting much more practical and inexpensive solutions. I believe that within say 7 years Nvidia and ATI will either acquihire or build hardware themselves that makes real-time path-tracing, procedural generation, and real-time physics, convenient and efficient.
- Geee 13y ago
  By the way, if you are interested in (realtime) ray tracing technologies, check out this forum: http://ompf2.com/ http://ompf2.com/ By procedural/scanned I meant mainly procedural that is based on 3D scanning (scans used for learning set), such as FaceGen http://www.facegen.com/ http://www.facegen.com/ Lots of other stuff could be made this way, too.
JetSpiegel 13y ago
You don't understand. The article is arguing in a different level. It says that reality is inherently artistically irrelevant, and this race to photorealism is a wild goose chase. You can't 3D scan Skyrim!

eslachance 13y ago

About the last point about generating content, I think it's extremely likely that this leads us down the path of Voxels and Procedural Generation. Both those techniques together would help us at least get a starting point for an environment that's generated from certain chosen parameters, and then the "least effort" becomes modifying that environment for your game needs. I mean, if you can have a procedurally generated jungle with a huge mountain in the middle and a river cutting through the jungle, all you need to do really is to cut through the mountain for your secret underground base, create a couple of roads, camps and checkpoints and you've got a better remake of FarCry 1 (yeah I'm extrapolating of course but you get the idea).

aw3c2 13y ago

Wasn't there something about people actually prefering non-photorealistic renderings more, at least in Google Earth?

bitwize 13y ago

And that's a big part of why video games suck these days. Creating a photorealistic or cinematic experience is extremely difficult and requires a huge amount of artistic assets, and even then the illusion only holds up under certain tight constraints. So game developers do as much as possible to enforce these constraints all throughout the game.

lttlrck 13y ago
Or it's because the games just suck and no amount of photorealism is going to cover the cracks.
- wmeredith 13y ago
  100 times this. If you want to know how important photorealism is to getting your subject involved with the medium, go read a paperback copy of Lord of The Rings. It's pretty lo-fi.
  - lotyrin 13y ago
    And this is why I point to Dwarf Fortress as the state of the art in games.
    - darkmighty 13y ago
      I agree, it is a proof that content >> presentation, but there's no doubt a better interface and graphics wouldn't benefit DF greatly -- it currently has a huge cost barrier before the fun part, which most people just aren't willing to try and break.
    - coldpie 13y ago
      I would love to play a DF with a decent UI, and preferably with simple 2D graphics. And I'm an ASCII DCSS player. Are there any hacks out there that make the UI less memorization-focused? I know there are graphics hacks, but I was never able to get them to work...

mattgreenrocks 13y ago

I noticed the most recent crop of engines (thinking of Frostbite 3 here mostly) do a lot of cheating to run well. This is most apparent on the awful draw distance of terrain decoration. Draw-in is very apparent at even the highest graphics setting. Worse, despite games like Battlefield having beautiful art assets, it takes very little for the illusion of photorealism to be destroyed; it could be as simple as getting too close to a piece of tall grass and realizing it's actually just a sprite. Or noticing that a flag is always flapping one way, and smoke is drifting the opposite way. It's unfortunate, but the brain is really good at picking these inconsistencies out. More stylized games seem to age much better; TF2 still looks great.

olouv 13y ago

Reminded me this "What Is Real" trailer (https://www.youtube.com/watch?v=4dnw9dWISvw https://www.youtube.com/watch?v=4dnw9dWISvw), I welcome any trick that can help the player's immersion.

marcusmolchany 13y ago

I've always wondered, given all the issues presented in this article and the expertise required to solve them does $50-$60 dollars properly pay the teams working on these games? Not even just engineers, there are artists and even actors capturing motion of characters and authors writing up scripts it doesn't seem like $60 would cover everyone involved in creating a video game.

SixSigma 13y ago
You can buy LOTR on Blu-Ray for about $20.

beloch 13y ago

It's often been said that the difference between Steven Spielberg and Ed Wood is merely thousands of subtleties. Tiny things add up to either make or break created worlds. Clipping is one of my biggest pet-peeves with current video game engines. It really destroys the illusion when stuff that looks realistic and solid starts magically clipping through something else that looks realistic and solid. How about when the camera passes too close to something and you see the hollow insides? This is especially jarring when it happens to be a character's head and you see the backside of his/her eyeballs and teeth! Pick any AAA title randomly, make a character, and then look at that character closely while standing in his/her default start-of-the-game gear. The odds are good that you'll see dozens of little bits of his/her gear clipping into itself while that character is engaged in the default standing animation. It's telling that even excellent artists can't solve this problem for the one combination of gear and poses they know the player will see with 100% certainty. The next big step for computer game realism is going to be material physics. Simulated materials instead of textured polygons and bounding boxes. This is going to be computationally intensive, but the boost to realism is probably going to be far greater than we'll gain from ray-tracing.

angersock 13y ago
"Simulated materials instead of textured polygons and bounding boxes." What does this mean? Like, what do you envision this changing?
- nuclear_eclipse 13y ago
  I would guess that it would imply that meshes are no longer static, and that they could bend, stretch, and fold based on their properties. In some ways, we already get this in modern games that try their best to simulate cloth in flags and such, but it's mostly still smoke and mirrors.
- beloch 13y ago
  Think about how a game treats a block of stone right now. It's a hollow cube with textures pasted on it. It's bounding box is pretty easy to calculate, but a complex object that doesn't match it's own bounding box closely will still clip right through it. Now, imagine if we replaced that single block of stone with a matrix of tiny blocks. Also, imagine that we've made the bounding box of the complex object fit its geometry much more closely. If the same motion that caused clipping before is carried out, we'll detect it. It's going to take a lot more computation, but we will detect it. Not only will we detect collisions that we wouldn't have before, we'll be able to handle them in a variety of gameplay enhancing ways. If the stone is meant to be strong stuff, we make it's mass high and the bonds between sub-blocks strong so that the complex object will come to a jarring halt when it makes contact instead of passing into the stone. If the stone is meant to be softer, we dial down the bond strength between the sub-blocks a bit so that they can be broken. The object might dig in a bit and leave a noticeable gouge as some of the sub-blocks are separated from the main block. If the stone is meant to be more like dirt, we make the bond strength very low so that a blow will cause a chain-reaction that will cause the whole block to tumble apart. We can change the probability of blocks sticking together to simulate either wet, clumpy earth or dust that's ready to fly off in all directions when it's touched. Deforming the complex object is also an option. It could be similarly be composed of sub-blocks, but with tweakable levels of stickiness/stretchiness holding them together that would allow us to capture things like flexing or bending. In short, we really oversimplify the physical nature of objects in video games at present. This greatly reduces computation, but it makes the world seem unrealistic in how objects interact. By splitting things into smaller chunks that more closely resemble real matter we can start to simulate real materials, but at the cost of greater computation that isn't devoted directly towards making things pretty. Prying resources away from the graphics engine has long been something developers just don't want to do.

vacri 13y ago

One of the things missing from 'realistic' games is specular reflection and friends. Little subtleties like that help. One of my favourite moments in the realism wars was that phase where everyone had to have 'realistic' water, and I remember logging into Morrowind for the first time. "The water does look nice, shimmery and shiny"... but it drew attention to the problem that nothing else in the game shined at all, it was all dead flat.

radiorental 13y ago

I look at the loading scenes in GTAV and I wish the game looked like that, rather than the 'uncanny valley' that it is. Borderlands II is a great example of not trying to do something you cant, instead it has great art that you can lose yourself in.

drakaal 13y ago

It should be "the problem OF photorealism" not "WITH". We all enjoy photorealism, but it is hard to make. We have a problem OF 3 times 6. Not a problem with 3 times 16. Any how.... I worked on Flight sims in the mid 90's. We had a problem of Physics back then. So we didn't do real physics, we precalculated the physics based on a number of factors and loaded them in to tables, and if you were between point A and Point B in a table we took the weighted average of the two. It worked well enough. Animations today do the same thing. Anti-aliasing is now done based on point of contrast rather than the whole image. This works well enough, you notice the jaggies on white and black borders more than 45% grey next to 65% grey. This doesn't strike me as a well informed post. Others have pointed out that it is directed at those new to the space, fine, but if that's the audience then explain "uncanny valley" in there somewhere. Where you can't be "right" be far enough off people accept it as fake because it is. We see this all the time in Pixar. We also see it in racing games with "arcade physics".

ANTSANTS 13y ago

I prefer comics and cartoons to live-action movies. It's ironic, but somehow, a primitive smile or frown on a cartoon character's face feels more genuine and "real" to me than a professional actor pretending to be happy or sad. I can take a drawing as the unfiltered expression of the artist, but when I watch a live-action film and see real people carrying out the trappings of fiction, I have a hard time taking it seriously. It feels silly, kinda like how the so called "uncanny valley" effect makes bad CG creepy and not just unconvincing. I don't think that's a common opinion, given the lasting appeal of television and Hollywood, and the lack of respect comics receive in comparison, but in my mind, if a drawing feels more "real" than an actual human acting in front of a camera, then photorealistic CG doesn't stand a chance.

Xixi 13y ago
On the other hand it might just be a matter of getting used to it. Japanese live-action movies are a textbook definition of overplayed, and few years ago I couldn't stand watching them. It was making me physically uncomfortable. Fast forward a few years and I enjoy watching them...

z3phyr 13y ago

Can someone list the possible technical advancements in gamedev yet to happen? -> AI programmers are negative towards AI, saying gamers want 'fun' not Intelligence. -> Graphics programmers say, the graphics will be photorealastic soon. What will graphics programmers do after that? -> Gameplay, they are focusing more on creating the same style?