6 ms·

The number of comments here that are negative is just staggering to me. These images are within a hair's breadth of being indistinguishable from reality. It's

The number of comments here that are negative is just staggering to me. These images are within a hair's breadth of being indistinguishable from reality. It's an enormous leap. An enormous leap from the previous techniques and an enormous leap towards photorealistic realtime graphics. You don't want your games to look realistic? Go play pacman or outrun. You want to use path tracing instead? You have no idea how much cheaper this technique is. And on top of that the neural network is also fixing up the unrealistic texture work on all kinds of things in the scene. The first games to use this are going to be from small game studios with less investment in the status quo and they are going to blow the competition out of the water.

HellDunkel 5y ago
Maching learning is often oversold as a shortcut solution for some very hard problems and this is a perfect example. What you predict about small studios seems unlikely as ML is a beast to tame, needs highly specialized engineers and huge high qualty datasets for training.
- plutonorm 5y ago
  So were ajax crud web apps, 25 years ago.
  - zokier 5y ago
    And they still suck, 25 years later
bayindirh 5y ago
Actually, I'm pretty amazed by the results. This is indeed a giant leap. Also it's coming from Intel instead of AMD or Nvidia, which makes me hopeful about its wide availability. The only thing I didn't agree is that Rockstar is incapable of improving their engine further. Since GTA-III, this is somewhat a deliberate stylistic choice up to a certain point, like how Source engine is also opinionated about how its games look. Also, we need to keep in mind that this method is way more surgical than a simple post-processor. They've probably grafted a lot of tapping points to video driver to be able to implement this.
- formerly_proven 5y ago
  > The only thing I didn't agree is that Rockstar is incapable of improving their engine further. Since GTA-III, this is somewhat a deliberate stylistic choice up to a certain point, like how Source engine is also opinionated about how its games look. RDR2 is based on further development of the GTA5 engine and is arguably the best looking open world game at this time.
- garaetjjte 5y ago
  >They've probably grafted a lot of tapping points to video driver to be able to implement this. They didn't mention using anything else than output frames and G-buffers.
  - bayindirh 5y ago
    I'm not an expert in graphics programming but I don't think that accessing G-Buffers of an application and grafting your ML based post processor into the pipeline is exactly trivial. From my point of view it's only possible by either engine provided tapping points, or you tap the processing pipeline at the driver level directly. I'd love to be corrected if I'm wrong, BTW.
    - garaetjjte 5y ago
      You need to hook into game process, but it's nothing too hard. Either patching game binary in right place to funnel pointer to needed buffers to your DLL, or hooking something like D3D11 OMSetRenderTargets and doing some clever guessing which one of buffers this is. Doing this at driver side wouldn't make any difference, as driver doesn't have any extra semantic info what the buffer is used for anyway.
alkonaut 5y ago
> You don't want your games to look realistic? The creators aren’t necessarily aiming for photorealism. They want their games to look good enough and be entertaining. Adding more realism could make it look worse. Obviously if AI could solve the hard problems like global illumination that’s great. And I’m sure it’ll be possible to e.g easily add more variation to textures by using larger datasets. But changing the tonemapping to a colder palette, or just replacing the asphalt textures with smoother ones? The creators of GTA didn’t use the sunset tones in the game by some creative mistake or lack of computing power. They obviously went for non-photorealism deliberately! It’s a stylized view of California. It’s like the burger in the commercial doesn’t look like the real one - it looks a bit unrealistically perfect because that’s actually more attractive than the real thing. Realistic and photorealistic are tangentially related. The comparison here should perhaps be between frames of the game that already attempted to look like the cityscapes dataset. GTA obviously doesn’t have a ground texture that isn’t smooth because it wouldn’t be possible, or “too warm” palette because it’s too computationally expensive to have the bleak cityscapes tone mapping. A GTA frame rendered at its current frame cost, with textures the same size (but different) and a customized tone mapping, would look more photorealistic and make the difference to the cityscapes dataset much much smaller. The authors could likely have done this comparison since they probably have the ability to modify textures and shaders to set tonemapping. That would have separated out the differences that are only artistic from those that are lacking photorealism because of computational expense.
- deleted 5y ago
  [deleted]
- dplavery92 5y ago
  I am not very interested in whether or not the dataset that these authors chose is a pleasing dataset to use as a basis for a video game aesthetic. I am interested in in the technical accomplishments that this model was able to achieve in regards to preserving image features of semantic importance (e.g. writing, symbols, logos), not "hallucinating" new image features, and maintaining frame-to-frame stability in the output video product. These are very notable accomplishments for an adversarial model in this space, and the techniques they used to accomplish them are interesting. That Rockstar could have made their game more Cityscapes-like without this tech is missing the forest for the trees.
cycomanic 5y ago
I completely agree with you. Also those people who complain about the "dull european" look, did you scroll down and look at the images altered using the Mapillary Vistas set? The cool thing is that this gives game-designers an extremely powerful additional tool. You could for example easily go for a more retro-look by training with kodakcolor images ... And I don't buy the argument that GTA doesn't look more realistic because that's what the designers were going for. If this was a game with a clear "comic appearance" that might be true, however I would argue it looks like it does, because the designers did the maximum that they could achieve with their tools and within a certain budget.
- tallanvor 5y ago
  I almost clicked away from the the article because those Cityscapes transformations looked worse in my opinion - just washed out and dull, so I can understand why people didn't like the look. The Mapillary Vistas processing definitely was nicer, although it looked uneven to me - the road still looked washed out, which created an almost disturbing level of contrast with the greenery. What they've done is interesting, and I think with further refinement it is certainly a technique to consider.
  - Cthulhu_ 5y ago
    Definitely; I mean if they had a dataset of the cities and locations GTAV was based on I think it might look uncanny.
  - marcodiego 5y ago
    I think expectations influenced by culture play a role here. Since I live in a Brazilian metropolis (4mi population), the Cityscapes transformations looked more real. That is probably because they look more like what I'm used to see when I walk the streets. People from rich neighborhoods in the US will likely find the Mapillary Vistas more realistic since that is what they used to see. Impressive result nevertheless. Improving this with adversarial networks will probably generate images which are indistinguishable from reality in just a few years.
codeflo 5y ago
> You don't want your games to look realistic? Go play pacman or outrun. That's a silly dichotomy. We don't even want films to look completely realistic, that's why CGI exists in the first place. Imagine that you ran this over something like an Avengers movie, or even better, Blade Runner 2049. The AI would remove all the stylized/unrealistic details, and someone here would claim that this is clearly an improvement, and any criticism must come from luddites!
- saiya-jin 5y ago
  The primary reason why CGI is used is to save money. The secondary is to ease production of movies, which at the end saves money and time (which is money). Styling as done in some of them is an optional bonus if creators strive for some additional artistic message, but otherwise its all about money, money, money. You even contradict yourself - they tried to look Avengers as realistically as possible within the realm of comic book fairy tale, that's why they changed character of Thanos significantly well after introducing him (arguably a good choice). From cartoonish styling to realism. Nobody would take the epic battles so seriously if they all looked like Ready Player One.
  - klmadfejno 5y ago
    If I want to film a dragon, I'm probably going to choose CGI because it gets me the best dragon. It is coincidentally cheaper than building a puppet, or genetically engineering a lizard to look like a dragon, but cost isn't the motivating factor here.
- Hard_Space 5y ago
  The only director who laboriously follows through on 'I want it to not look like CGI' is Christopher Nolan. Others seem to favor stylized beauty over absolute CGI realism.
  - thenoxx 5y ago
    Uh, no, you just don't notice the absolute gobs of CGI used in tons of films because it is meant to look hyperrealistic. There are several, several youtube videos showcasing the incredible work of studios like Industrial Light and Magic creating most of the backdrops you thought were shot on location.
snorkel 5y ago
This is an enormous leap. Texturing and lighting are two very hard problems in CGI. Realistic lighting using raytracing requires a lot of radiosity passes, which means adjusting lighting of each surface to account for each ray of light bouncing from surface to surface to surface to surface like a 3D billiard ball, more bounces computed yields better results but there’s computational limits to the number of reflections per ray, and many parameters and formulas to choose from each with its own trade offs. Texturing and shader pipelines is another matrix of trade offs to get a good-enough result. Instead of all of that complexity, applying ML this way cleverly hops over the most difficult parts to a really impressive result. Some don’t want games to look this way. Nonetheless this technique can be applied to give games and movies a different look that would very hard to acheive by standard rendering pipelines, and it doesn’t have to look washed out like this example, that’s all based on the training data set and how much this ML lighting and texturing is blended into the scene, and that’s another creative choice. I can imagine filmmakers wanting to adjust CG lighting and texturing of a scene using a training data set from old TV shows to make a convincing reboot of a classic TV series, or new series done in that style.
amelius 5y ago
Photorealism shouldn't be the end goal. Watch a B/W television for a week and you won't be missing the colors. Photorealism is much like sugar in that it is addictive, but you won't miss the sugar if you didn't eat it for a while.
- Cthulhu_ 5y ago
  It's not the end goal, but it's definitely an aesthetic direction that a lot of games want to go towards. And not just video games, but films too. Most films already have photorealistic CGI, sometimes applying it without you even noticing it (e.g. backgrounds).
- anthk 5y ago
  Better, play IF games/roguelikes such as Slashem/DCSS, Gearhead, Anchorhead, Vicyous Cycles, Slouch over Bedlam and All Thing Devours. Then, every AAA game looks dumb, non-interactive and limited even with HQ graphics.
  - klmadfejno 5y ago
    And yet, the vast majority of DCSS players choose tiles over ASCII. Liking games without nice visuals doesn't mean people don't like nice visuals.
Cthulhu_ 5y ago
Could they do this process at video game speeds though? (25-60 times a second) I think this technique has a lot of potential, but will probably need a lot of work before it can be applied to video games. It might first show up as a filter for 'photo mode' in some games though, that would be interesting.
- cecilpl2 5y ago
  The first video on that page says they can run it at interactive framerates.
ksec 5y ago
>These images are within a hair's breadth of being indistinguishable from reality. This. I thought those were photos they took somewhere that looks exactly the same as GTA V, but then when I watch the video explaining it I was like holy crap. No Ray Tracing or whatever tech demo was ever this close to photo realistic. >You have no idea how much cheaper this technique is. Could someone explain this in greater details? I suppose you will still need to create some decent base image or graphics for the enhancement to work to its maximum?
- 0-_-0 5y ago
  I think the source game could be lower polygon count and texture resolution, but the neural network might be quite heavy...
choeger 5y ago
While I agree that the quality of this method is impressive, there are two caveats: 1. We do not know the set of pictures that the authors did not show us. So maybe this technique works really well only for a small fraction of cases. 2. Games (especially games that involve moving cars) tend to simplify mechanics so much that the moving end result will never look realistic, regardless how perfect the still picture is. So yes, I expect great outcomes from such a method, but I don't think its going to be a revolutionary silver bullet.
solotronics 5y ago
I thought the video was from real life at first until I listened to the explanation. It's amazing that this is possible in real time.
draw_down 5y ago
Whenever there is a sanctimonious comment like this at the top complaining about all the negative posts, I usually agree with the other posts. As in this case. I actually don’t think this looks any less “like a video game”, but its color is more washed-out, and darker. It seems like a net loss to me, sorry. Not for fancy theoretical reasons, I just think the GTA looks better in all the examples.
andrew-dc 5y ago
I think photo-realism in games is not merely a black and white, positive or negative. There are trade-offs, and I think it has to do with perception around aesthetics. (also, this is simply one guy's opinion, but I am curious if anyone else feels the same) For instance, in my view, I tend to prefer a stylized less hyper-realism in games. I think this is because the enhanced color, light, and brilliance is more aesthetically pleasant to look at, even though it is certainly more uncommon in the real world. I don't believe this is because saturated color, and brilliant light doesn't exist in reality, but these occurrences happen more rarely and in specific, often planned, situations and settings. And some people (trained or gifted photographers, for instance), have learned how to capture this look, or to take something rather ordinary, and capture it to appear extraordinary, much like the difference between your average person snapping a photo with a disposable camera, and Ansel Adams. The former will produce a photo that is far more common in appearance, and thus...boring, or uninteresting. So when it comes to games, if you simulate the "average look of reality" the game certainly appears more photo-realistic, but may also be less aesthetically interesting (to me, at least). This says absolutely nothing about the technology developed here, which is incredibly impressive. Nor do I think there is a right or wrong way create game visuals. I think that misses the point. However, I feel I'm not the only person who can be blown away with how realistic something can be made to look, yet ...also find it more aesthetically drab and boring to look at. And in the case of games, as I mentioned above, I tend to prefer to see things that are rare, interesting, and extra-ordinary, and thus less "realistic". Anyone else feel the same?
armageddon 5y ago
> You don't want your games to look realistic? Go play pacman or outrun. I can't just play GTA 5 as is?
kfarr 5y ago
Yeah the foliage is remarkably life-like and that is so hard and labor (art) intensive to do well. This is an amazing tool for indies without tons of resources.
malka 5y ago
> You don't want your games to look realistic? Go play pacman or outrun. I want my games to look cool. I dont wanna them to look like streetview or geoguessr. The result is impressive, but I'm not sure it would be good for video games. Maybe with some added filters it would be a lot better.
ryankrage77 5y ago
> These images are within a hair's breadth of being indistinguishable from reality Without a comparison to the original pictures, they do look very good, the roads in particular look more realistic, as though taken from a dashcam or medium quality phone camera. However the foliage just looks blurred. A lot of (interactive) CGI already looks more realistic than this. Take for example, this Australian scene made in Unreal Engine 4 - https://www.youtube.com/watch?v=Pg75bfkegtU https://www.youtube.com/watch?v=Pg75bfkegtU And that's not just pre-rendered, here's someone exploring the same scene interactively - https://www.youtube.com/watch?v=y87QL7IPE34 https://www.youtube.com/watch?v=y87QL7IPE34
- plutonorm 5y ago
  This is because its from scanned assets, I.e. there's no artist trying to imitate reality. The neural network above can take your artist created graphics and make them appear like they were scanned in. Both techniques achieve the same thing.
ricklamers 5y ago
I think you’re absolutely right. The innovations in deep learning make it such an exciting time to be an indie game developer. What used to take armies of content creators can now be a tiny team. Unity should be all over this kind of tech.
joshspankit 5y ago
Not only am I extremely impressed with the current realism they were able to accomplish, I suspect that others will figure out what it is about the final result and that will feed back in to even further realism in games (just like the breakthrough of subsurface scattering in digital skin). As well, I’m very excited for the scrappy devs who will turn this technique in to something wildly unexpected.

HellDunkel 5y ago

Maching learning is often oversold as a shortcut solution for some very hard problems and this is a perfect example. What you predict about small studios seems unlikely as ML is a beast to tame, needs highly specialized engineers and huge high qualty datasets for training.

plutonorm 5y ago
So were ajax crud web apps, 25 years ago.
- zokier 5y ago
  And they still suck, 25 years later

bayindirh 5y ago

Actually, I'm pretty amazed by the results. This is indeed a giant leap. Also it's coming from Intel instead of AMD or Nvidia, which makes me hopeful about its wide availability. The only thing I didn't agree is that Rockstar is incapable of improving their engine further. Since GTA-III, this is somewhat a deliberate stylistic choice up to a certain point, like how Source engine is also opinionated about how its games look. Also, we need to keep in mind that this method is way more surgical than a simple post-processor. They've probably grafted a lot of tapping points to video driver to be able to implement this.

formerly_proven 5y ago
> The only thing I didn't agree is that Rockstar is incapable of improving their engine further. Since GTA-III, this is somewhat a deliberate stylistic choice up to a certain point, like how Source engine is also opinionated about how its games look. RDR2 is based on further development of the GTA5 engine and is arguably the best looking open world game at this time.
garaetjjte 5y ago
>They've probably grafted a lot of tapping points to video driver to be able to implement this. They didn't mention using anything else than output frames and G-buffers.
- bayindirh 5y ago
  I'm not an expert in graphics programming but I don't think that accessing G-Buffers of an application and grafting your ML based post processor into the pipeline is exactly trivial. From my point of view it's only possible by either engine provided tapping points, or you tap the processing pipeline at the driver level directly. I'd love to be corrected if I'm wrong, BTW.
  - garaetjjte 5y ago
    You need to hook into game process, but it's nothing too hard. Either patching game binary in right place to funnel pointer to needed buffers to your DLL, or hooking something like D3D11 OMSetRenderTargets and doing some clever guessing which one of buffers this is. Doing this at driver side wouldn't make any difference, as driver doesn't have any extra semantic info what the buffer is used for anyway.

alkonaut 5y ago

> You don't want your games to look realistic? The creators aren’t necessarily aiming for photorealism. They want their games to look good enough and be entertaining. Adding more realism could make it look worse. Obviously if AI could solve the hard problems like global illumination that’s great. And I’m sure it’ll be possible to e.g easily add more variation to textures by using larger datasets. But changing the tonemapping to a colder palette, or just replacing the asphalt textures with smoother ones? The creators of GTA didn’t use the sunset tones in the game by some creative mistake or lack of computing power. They obviously went for non-photorealism deliberately! It’s a stylized view of California. It’s like the burger in the commercial doesn’t look like the real one - it looks a bit unrealistically perfect because that’s actually more attractive than the real thing. Realistic and photorealistic are tangentially related. The comparison here should perhaps be between frames of the game that already attempted to look like the cityscapes dataset. GTA obviously doesn’t have a ground texture that isn’t smooth because it wouldn’t be possible, or “too warm” palette because it’s too computationally expensive to have the bleak cityscapes tone mapping. A GTA frame rendered at its current frame cost, with textures the same size (but different) and a customized tone mapping, would look more photorealistic and make the difference to the cityscapes dataset much much smaller. The authors could likely have done this comparison since they probably have the ability to modify textures and shaders to set tonemapping. That would have separated out the differences that are only artistic from those that are lacking photorealism because of computational expense.

deleted 5y ago
[deleted]
dplavery92 5y ago
I am not very interested in whether or not the dataset that these authors chose is a pleasing dataset to use as a basis for a video game aesthetic. I am interested in in the technical accomplishments that this model was able to achieve in regards to preserving image features of semantic importance (e.g. writing, symbols, logos), not "hallucinating" new image features, and maintaining frame-to-frame stability in the output video product. These are very notable accomplishments for an adversarial model in this space, and the techniques they used to accomplish them are interesting. That Rockstar could have made their game more Cityscapes-like without this tech is missing the forest for the trees.

cycomanic 5y ago

I completely agree with you. Also those people who complain about the "dull european" look, did you scroll down and look at the images altered using the Mapillary Vistas set? The cool thing is that this gives game-designers an extremely powerful additional tool. You could for example easily go for a more retro-look by training with kodakcolor images ... And I don't buy the argument that GTA doesn't look more realistic because that's what the designers were going for. If this was a game with a clear "comic appearance" that might be true, however I would argue it looks like it does, because the designers did the maximum that they could achieve with their tools and within a certain budget.

tallanvor 5y ago
I almost clicked away from the the article because those Cityscapes transformations looked worse in my opinion - just washed out and dull, so I can understand why people didn't like the look. The Mapillary Vistas processing definitely was nicer, although it looked uneven to me - the road still looked washed out, which created an almost disturbing level of contrast with the greenery. What they've done is interesting, and I think with further refinement it is certainly a technique to consider.
- Cthulhu_ 5y ago
  Definitely; I mean if they had a dataset of the cities and locations GTAV was based on I think it might look uncanny.
- marcodiego 5y ago
  I think expectations influenced by culture play a role here. Since I live in a Brazilian metropolis (4mi population), the Cityscapes transformations looked more real. That is probably because they look more like what I'm used to see when I walk the streets. People from rich neighborhoods in the US will likely find the Mapillary Vistas more realistic since that is what they used to see. Impressive result nevertheless. Improving this with adversarial networks will probably generate images which are indistinguishable from reality in just a few years.

codeflo 5y ago

> You don't want your games to look realistic? Go play pacman or outrun. That's a silly dichotomy. We don't even want films to look completely realistic, that's why CGI exists in the first place. Imagine that you ran this over something like an Avengers movie, or even better, Blade Runner 2049. The AI would remove all the stylized/unrealistic details, and someone here would claim that this is clearly an improvement, and any criticism must come from luddites!

saiya-jin 5y ago
The primary reason why CGI is used is to save money. The secondary is to ease production of movies, which at the end saves money and time (which is money). Styling as done in some of them is an optional bonus if creators strive for some additional artistic message, but otherwise its all about money, money, money. You even contradict yourself - they tried to look Avengers as realistically as possible within the realm of comic book fairy tale, that's why they changed character of Thanos significantly well after introducing him (arguably a good choice). From cartoonish styling to realism. Nobody would take the epic battles so seriously if they all looked like Ready Player One.
- klmadfejno 5y ago
  If I want to film a dragon, I'm probably going to choose CGI because it gets me the best dragon. It is coincidentally cheaper than building a puppet, or genetically engineering a lizard to look like a dragon, but cost isn't the motivating factor here.
Hard_Space 5y ago
The only director who laboriously follows through on 'I want it to not look like CGI' is Christopher Nolan. Others seem to favor stylized beauty over absolute CGI realism.
- thenoxx 5y ago
  Uh, no, you just don't notice the absolute gobs of CGI used in tons of films because it is meant to look hyperrealistic. There are several, several youtube videos showcasing the incredible work of studios like Industrial Light and Magic creating most of the backdrops you thought were shot on location.

snorkel 5y ago

This is an enormous leap. Texturing and lighting are two very hard problems in CGI. Realistic lighting using raytracing requires a lot of radiosity passes, which means adjusting lighting of each surface to account for each ray of light bouncing from surface to surface to surface to surface like a 3D billiard ball, more bounces computed yields better results but there’s computational limits to the number of reflections per ray, and many parameters and formulas to choose from each with its own trade offs. Texturing and shader pipelines is another matrix of trade offs to get a good-enough result. Instead of all of that complexity, applying ML this way cleverly hops over the most difficult parts to a really impressive result. Some don’t want games to look this way. Nonetheless this technique can be applied to give games and movies a different look that would very hard to acheive by standard rendering pipelines, and it doesn’t have to look washed out like this example, that’s all based on the training data set and how much this ML lighting and texturing is blended into the scene, and that’s another creative choice. I can imagine filmmakers wanting to adjust CG lighting and texturing of a scene using a training data set from old TV shows to make a convincing reboot of a classic TV series, or new series done in that style.

amelius 5y ago

Photorealism shouldn't be the end goal. Watch a B/W television for a week and you won't be missing the colors. Photorealism is much like sugar in that it is addictive, but you won't miss the sugar if you didn't eat it for a while.

Cthulhu_ 5y ago
It's not the end goal, but it's definitely an aesthetic direction that a lot of games want to go towards. And not just video games, but films too. Most films already have photorealistic CGI, sometimes applying it without you even noticing it (e.g. backgrounds).
anthk 5y ago
Better, play IF games/roguelikes such as Slashem/DCSS, Gearhead, Anchorhead, Vicyous Cycles, Slouch over Bedlam and All Thing Devours. Then, every AAA game looks dumb, non-interactive and limited even with HQ graphics.
- klmadfejno 5y ago
  And yet, the vast majority of DCSS players choose tiles over ASCII. Liking games without nice visuals doesn't mean people don't like nice visuals.

Cthulhu_ 5y ago

Could they do this process at video game speeds though? (25-60 times a second) I think this technique has a lot of potential, but will probably need a lot of work before it can be applied to video games. It might first show up as a filter for 'photo mode' in some games though, that would be interesting.

cecilpl2 5y ago
The first video on that page says they can run it at interactive framerates.

ksec 5y ago

>These images are within a hair's breadth of being indistinguishable from reality. This. I thought those were photos they took somewhere that looks exactly the same as GTA V, but then when I watch the video explaining it I was like holy crap. No Ray Tracing or whatever tech demo was ever this close to photo realistic. >You have no idea how much cheaper this technique is. Could someone explain this in greater details? I suppose you will still need to create some decent base image or graphics for the enhancement to work to its maximum?

0-_-0 5y ago
I think the source game could be lower polygon count and texture resolution, but the neural network might be quite heavy...

choeger 5y ago

While I agree that the quality of this method is impressive, there are two caveats: 1. We do not know the set of pictures that the authors did not show us. So maybe this technique works really well only for a small fraction of cases. 2. Games (especially games that involve moving cars) tend to simplify mechanics so much that the moving end result will never look realistic, regardless how perfect the still picture is. So yes, I expect great outcomes from such a method, but I don't think its going to be a revolutionary silver bullet.

solotronics 5y ago

I thought the video was from real life at first until I listened to the explanation. It's amazing that this is possible in real time.

draw_down 5y ago

Whenever there is a sanctimonious comment like this at the top complaining about all the negative posts, I usually agree with the other posts. As in this case. I actually don’t think this looks any less “like a video game”, but its color is more washed-out, and darker. It seems like a net loss to me, sorry. Not for fancy theoretical reasons, I just think the GTA looks better in all the examples.

andrew-dc 5y ago

I think photo-realism in games is not merely a black and white, positive or negative. There are trade-offs, and I think it has to do with perception around aesthetics. (also, this is simply one guy's opinion, but I am curious if anyone else feels the same) For instance, in my view, I tend to prefer a stylized less hyper-realism in games. I think this is because the enhanced color, light, and brilliance is more aesthetically pleasant to look at, even though it is certainly more uncommon in the real world. I don't believe this is because saturated color, and brilliant light doesn't exist in reality, but these occurrences happen more rarely and in specific, often planned, situations and settings. And some people (trained or gifted photographers, for instance), have learned how to capture this look, or to take something rather ordinary, and capture it to appear extraordinary, much like the difference between your average person snapping a photo with a disposable camera, and Ansel Adams. The former will produce a photo that is far more common in appearance, and thus...boring, or uninteresting. So when it comes to games, if you simulate the "average look of reality" the game certainly appears more photo-realistic, but may also be less aesthetically interesting (to me, at least). This says absolutely nothing about the technology developed here, which is incredibly impressive. Nor do I think there is a right or wrong way create game visuals. I think that misses the point. However, I feel I'm not the only person who can be blown away with how realistic something can be made to look, yet ...also find it more aesthetically drab and boring to look at. And in the case of games, as I mentioned above, I tend to prefer to see things that are rare, interesting, and extra-ordinary, and thus less "realistic". Anyone else feel the same?

armageddon 5y ago

> You don't want your games to look realistic? Go play pacman or outrun. I can't just play GTA 5 as is?

kfarr 5y ago

Yeah the foliage is remarkably life-like and that is so hard and labor (art) intensive to do well. This is an amazing tool for indies without tons of resources.

malka 5y ago

> You don't want your games to look realistic? Go play pacman or outrun. I want my games to look cool. I dont wanna them to look like streetview or geoguessr. The result is impressive, but I'm not sure it would be good for video games. Maybe with some added filters it would be a lot better.

ryankrage77 5y ago

> These images are within a hair's breadth of being indistinguishable from reality Without a comparison to the original pictures, they do look very good, the roads in particular look more realistic, as though taken from a dashcam or medium quality phone camera. However the foliage just looks blurred. A lot of (interactive) CGI already looks more realistic than this. Take for example, this Australian scene made in Unreal Engine 4 - https://www.youtube.com/watch?v=Pg75bfkegtU https://www.youtube.com/watch?v=Pg75bfkegtU And that's not just pre-rendered, here's someone exploring the same scene interactively - https://www.youtube.com/watch?v=y87QL7IPE34 https://www.youtube.com/watch?v=y87QL7IPE34

plutonorm 5y ago
This is because its from scanned assets, I.e. there's no artist trying to imitate reality. The neural network above can take your artist created graphics and make them appear like they were scanned in. Both techniques achieve the same thing.

ricklamers 5y ago

I think you’re absolutely right. The innovations in deep learning make it such an exciting time to be an indie game developer. What used to take armies of content creators can now be a tiny team. Unity should be all over this kind of tech.

joshspankit 5y ago

Not only am I extremely impressed with the current realism they were able to accomplish, I suspect that others will figure out what it is about the final result and that will feed back in to even further realism in games (just like the breakthrough of subsurface scattering in digital skin). As well, I’m very excited for the scrappy devs who will turn this technique in to something wildly unexpected.