6 ms·

It’s even harder to convince other people. Once you realize that we do not understand how to make graphics look real, the pattern appears everywhere. Yet no one

It’s even harder to convince other people. Once you realize that we do not understand how to make graphics look real, the pattern appears everywhere. Yet no one will acknowledge it. Graphics — even movie quality graphics — don’t look anything like what a camera produces. The sole exception is when you’re mixing in real video with CG elements. But try to synthesize an entire frame from scratch, then put it next to a video camera’s output, and people can tell the difference. Also, screenshots are misleading. You have to test video, not screencaps. Video is processed differently by the brain. 10 out of 10 times, all the graphics engineers come out of the woodwork going “but actually we do know how! It’s a matter of using so and so calculations and measuring BDRF and” none of those equations work.

pdelbarba 6y ago
I think it depends on context. There are a lot of sim games where the environment is very controlled and forums are littered with people who can't tell whether the occasional picture is real or in-game. A forest is hard to render accurately but a plane in flight is pretty trivial
- i-am-curious 6y ago
  Forests have pretty much been solved. Look at games like Shadow of the Tomb Raider, forest looks amazing. The difficulty now is limited to finer details, like hair. Hair is still pretty much unsolved.
  - sillysaurusx 6y ago
    See? It’s almost deterministic. People really can’t accept that we don’t know how to do something. Plop a nature video next to your forest rendering and it’ll become apparent just how unsolved trees are. And everything else, for that matter. The precise claim is this: viewers should be able to identify a rendered video no better than random chance. If you conduct this experiment, you’ll see that real videos from actual video cameras wipe the floor.
    - EForEndeavour 6y ago
      > If you conduct this experiment, you’ll see that real videos from actual video cameras wipe the floor. To be fair: have you run such an experiment yourself, or are you just assuming that this conclusion will always result?
    - mrob 6y ago
      The motion blur will probably give it away. Accurate video motion blur is computationally expensive but conceptually simple. Just render at about 100 times your target frame rate and average batches of frames together in linear colorspace. You can speed this up by rendering at a lower frame rate (e.g. 10 times your target frame rate), estimating motion, and blurring along the motion vectors before averaging the frames. You can further speed it up by using an adaptive frame rate depending on motion speed and contrast. But a lot of rendered video doesn't even try. Look at a fast-moving bright point of light and you'll easily see the difference. (But note this is only replicating video, not reality. Truly realistic motion blur requires ultra-high displayed frame rates beyond the capabilities of current hardware.)
      - Sohcahtoa82 6y ago
        A lot of games fake the motion blur so badly. Racing games are often especially guilty. If you're driving at 200 mph, and there's a car next to you also going 200 mph, it shouldn't be blurry. Also, the length of the blur should not exceed the distance an object travels on your screen in 1 frame. In other words, if an object moves 30 pixels from one frame to the next, then the blurred image shouldn't be more than 30 pixels wide.
    - dahart 6y ago
      This feel like multiple logical fallacies. It seems funny to me to claim in advance that people are going to argue with something that was intentionally non-specific, vague, controversial, unproven, and at least partly wrong, and then when someone argues with you, claim you were proven right. It's easy to predict that someone will argue with something misleading or wrong, and that doesn't give your CG argument any credibility. What CG videos are you considering, what specifically have you looked at? Can you show some good faith examples of the best CG forests ever made, compared to some specific nature videos? Are you talking about attempts to match a nature video, and saying it's not possible regardless of what's in the shot? Are you looking at the best examples of CG forests lately? There are some CG full frame video examples of forests I don't believe people would reliably identify as CG, if they didn't know before hand and you left out the explosions & spaceships. https://www.youtube.com/watch?v=xOpuDhWzV1I https://www.youtube.com/watch?v=xOpuDhWzV1I
devit 6y ago
We definitely know how to make graphics look real - look at the unbiased renderers like LuxRender [https://luxcorerender.org/gallery/ https://luxcorerender.org/gallery/]. The problem is that we can't do that in 1/60 second on consumer-priced hardware, and also both scanning real objects and manually modelling are expensive.
- natdempk 6y ago
  Which of those images look real to you? To me, pretty much all of them look strictly like rendered images or have some aspect of them which gives away the fact that they are rendered.
  - CydeWeys 6y ago
    We need a blind test where you evaluate a set of images on whether you think each one is real or rendered.
    - natdempk 6y ago
      Heh, I agree! I'd love to try something like that if it exists.
    - bonoboTP 6y ago
      With a caveat: the real images should not be cherry picked to look as close to cgi as possible but the other way around. Real and render can look indistinguishable that way too, but we want real looking cgi, not carefully arranged cgi-looking reality. The biggest giveaway is the simplicity and pristine sterility of the rendered scenes, no mess, clutter, irregularity. Just look at photos that people haphazardly snap in the real world.pick those for a blind study and cgi is nowhere near. Pick carefully lit and arranged artificial scenes with heavy post-processing, like studio shoots or hotel brochures or car ads and the difference will be less obvious.
      - CydeWeys 6y ago
        I love this idea. Quick question for you, is this scene real or rendered? https://imgur.com/a/MpKMo6j https://imgur.com/a/MpKMo6j If you think the answer is obvious, then CGI definitely has more work to do. If not ... ??
ponker 6y ago
> Once you realize that we do not understand how to make graphics look real, the pattern appears everywhere. Yet no one will acknowledge it. Making graphics that look real is almost equivalent to the Turing Test, I think plenty of people are willing to acknowledge that it's unsolved.
- Terr_ 6y ago
  > Turing Test Amusingly relevant yet slightly off-topic: https://existentialcomics.com/comic/357 https://existentialcomics.com/comic/357
drran 6y ago
Unreal engine looks pretty convincing with proper camera setup: https://youtu.be/zKu1Y-LlfNQ?t=118 https://youtu.be/zKu1Y-LlfNQ?t=118 .
Scene_Cast2 6y ago
So I've dabbled a bit in graphics, and it feels like a content problem. Sure, you can have an artist pump out some models and textures. But for every level of detail increase, the artist must spend ~3x the time of the previous level. So for example, even making just a simple a dirt road look _really_ good and photo-realistic would involve much more time (a week? a month?) than one can reasonably spend on a commercial project (where you have bajillions of assets to worry about).
Terr_ 6y ago
> don’t look anything like what a camera produces The reverse-problem is a pet-peeve of mine: It seems many people have been accidentally brainwashed by Hollywood into thinking that film-camera effects are signs of "realism." So then the first-person game introduces something like lens-flares, and everyone goes: "OMG it's so realistic now", even though the truth is the exact opposite. If you were "really there" as Hero Protagonist, you wouldn't have camera-lenses for eyeballs except in a cyber-punk genre.
Polylactic_acid 6y ago
I didn't study this movie frame by frame, but I feel the new lion king movie was photo realistic. It seems like that given enough time, human and computer, you can achieve a great result.
dahart 6y ago
Your comment seems to be ignoring the fact that today’s graphics look better than yesterday’s, i.e. the main point of the comment you replied to. The monotonic trend toward realism has continued ever since computer graphics was invented 40ish / 50ish years ago. VFX and games from 10 and 20 years ago look much less realistic that today. In addition to stills, there is an increasing number of videos and film scenes that are getting harder to distinguish from shot on film. Are you saying the trend will stop? If so, why? > we do not understand how to make graphics look real > none of those equations work Why do you claim this, and what do you mean exactly? Surely you aren’t claiming we can’t make any graphics look real at all. Some people are getting pretty good at understanding how to make some graphics look real, even if we aren’t. The math for approximating materials and lighting is getting pretty good these days, and the equations have been changing over time. It’ll continue to improve, but it’s certainly not the main thing holding CG back today. IMO less realistic CG comes much more from low resolution or low fidelity sampling / geometry / textures / animation / input data in general. Most of that is due to lack of time & money. I’d say we understand how to make graphics look real in many cases, but not in all cases. We understand how to make good looking graphics, but how to do it quickly and efficiently enough to be practical and done with limited time and effort is an open problem.

pdelbarba 6y ago

I think it depends on context. There are a lot of sim games where the environment is very controlled and forums are littered with people who can't tell whether the occasional picture is real or in-game. A forest is hard to render accurately but a plane in flight is pretty trivial

i-am-curious 6y ago
Forests have pretty much been solved. Look at games like Shadow of the Tomb Raider, forest looks amazing. The difficulty now is limited to finer details, like hair. Hair is still pretty much unsolved.
- sillysaurusx 6y ago
  See? It’s almost deterministic. People really can’t accept that we don’t know how to do something. Plop a nature video next to your forest rendering and it’ll become apparent just how unsolved trees are. And everything else, for that matter. The precise claim is this: viewers should be able to identify a rendered video no better than random chance. If you conduct this experiment, you’ll see that real videos from actual video cameras wipe the floor.
  - EForEndeavour 6y ago
    > If you conduct this experiment, you’ll see that real videos from actual video cameras wipe the floor. To be fair: have you run such an experiment yourself, or are you just assuming that this conclusion will always result?
  - mrob 6y ago
    The motion blur will probably give it away. Accurate video motion blur is computationally expensive but conceptually simple. Just render at about 100 times your target frame rate and average batches of frames together in linear colorspace. You can speed this up by rendering at a lower frame rate (e.g. 10 times your target frame rate), estimating motion, and blurring along the motion vectors before averaging the frames. You can further speed it up by using an adaptive frame rate depending on motion speed and contrast. But a lot of rendered video doesn't even try. Look at a fast-moving bright point of light and you'll easily see the difference. (But note this is only replicating video, not reality. Truly realistic motion blur requires ultra-high displayed frame rates beyond the capabilities of current hardware.)
    - Sohcahtoa82 6y ago
      A lot of games fake the motion blur so badly. Racing games are often especially guilty. If you're driving at 200 mph, and there's a car next to you also going 200 mph, it shouldn't be blurry. Also, the length of the blur should not exceed the distance an object travels on your screen in 1 frame. In other words, if an object moves 30 pixels from one frame to the next, then the blurred image shouldn't be more than 30 pixels wide.
  - dahart 6y ago
    This feel like multiple logical fallacies. It seems funny to me to claim in advance that people are going to argue with something that was intentionally non-specific, vague, controversial, unproven, and at least partly wrong, and then when someone argues with you, claim you were proven right. It's easy to predict that someone will argue with something misleading or wrong, and that doesn't give your CG argument any credibility. What CG videos are you considering, what specifically have you looked at? Can you show some good faith examples of the best CG forests ever made, compared to some specific nature videos? Are you talking about attempts to match a nature video, and saying it's not possible regardless of what's in the shot? Are you looking at the best examples of CG forests lately? There are some CG full frame video examples of forests I don't believe people would reliably identify as CG, if they didn't know before hand and you left out the explosions & spaceships. https://www.youtube.com/watch?v=xOpuDhWzV1I https://www.youtube.com/watch?v=xOpuDhWzV1I

devit 6y ago

We definitely know how to make graphics look real - look at the unbiased renderers like LuxRender [https://luxcorerender.org/gallery/ https://luxcorerender.org/gallery/]. The problem is that we can't do that in 1/60 second on consumer-priced hardware, and also both scanning real objects and manually modelling are expensive.

natdempk 6y ago
Which of those images look real to you? To me, pretty much all of them look strictly like rendered images or have some aspect of them which gives away the fact that they are rendered.
- CydeWeys 6y ago
  We need a blind test where you evaluate a set of images on whether you think each one is real or rendered.
  - natdempk 6y ago
    Heh, I agree! I'd love to try something like that if it exists.
  - bonoboTP 6y ago
    With a caveat: the real images should not be cherry picked to look as close to cgi as possible but the other way around. Real and render can look indistinguishable that way too, but we want real looking cgi, not carefully arranged cgi-looking reality. The biggest giveaway is the simplicity and pristine sterility of the rendered scenes, no mess, clutter, irregularity. Just look at photos that people haphazardly snap in the real world.pick those for a blind study and cgi is nowhere near. Pick carefully lit and arranged artificial scenes with heavy post-processing, like studio shoots or hotel brochures or car ads and the difference will be less obvious.
    - CydeWeys 6y ago
      I love this idea. Quick question for you, is this scene real or rendered? https://imgur.com/a/MpKMo6j https://imgur.com/a/MpKMo6j If you think the answer is obvious, then CGI definitely has more work to do. If not ... ??

ponker 6y ago

> Once you realize that we do not understand how to make graphics look real, the pattern appears everywhere. Yet no one will acknowledge it. Making graphics that look real is almost equivalent to the Turing Test, I think plenty of people are willing to acknowledge that it's unsolved.

Terr_ 6y ago
> Turing Test Amusingly relevant yet slightly off-topic: https://existentialcomics.com/comic/357 https://existentialcomics.com/comic/357

drran 6y ago

Unreal engine looks pretty convincing with proper camera setup: https://youtu.be/zKu1Y-LlfNQ?t=118 https://youtu.be/zKu1Y-LlfNQ?t=118 .

Scene_Cast2 6y ago

So I've dabbled a bit in graphics, and it feels like a content problem. Sure, you can have an artist pump out some models and textures. But for every level of detail increase, the artist must spend ~3x the time of the previous level. So for example, even making just a simple a dirt road look _really_ good and photo-realistic would involve much more time (a week? a month?) than one can reasonably spend on a commercial project (where you have bajillions of assets to worry about).

Terr_ 6y ago

> don’t look anything like what a camera produces The reverse-problem is a pet-peeve of mine: It seems many people have been accidentally brainwashed by Hollywood into thinking that film-camera effects are signs of "realism." So then the first-person game introduces something like lens-flares, and everyone goes: "OMG it's so realistic now", even though the truth is the exact opposite. If you were "really there" as Hero Protagonist, you wouldn't have camera-lenses for eyeballs except in a cyber-punk genre.

Polylactic_acid 6y ago

I didn't study this movie frame by frame, but I feel the new lion king movie was photo realistic. It seems like that given enough time, human and computer, you can achieve a great result.

dahart 6y ago

Your comment seems to be ignoring the fact that today’s graphics look better than yesterday’s, i.e. the main point of the comment you replied to. The monotonic trend toward realism has continued ever since computer graphics was invented 40ish / 50ish years ago. VFX and games from 10 and 20 years ago look much less realistic that today. In addition to stills, there is an increasing number of videos and film scenes that are getting harder to distinguish from shot on film. Are you saying the trend will stop? If so, why? > we do not understand how to make graphics look real > none of those equations work Why do you claim this, and what do you mean exactly? Surely you aren’t claiming we can’t make any graphics look real at all. Some people are getting pretty good at understanding how to make some graphics look real, even if we aren’t. The math for approximating materials and lighting is getting pretty good these days, and the equations have been changing over time. It’ll continue to improve, but it’s certainly not the main thing holding CG back today. IMO less realistic CG comes much more from low resolution or low fidelity sampling / geometry / textures / animation / input data in general. Most of that is due to lack of time & money. I’d say we understand how to make graphics look real in many cases, but not in all cases. We understand how to make good looking graphics, but how to do it quickly and efficiently enough to be practical and done with limited time and effort is an open problem.