12 ms·

The Gates of Argonath in VR at 50 FPS on Unreal Engine

203 points by huhtenberg 4y ago

djmips 4y ago
50 FPS is too low. There's a reason that 90 FPS is the low bar for VR.
- contravariant 4y ago
  For something this big and far away you'd think there would be a way to cheat. It's not as if things are going to move massively if you move your head a bit, you should be able to precalculate things to an extent.
  - jffry 4y ago
    There is such a way to cheat, and it's pretty clever. Store a depth map and just approximate parallax while you're waiting for a real new frame. https://en.wikipedia.org/wiki/Asynchronous_reprojection https://en.wikipedia.org/wiki/Asynchronous_reprojection
  - jayd16 4y ago
    Most VR headsets/runtimes support this but it only goes so far. The closer things are the more occluded sections will need to be in-painted. The farther the scene is the more it might as well be a skybox at infinite distance.
- Toutouxc 4y ago
  90 FPS for redrawing the image according to the motion of your head, but the actual rendering of the scene can be much slower. Motion reprojection is basically the only way to make GPU-intensive games like MSFS playable in VR on < $1000 GPUs.
  - jayd16 4y ago
    Retrojection is great but it can't make up for 50 fps.
- rngname22 4y ago
  Nah. Not once you are experienced. Agreed as far as beginners.
kobaroko 4y ago
This looks so great. When I see this I'm somewhat sad that I don0t have computer powerful enough to try to do something like this by myself.
- yrgulation 4y ago
  VR capable computers are quite cheap. Cheaper than a low spec apple macbook pro running m1. For 2-3k you can get a beast of a machine running everything on real high settings.
  - alt227 4y ago
    Considring a brand new top of the line Nvidia GPU will currently cost you almost 2k, your prices feel a little ~4-5 years ago.
    - volkk 4y ago
      i don't think you need a 4090 for this. and even if you did get one, its 1600$. another 1000$ for good CPU and ram, etc. and you're well under 3k.
    - smoldesu 4y ago
      Right. You only need a card that's more powerful than the GTX 1060 for VR though, so the actual price-of-admission is more in the $250-400 price range.
      - charcircuit 4y ago
        The original comment wanted to try a demo like this. A RTX 3080 only runs it at 50 fps. A 1060 isn't good enough.
        smoldesu 4y ago
        The parent I'm responding to was interpreting "VR capable" as "top of the line" which is demonstrably false. You probably can't even get this scene in the first place, so it's kinda a moot point.
Tepix 4y ago
I think it will be commonplace to virtually walk around in movie scenes turned into 3d scenery by AI by the end of next year. I can see this turning into a real industry. Who wouldn't like to wander, fly and swim around Pandora, the world of Avatar? It will also lead to questions about the copyright of the resulting models, probably somewhat dependant on how automated this process is.
- avian 4y ago
  > It will also lead to questions about the copyright of the resulting models I'm sure there will be no questions there. The questions about the copyright of models like DALL-E, Stable Diffusion, etc. exist solely because it infringes on the rights of thousands of random artists all around the world. There is no one single entity with enough power to clearly oppose it so people making the models can get away with "questionable copyright", "legal gray area", etc. kind of talk. Try the same with a model trained on a single movie created by a multinational with infinite money and all this will be resolved and defined real quick and strongly worded cease & desist letters sent out by end of work day.
  - sandworm101 4y ago
    The day after movie-to-VR becomes easy to do, movie owners will monitize it, which will end the current free-for-all. Want to walk around The Shire in VR? You better have paid for the Hobbits Online subscription package.
    - zimpenfish 4y ago
      Sam: This is it. Frodo: What? Sam: If I take one more step, I'll be the farthest away from home I've ever been. Popup: And you can do that for just $29.99 when you buy The Hobbits Shire Expansion DLC!
    - SideburnsOfDoom 4y ago
      > The day after movie-to-VR becomes easy to do, movie owners will monitize it, which will end the current free-for-all. The movie making studios already own those 3d "digital assets" from their films. IDK if there is a real "free-for-all" right now, but the monetisation play has already been planned for.
peoplefromibiza 4y ago
As someone who never appreciated VR, this looks great. Now please do the Ishtar Gate https://en.wikipedia.org/wiki/Ishtar_Gate https://en.wikipedia.org/wiki/Ishtar_Gate
huhtenberg 4y ago
I think it's worth taking a step back and just marveling at what's now possible to do with an off-the-shelf software. Yeah, it's "just" 50 fps, but ... would you just look at it! 4 years ago there was an equally fascinating "Paris apartment" VR demo, but the movement was limited and the amount of work that went into appears to have been orders of magnitude more - https://www.benoitdereau.com/ https://www.benoitdereau.com/ Go back 10 years and seeing this back then would've been absolutely bananas. Makes you wonder what we'll have in 10 years from now.
- Daub 4y ago
  Off the shelf yes, but wow…. The requirements for Unreal are unreal. Recent versions handle real-time global illumination (Lumen) and dynamically render geometry to insane levels of detail (nanite). Both of these are groundbreaking achievements. But having had to install this in a bunch of university computers I can tell you that it comes at a cost. Interestingly, the most significant issue is the space these installations require. We have optimized our installation as much as we can, but we nonetheless had to introduce systems whereby local storage was wiped clean every two weeks.
  - solardev 4y ago
    Wait, disk space was the limiting factor, not GPUs?
    - smoldesu 4y ago
      I couldn't find the exact video, but Digital Foundry did a great breakdown on the performance bottlenecks in Unreal 5. They showed how disk speed directly influenced asset streaming, and it was pretty interesting to watch. Their HDD-speed drive choked constantly when running the UE5 demo, but a SATA SSD had enough bandwidth to handle the assets. So, it's kinda a weird spot to be. Not every scene was streaming 250mb/s of assets, but when it did the worse drives took a notable hit to performance.
      - adgjlsfhk1 4y ago
causi 4y ago
Am I alone in thinking most of this year's Unreal Engine demonstrations are just...not that great? Like, this is what graphics looked like on the Xbox One from nine years ago: https://halo.wiki.gallery/images/1/12/H5G-Waypoint-Swords_of_Sanghelios.jpg https://halo.wiki.gallery/images/1/12/H5G-Waypoint-Swords_of...
- kuschku 4y ago
  The quality of nanite heavily depends on your model. You can use photogrammetry scans or CAD exports directly, and nanite will make sure they'll look good. But if you just use existing videogame or fanmade assets (like here), the quality obviously suffers.
- bogwog 4y ago
  Nanite gives you high fidelity and Lumen gives you physically accurate lighting, but those things can't replace good art direction.
- ainiriand 4y ago
  Well that screen capture is taken from a cinematic scene if I am correct. UE demonstrations are real-time playable environments. Have you tested the Valley of the Ancients demo on your computer? You will see what I mean.
- sk0g 4y ago
  That screenshot does not look rendered in game, and no idea what the resolution is - at least not by an Xbox One. If current games are indistinguishable from single-frame renders that take minutes if not hours, I'd say that's a massive win! This looks more like the actual in-game rendering: http://cdn.themis-media.com/media/global/images/library/deriv/997/997367.jpg http://cdn.themis-media.com/media/global/images/library/deri... What stands out to me is the low resolution textures everywhere, obvious seams between terrain and assets (runtime virtual texturing resolves that), light seams which have odd seams, and the entire ambience being vaguely similar to how light acts, but not really - primarily the lack of bounce lighting affecting anything. With the move to HDR, the more binary lighting of previous games looks weird too now. Contrast that with the rich detail in lighting you can see in Fortnite's new update - https://i.redd.it/g2wqe10no44a1.jpg https://i.redd.it/g2wqe10no44a1.jpg Nanite, especially foliage, is something you have to experience for yourself though. Foliage pop-in not being so stark and alarming is huge for me, and the rich lighting capabilities improves immersion massively. Regarding assets, high polygon assets have been created for ages too, but were never actually brought in to games. You bake those details into your normal maps, but with Nanite you can just bring them into games, and get the full detail when the player is up close to them. When games can fully embrace Lumen and Nanite, the developer iteration improvement will be huge.
dncornholio 4y ago
This is super cool. Performance doesn't really matter, as long as it can keep 45 fps, the experience would be good enough with motion reprojection turned on.
Loic 4y ago
Thinking about it, it could be a very good tool to help cure fear of heights (or vertigo, not sure about the right term in English).
- time_to_smile 4y ago
  If you haven't played it, I highly recommend Richie's Plank Experience [0]. The main focus of the game is you take an elevator to the top of a building, there is a plank an you jump off it. I've found depending on the age of the participant the experience ranges from light fun to harrowing encounter with our own fears of death. I've seen people that, if it's their first time in VR, can take nearly an hour to finally do it. 0. https://www.oculus.com/experiences/quest/1642239225880682/ https://www.oculus.com/experiences/quest/1642239225880682/
- pwdisswordfish0 4y ago
  I sometimes fear I shouldn’t play driving games too much, so that I don’t get desensitised against those intrusive thoughts I sometimes get on the road. I wonder how irrational that actually is, because it feels like pretty much the same thing as your idea. I mean it doesn’t seem far fetched that a very realistic VR driving simulation with subtly “easier” physics might make you a worse driver in reality. More closely related to the fear of heights, we also jump from greater heights in video games than we would in real life, and much more frequently. I practically never have a reason to jump down from anywhere, so I could imagine making some wrong estimations when, after years of doing it in VR, I’m somehow faced with a real-life situation.
zokier 4y ago
One thing that sticks out like sore thumb are those relatively low-res rock textures, especially close-up. Would be neat to see some procedural generation there. For reference here is the scene from the movie https://www.youtube.com/watch?v=EN7kG7Ui7Bg https://www.youtube.com/watch?v=EN7kG7Ui7Bg
norwalkbear 4y ago
Whoa that's amazing.

203 points by huhtenberg 4y ago

djmips 4y ago

50 FPS is too low. There's a reason that 90 FPS is the low bar for VR.

contravariant 4y ago
For something this big and far away you'd think there would be a way to cheat. It's not as if things are going to move massively if you move your head a bit, you should be able to precalculate things to an extent.
- jffry 4y ago
  There is such a way to cheat, and it's pretty clever. Store a depth map and just approximate parallax while you're waiting for a real new frame. https://en.wikipedia.org/wiki/Asynchronous_reprojection https://en.wikipedia.org/wiki/Asynchronous_reprojection
- jayd16 4y ago
  Most VR headsets/runtimes support this but it only goes so far. The closer things are the more occluded sections will need to be in-painted. The farther the scene is the more it might as well be a skybox at infinite distance.
Toutouxc 4y ago
90 FPS for redrawing the image according to the motion of your head, but the actual rendering of the scene can be much slower. Motion reprojection is basically the only way to make GPU-intensive games like MSFS playable in VR on < $1000 GPUs.
- jayd16 4y ago
  Retrojection is great but it can't make up for 50 fps.
rngname22 4y ago
Nah. Not once you are experienced. Agreed as far as beginners.

kobaroko 4y ago

This looks so great. When I see this I'm somewhat sad that I don0t have computer powerful enough to try to do something like this by myself.

yrgulation 4y ago
VR capable computers are quite cheap. Cheaper than a low spec apple macbook pro running m1. For 2-3k you can get a beast of a machine running everything on real high settings.
- alt227 4y ago
  Considring a brand new top of the line Nvidia GPU will currently cost you almost 2k, your prices feel a little ~4-5 years ago.
  - volkk 4y ago
    i don't think you need a 4090 for this. and even if you did get one, its 1600$. another 1000$ for good CPU and ram, etc. and you're well under 3k.
  - smoldesu 4y ago
    Right. You only need a card that's more powerful than the GTX 1060 for VR though, so the actual price-of-admission is more in the $250-400 price range.
    - charcircuit 4y ago
      The original comment wanted to try a demo like this. A RTX 3080 only runs it at 50 fps. A 1060 isn't good enough.
      - smoldesu 4y ago
        The parent I'm responding to was interpreting "VR capable" as "top of the line" which is demonstrably false. You probably can't even get this scene in the first place, so it's kinda a moot point.

Tepix 4y ago

I think it will be commonplace to virtually walk around in movie scenes turned into 3d scenery by AI by the end of next year. I can see this turning into a real industry. Who wouldn't like to wander, fly and swim around Pandora, the world of Avatar? It will also lead to questions about the copyright of the resulting models, probably somewhat dependant on how automated this process is.

avian 4y ago
> It will also lead to questions about the copyright of the resulting models I'm sure there will be no questions there. The questions about the copyright of models like DALL-E, Stable Diffusion, etc. exist solely because it infringes on the rights of thousands of random artists all around the world. There is no one single entity with enough power to clearly oppose it so people making the models can get away with "questionable copyright", "legal gray area", etc. kind of talk. Try the same with a model trained on a single movie created by a multinational with infinite money and all this will be resolved and defined real quick and strongly worded cease & desist letters sent out by end of work day.
- sandworm101 4y ago
  The day after movie-to-VR becomes easy to do, movie owners will monitize it, which will end the current free-for-all. Want to walk around The Shire in VR? You better have paid for the Hobbits Online subscription package.
  - zimpenfish 4y ago
    Sam: This is it. Frodo: What? Sam: If I take one more step, I'll be the farthest away from home I've ever been. Popup: And you can do that for just $29.99 when you buy The Hobbits Shire Expansion DLC!
  - SideburnsOfDoom 4y ago
    > The day after movie-to-VR becomes easy to do, movie owners will monitize it, which will end the current free-for-all. The movie making studios already own those 3d "digital assets" from their films. IDK if there is a real "free-for-all" right now, but the monetisation play has already been planned for.

peoplefromibiza 4y ago

As someone who never appreciated VR, this looks great. Now please do the Ishtar Gate https://en.wikipedia.org/wiki/Ishtar_Gate https://en.wikipedia.org/wiki/Ishtar_Gate

huhtenberg 4y ago

I think it's worth taking a step back and just marveling at what's now possible to do with an off-the-shelf software. Yeah, it's "just" 50 fps, but ... would you just look at it! 4 years ago there was an equally fascinating "Paris apartment" VR demo, but the movement was limited and the amount of work that went into appears to have been orders of magnitude more - https://www.benoitdereau.com/ https://www.benoitdereau.com/ Go back 10 years and seeing this back then would've been absolutely bananas. Makes you wonder what we'll have in 10 years from now.

Daub 4y ago
Off the shelf yes, but wow…. The requirements for Unreal are unreal. Recent versions handle real-time global illumination (Lumen) and dynamically render geometry to insane levels of detail (nanite). Both of these are groundbreaking achievements. But having had to install this in a bunch of university computers I can tell you that it comes at a cost. Interestingly, the most significant issue is the space these installations require. We have optimized our installation as much as we can, but we nonetheless had to introduce systems whereby local storage was wiped clean every two weeks.
- solardev 4y ago
  Wait, disk space was the limiting factor, not GPUs?
  - smoldesu 4y ago
    I couldn't find the exact video, but Digital Foundry did a great breakdown on the performance bottlenecks in Unreal 5. They showed how disk speed directly influenced asset streaming, and it was pretty interesting to watch. Their HDD-speed drive choked constantly when running the UE5 demo, but a SATA SSD had enough bandwidth to handle the assets. So, it's kinda a weird spot to be. Not every scene was streaming 250mb/s of assets, but when it did the worse drives took a notable hit to performance.
    - adgjlsfhk1 4y ago

causi 4y ago

Am I alone in thinking most of this year's Unreal Engine demonstrations are just...not that great? Like, this is what graphics looked like on the Xbox One from nine years ago: https://halo.wiki.gallery/images/1/12/H5G-Waypoint-Swords_of_Sanghelios.jpg https://halo.wiki.gallery/images/1/12/H5G-Waypoint-Swords_of...

kuschku 4y ago
The quality of nanite heavily depends on your model. You can use photogrammetry scans or CAD exports directly, and nanite will make sure they'll look good. But if you just use existing videogame or fanmade assets (like here), the quality obviously suffers.
bogwog 4y ago
Nanite gives you high fidelity and Lumen gives you physically accurate lighting, but those things can't replace good art direction.
ainiriand 4y ago
Well that screen capture is taken from a cinematic scene if I am correct. UE demonstrations are real-time playable environments. Have you tested the Valley of the Ancients demo on your computer? You will see what I mean.
sk0g 4y ago
That screenshot does not look rendered in game, and no idea what the resolution is - at least not by an Xbox One. If current games are indistinguishable from single-frame renders that take minutes if not hours, I'd say that's a massive win! This looks more like the actual in-game rendering: http://cdn.themis-media.com/media/global/images/library/deriv/997/997367.jpg http://cdn.themis-media.com/media/global/images/library/deri... What stands out to me is the low resolution textures everywhere, obvious seams between terrain and assets (runtime virtual texturing resolves that), light seams which have odd seams, and the entire ambience being vaguely similar to how light acts, but not really - primarily the lack of bounce lighting affecting anything. With the move to HDR, the more binary lighting of previous games looks weird too now. Contrast that with the rich detail in lighting you can see in Fortnite's new update - https://i.redd.it/g2wqe10no44a1.jpg https://i.redd.it/g2wqe10no44a1.jpg Nanite, especially foliage, is something you have to experience for yourself though. Foliage pop-in not being so stark and alarming is huge for me, and the rich lighting capabilities improves immersion massively. Regarding assets, high polygon assets have been created for ages too, but were never actually brought in to games. You bake those details into your normal maps, but with Nanite you can just bring them into games, and get the full detail when the player is up close to them. When games can fully embrace Lumen and Nanite, the developer iteration improvement will be huge.

dncornholio 4y ago

This is super cool. Performance doesn't really matter, as long as it can keep 45 fps, the experience would be good enough with motion reprojection turned on.

Loic 4y ago

Thinking about it, it could be a very good tool to help cure fear of heights (or vertigo, not sure about the right term in English).

time_to_smile 4y ago
If you haven't played it, I highly recommend Richie's Plank Experience [0]. The main focus of the game is you take an elevator to the top of a building, there is a plank an you jump off it. I've found depending on the age of the participant the experience ranges from light fun to harrowing encounter with our own fears of death. I've seen people that, if it's their first time in VR, can take nearly an hour to finally do it. 0. https://www.oculus.com/experiences/quest/1642239225880682/ https://www.oculus.com/experiences/quest/1642239225880682/
pwdisswordfish0 4y ago
I sometimes fear I shouldn’t play driving games too much, so that I don’t get desensitised against those intrusive thoughts I sometimes get on the road. I wonder how irrational that actually is, because it feels like pretty much the same thing as your idea. I mean it doesn’t seem far fetched that a very realistic VR driving simulation with subtly “easier” physics might make you a worse driver in reality. More closely related to the fear of heights, we also jump from greater heights in video games than we would in real life, and much more frequently. I practically never have a reason to jump down from anywhere, so I could imagine making some wrong estimations when, after years of doing it in VR, I’m somehow faced with a real-life situation.

zokier 4y ago

One thing that sticks out like sore thumb are those relatively low-res rock textures, especially close-up. Would be neat to see some procedural generation there. For reference here is the scene from the movie https://www.youtube.com/watch?v=EN7kG7Ui7Bg https://www.youtube.com/watch?v=EN7kG7Ui7Bg

norwalkbear 4y ago

Whoa that's amazing.