6 ms·

Cesium for Unreal Now Available

254 points by kylebarron 6y ago

mike_d 6y ago
From the demo video this looks really interesting for games/use cases that are centered around the >1000ft view. Even in the video showing off the technology things look downright terrible up close. If you pause at the very beginning you can see handrails on the stairs that are just odd floating blobs, cars have all melted into the ground. It is incredibly impressive that they have managed to get aerial LIDAR into a game engine, but the visuals are quite jarring up close.
- Gabriel_Martin 6y ago
  I've noticed a trend of games graphics being fairly oldschool while the mechanics are modern. Maybe this'll lend it's self to that trend?
- terhechte 6y ago
  It would still lend itself to some interesting games, such as Mech Battles or Warcraft with oversized vessels or hordes of leming-sized people in these environments.
  - pbhjpbhj 6y ago
    Maybe a game of Global Thermonuclear War?
    - unixhero 6y ago
      Shall we play a game?
- rebuilder 6y ago
  Also good for prototyping and seeding ideas for level design, I'd imagine. I saw someone proto their run-and-gun game on google maps geo - they'll have to model everything once they lock a location down, but can test complex environments very quickly. Also having a real location, however rough, as a base gives you a lot of ideas you would have a hard time coming up with, starting with a blank slate.
- tinco 6y ago
  This is due to the method of data acquisition. My company also uses cesium (in the browser) for visualization at sub cm level for inspection. Our quality bar is that inspectors have to be able to inspect the grain of the grout between the bricks. We acquire the data in 3 modes for high buildings, we do low altitude fly over with the 2m drone, then we do facade with the regular drone, and finally we shoot from the ground. We only do ground based lidar, but aerial lidar is improving so maybe that's next. The rest is all high detail photos fed into photogrammetry. The final results look perfect in cesium, some thin rails might not show up, and windows get a bit funky, but the geometry and the textures load in without a hitch, really cool technology that's so simple in the basis that we built an alternative viewer in Rust+WASM in 3 weeks.
  - StavrosK 6y ago
    Wait, if the technology is so simple that you can build an alternative viewer in 3 weeks, what are they selling? Is it the data format?
    - asutekku 6y ago
      They’re selling it so you don’t have to get it by yourself.
    - sbarre 6y ago
      3 weeks for a prototype 3 months for an alpha 3 quarters for a production release. You know how engineering estimates work... ;-)
    - tinco 6y ago
      They've got a whole bunch of features in their viewer, making a feature matching viewer would take years. The basic technology is a tree of JSON files describing what models or other JSON files to load for regions of space. So the nodes are JSON files and the leaves are 3d models basically. There's a simple algorithm that you give your current perspective (i.e. where you are in space, and what direction you're looking in) that will walk the tree and decide what models will be in your field of view and which won't be, and at what distance (which decides the level of detail). Then the last step is simply downloading those models, while at the same time evicting irrelevant 3d models from your gpu. They hand rolled an entire high performance 3D engine in Javascript, which is a big challenge in and of itself. We cheated by using WASM and a high performance programming language, making the 3D engine part trivial. Likewise, I'd be surprised if it took them more than a week or two to get their initial proof of concept working in Unreal Engine. Oh and I believe their business model is/was to sell hosting and processing for the 3D models. We've been paying them a couple hundred per month just for this service.
- rtpg 6y ago
  I imagine this will be used for stuff outside of games primarily. My understanding is that Unreal is now used a lot in specialized domains like film-making (globe fly-bys in the news and documentaries?) and sort of general "dashboard"-y stuff. I'm sure someone could think of an "interesting" game built off of this data but I highly doubt you go through all this engineering effort on that idea alone.
  - TeMPOraL 6y ago
    > Unreal is now used a lot in specialized domains like film-making (globe fly-bys in the news and documentaries?) It goes further than that now. For The Mandalorian, they famously set up a "20-foot high, 270-degree semicircular LED video wall"[0] to use as a backdrop for the shots, and that wall was rendering the desired environment using Unreal Engine. No need to chroma key an alien planet in post, when you can make it a part of the scene (and now the actors can see it too as they play!). More at [1]. -- [0] - https://www.polygon.com/tv/2020/2/20/21146152/the-mandalorian-making-of-video-unreal-engine-projection-screen https://www.polygon.com/tv/2020/2/20/21146152/the-mandaloria... [1] - https://www.unrealengine.com/en-US/blog/forging-new-paths-for-filmmakers-on-the-mandalorian https://www.unrealengine.com/en-US/blog/forging-new-paths-fo...
    - sbarre 6y ago
      ILMVFX's YouTube channel has some fantastic BTS footage of how this works, for anyone curious: https://www.youtube.com/channel/UCjnYk44Aj9E634TPucpIXnQ https://www.youtube.com/channel/UCjnYk44Aj9E634TPucpIXnQ
    - anentropic 6y ago
      From what I read about it previously I think they do actually re-render the backgrounds in post But the LED video walls serve two purposes: ensure the lighting and reflections on real objects in the scene match up with the CGI, and give the actors a visualisation to work with.
      - sbarre 6y ago
- zokier 6y ago
  It has very distinct low-res photogrammetry look yes. But if embraced and under good art direction I think it could be made to work as a stylized look. Maybe throw in some shader effects to spice it up a bit and make sure that the hand-modeled objects fit into the design, and it could be pretty cool.
notum 6y ago
Apologies for potentially asking a really silly question but: Cesium plugin is free and open source, however the Ion service (the actual data) is paid. So of what use is the plugin without the streamed data? Can we choose to patch into Bing photogrammetry data as an alternative?
- chrisseaton 6y ago
  > So of what use is the plugin without the streamed data? None, but so what? Being open source is useful even if the data you use it with is proprietary.
  - notum 6y ago
    Absolutely. They just make a huge point about the plugin being open source, how that's "how business should be done", I thought there would be more to it, some useful tech beyond the same-old vendor lock-in or anything.
    - lnanek2 6y ago
      As a developer, having an open source client side is hugely beneficial. With closed source mapping libs you often know what is wrong with the providers library from your stack traces, but have no way to fix it. Open source means trivial bugs handling things like nulls and deallocation and even more serious bugs like CPU use regressions (which are easy to introduce with an incorrect condition somewhere and can slip through the cracks until end user's batteries are dying) become trivial fixes instead of: notify provider, provider may or may not road map a fix, provider may or may not provide the fix weeks later in their next closed source SDK update.
- willio58 6y ago
  In theory you could create your own data source. I mean, good luck, but you could. I think keeping the data closed is totally fine in this case.
- ihsw 6y ago
  The goal is (probably) for the Cesium data platform to be number one 3D geospatial content standard and for the Ion service to the first (and best) provider of Cesium data. There is also an implicit service where they (Cesium GS Inc) offer consulting and collaboration on curating high-quality private Cesium data sets in parallel to their own, eg: military applications.
a_c 6y ago
I think it is a bedrock in bridging gaming and real life. From now on you can visit virtual store in real-but-in-game location and do all the purchase without leaving the game. Imagine GTA, but everything you buy is real
dvdkon 6y ago
Oh, I thought someone made an addon for the original Unreal from 1998. I should have known better, but it's still funny to me.
- TheCraiggers 6y ago
  Frankly, I think that would be more HN worthy, and much more interesting as it would like likely involved a clever hack.
klmadfejno 6y ago
Can anyone comment on how good these datasets are? Are these going to be down to like, an individual tree? I also note it mentions integrating with unreal's water engine. Unreal's water engine is excellent, but as of yet, not so great with tiled landscapes. Curious how that'll work.
tpoindex 6y ago
Nice models of Denver on the video, especially Union Station, City & County Building, airliner flyover. But the car is driving the wrong way on Broadway, even though the parked cars on the right side are pointed in the correct direction. Maybe a new GTA Denver edition is in the works? heh.

254 points by kylebarron 6y ago

From the demo video this looks really interesting for games/use cases that are centered around the >1000ft view. Even in the video showing off the technology things look downright terrible up close. If you pause at the very beginning you can see handrails on the stairs that are just odd floating blobs, cars have all melted into the ground. It is incredibly impressive that they have managed to get aerial LIDAR into a game engine, but the visuals are quite jarring up close.

Gabriel_Martin 6y ago
I've noticed a trend of games graphics being fairly oldschool while the mechanics are modern. Maybe this'll lend it's self to that trend?
terhechte 6y ago
It would still lend itself to some interesting games, such as Mech Battles or Warcraft with oversized vessels or hordes of leming-sized people in these environments.
- pbhjpbhj 6y ago
  Maybe a game of Global Thermonuclear War?
  - unixhero 6y ago
    Shall we play a game?
rebuilder 6y ago
Also good for prototyping and seeding ideas for level design, I'd imagine. I saw someone proto their run-and-gun game on google maps geo - they'll have to model everything once they lock a location down, but can test complex environments very quickly. Also having a real location, however rough, as a base gives you a lot of ideas you would have a hard time coming up with, starting with a blank slate.
tinco 6y ago
This is due to the method of data acquisition. My company also uses cesium (in the browser) for visualization at sub cm level for inspection. Our quality bar is that inspectors have to be able to inspect the grain of the grout between the bricks. We acquire the data in 3 modes for high buildings, we do low altitude fly over with the 2m drone, then we do facade with the regular drone, and finally we shoot from the ground. We only do ground based lidar, but aerial lidar is improving so maybe that's next. The rest is all high detail photos fed into photogrammetry. The final results look perfect in cesium, some thin rails might not show up, and windows get a bit funky, but the geometry and the textures load in without a hitch, really cool technology that's so simple in the basis that we built an alternative viewer in Rust+WASM in 3 weeks.
- StavrosK 6y ago
  Wait, if the technology is so simple that you can build an alternative viewer in 3 weeks, what are they selling? Is it the data format?
  - asutekku 6y ago
    They’re selling it so you don’t have to get it by yourself.
  - sbarre 6y ago
    3 weeks for a prototype 3 months for an alpha 3 quarters for a production release. You know how engineering estimates work... ;-)
  - tinco 6y ago
    They've got a whole bunch of features in their viewer, making a feature matching viewer would take years. The basic technology is a tree of JSON files describing what models or other JSON files to load for regions of space. So the nodes are JSON files and the leaves are 3d models basically. There's a simple algorithm that you give your current perspective (i.e. where you are in space, and what direction you're looking in) that will walk the tree and decide what models will be in your field of view and which won't be, and at what distance (which decides the level of detail). Then the last step is simply downloading those models, while at the same time evicting irrelevant 3d models from your gpu. They hand rolled an entire high performance 3D engine in Javascript, which is a big challenge in and of itself. We cheated by using WASM and a high performance programming language, making the 3D engine part trivial. Likewise, I'd be surprised if it took them more than a week or two to get their initial proof of concept working in Unreal Engine. Oh and I believe their business model is/was to sell hosting and processing for the 3D models. We've been paying them a couple hundred per month just for this service.
rtpg 6y ago
I imagine this will be used for stuff outside of games primarily. My understanding is that Unreal is now used a lot in specialized domains like film-making (globe fly-bys in the news and documentaries?) and sort of general "dashboard"-y stuff. I'm sure someone could think of an "interesting" game built off of this data but I highly doubt you go through all this engineering effort on that idea alone.
- TeMPOraL 6y ago
  > Unreal is now used a lot in specialized domains like film-making (globe fly-bys in the news and documentaries?) It goes further than that now. For The Mandalorian, they famously set up a "20-foot high, 270-degree semicircular LED video wall"[0] to use as a backdrop for the shots, and that wall was rendering the desired environment using Unreal Engine. No need to chroma key an alien planet in post, when you can make it a part of the scene (and now the actors can see it too as they play!). More at [1]. -- [0] - https://www.polygon.com/tv/2020/2/20/21146152/the-mandalorian-making-of-video-unreal-engine-projection-screen https://www.polygon.com/tv/2020/2/20/21146152/the-mandaloria... [1] - https://www.unrealengine.com/en-US/blog/forging-new-paths-for-filmmakers-on-the-mandalorian https://www.unrealengine.com/en-US/blog/forging-new-paths-fo...
  - sbarre 6y ago
    ILMVFX's YouTube channel has some fantastic BTS footage of how this works, for anyone curious: https://www.youtube.com/channel/UCjnYk44Aj9E634TPucpIXnQ https://www.youtube.com/channel/UCjnYk44Aj9E634TPucpIXnQ
  - anentropic 6y ago
    From what I read about it previously I think they do actually re-render the backgrounds in post But the LED video walls serve two purposes: ensure the lighting and reflections on real objects in the scene match up with the CGI, and give the actors a visualisation to work with.
    - sbarre 6y ago
zokier 6y ago
It has very distinct low-res photogrammetry look yes. But if embraced and under good art direction I think it could be made to work as a stylized look. Maybe throw in some shader effects to spice it up a bit and make sure that the hand-modeled objects fit into the design, and it could be pretty cool.

notum 6y ago

Apologies for potentially asking a really silly question but: Cesium plugin is free and open source, however the Ion service (the actual data) is paid. So of what use is the plugin without the streamed data? Can we choose to patch into Bing photogrammetry data as an alternative?

chrisseaton 6y ago
> So of what use is the plugin without the streamed data? None, but so what? Being open source is useful even if the data you use it with is proprietary.
- notum 6y ago
  Absolutely. They just make a huge point about the plugin being open source, how that's "how business should be done", I thought there would be more to it, some useful tech beyond the same-old vendor lock-in or anything.
  - lnanek2 6y ago
    As a developer, having an open source client side is hugely beneficial. With closed source mapping libs you often know what is wrong with the providers library from your stack traces, but have no way to fix it. Open source means trivial bugs handling things like nulls and deallocation and even more serious bugs like CPU use regressions (which are easy to introduce with an incorrect condition somewhere and can slip through the cracks until end user's batteries are dying) become trivial fixes instead of: notify provider, provider may or may not road map a fix, provider may or may not provide the fix weeks later in their next closed source SDK update.
willio58 6y ago
In theory you could create your own data source. I mean, good luck, but you could. I think keeping the data closed is totally fine in this case.
ihsw 6y ago
The goal is (probably) for the Cesium data platform to be number one 3D geospatial content standard and for the Ion service to the first (and best) provider of Cesium data. There is also an implicit service where they (Cesium GS Inc) offer consulting and collaboration on curating high-quality private Cesium data sets in parallel to their own, eg: military applications.

a_c 6y ago

I think it is a bedrock in bridging gaming and real life. From now on you can visit virtual store in real-but-in-game location and do all the purchase without leaving the game. Imagine GTA, but everything you buy is real

dvdkon 6y ago

Oh, I thought someone made an addon for the original Unreal from 1998. I should have known better, but it's still funny to me.

TheCraiggers 6y ago
Frankly, I think that would be more HN worthy, and much more interesting as it would like likely involved a clever hack.

klmadfejno 6y ago

Can anyone comment on how good these datasets are? Are these going to be down to like, an individual tree? I also note it mentions integrating with unreal's water engine. Unreal's water engine is excellent, but as of yet, not so great with tiled landscapes. Curious how that'll work.

tpoindex 6y ago

Nice models of Denver on the video, especially Union Station, City & County Building, airliner flyover. But the car is driving the wrong way on Broadway, even though the parked cars on the right side are pointed in the correct direction. Maybe a new GTA Denver edition is in the works? heh.