3 ms·

It'd be nice to see this new technique replace the very crude warping animation in Google maps' street view.

by nayuki 6y ago

crazygringo 6y ago
Google Maps Street View already collects some low-res lidar data, so they don't even need this. They just need to build it into their viewer, which I also wish they would do.
- moron4hire 6y ago
  They do use the Lidar data in the viewer. You can see it with the positioning of the cursor on "surfaces" when you're looking around. The reprojection animation they do between photospheres is based on the Lidar data, it's just that's about the best you can do with the low-res data they've collected. Imagine a situation where you're standing on a street corner with a mailbox or something standing on the corner. You move to the next photo position, which takes you past the mailbox. The smearing and warping you are seeing is from the continous mesh they've created from the Lidar data not matching the real world mesh in areas that were occluded from the original point of view. You get a moment of seeing "behind" the mailbox, but there is no data for what is behind the mailbox, so it all gets interpolated from the data that is known in the surrounding visual area. The mailbox becomes a rectangular prism that extends all the way from the mailbox's location through to the intersection with the ground that we can see behind the mailbox. These are just the problems with single-image mesh recreation. You can't really get around them without some form of inference of the data that doesn't exist. You even see it in Facebook's images in the linked article, if you look closely. They try to cut the videos early so you don't see it, but it's there if you know what to look for.
  - crazygringo 6y ago
    > They do use the Lidar data in the viewer. Oh wow, you're right, I just tried it out. But it's so blocky and apparently limited to (mostly) 90° geometry, that I'd never realized Google was doing anything but modeling a one-size-fits-all "rectangular corridor" along each street. It seems like it's not exposing any kind of raw Lidar data, but a very simple geometric simplification of it. (E.g. trees are either ignored completely, or if there are enough of them they're treated as the side of a building instead.) I completely understand what you're saying about the problems with smearing and warping due to not enough data. But I still can't help but wonder what it would look like if they were able to generate a "raw" (but denoised) Lidar geometry, so that trees and cars were treated as individual objects, rather than just either as part of the street floor or part of the building walls they way they are now.
londons_explore 6y ago
I believe the crude warp is caused by the difficult combination of: * Need to do animation with very low latency (can't go to the network to collect data for the animation when the user clicks) * Need to do the animation without much CPU/GPU power in the browser. * Need to not download much data for the animation ahead of time (browsing panoramas is already very data heavy).
anorakoverflow 6y ago
I’ve tried out Apple’s Look Around recently and was impressed with the transition animation compared to Google Maps. If you have an Apple device, try it out in London for instance: Moving past a red telephone box shows how the object kind of retains its shape during the transition. Traffic moving around you also looks quite different compared to Google.

It'd be nice to see this new technique replace the very crude warping animation in Google maps' street view.

by nayuki 6y ago

It'd be nice to see this new technique replace the very crude warping animation in Google maps' street view.

Google Maps Street View already collects some low-res lidar data, so they don't even need this. They just need to build it into their viewer, which I also wish they would do.

moron4hire 6y ago
They do use the Lidar data in the viewer. You can see it with the positioning of the cursor on "surfaces" when you're looking around. The reprojection animation they do between photospheres is based on the Lidar data, it's just that's about the best you can do with the low-res data they've collected. Imagine a situation where you're standing on a street corner with a mailbox or something standing on the corner. You move to the next photo position, which takes you past the mailbox. The smearing and warping you are seeing is from the continous mesh they've created from the Lidar data not matching the real world mesh in areas that were occluded from the original point of view. You get a moment of seeing "behind" the mailbox, but there is no data for what is behind the mailbox, so it all gets interpolated from the data that is known in the surrounding visual area. The mailbox becomes a rectangular prism that extends all the way from the mailbox's location through to the intersection with the ground that we can see behind the mailbox. These are just the problems with single-image mesh recreation. You can't really get around them without some form of inference of the data that doesn't exist. You even see it in Facebook's images in the linked article, if you look closely. They try to cut the videos early so you don't see it, but it's there if you know what to look for.
- crazygringo 6y ago
  > They do use the Lidar data in the viewer. Oh wow, you're right, I just tried it out. But it's so blocky and apparently limited to (mostly) 90° geometry, that I'd never realized Google was doing anything but modeling a one-size-fits-all "rectangular corridor" along each street. It seems like it's not exposing any kind of raw Lidar data, but a very simple geometric simplification of it. (E.g. trees are either ignored completely, or if there are enough of them they're treated as the side of a building instead.) I completely understand what you're saying about the problems with smearing and warping due to not enough data. But I still can't help but wonder what it would look like if they were able to generate a "raw" (but denoised) Lidar geometry, so that trees and cars were treated as individual objects, rather than just either as part of the street floor or part of the building walls they way they are now.

londons_explore 6y ago

I believe the crude warp is caused by the difficult combination of: * Need to do animation with very low latency (can't go to the network to collect data for the animation when the user clicks) * Need to do the animation without much CPU/GPU power in the browser. * Need to not download much data for the animation ahead of time (browsing panoramas is already very data heavy).

anorakoverflow 6y ago

I’ve tried out Apple’s Look Around recently and was impressed with the transition animation compared to Google Maps. If you have an Apple device, try it out in London for instance: Moving past a red telephone box shows how the object kind of retains its shape during the transition. Traffic moving around you also looks quite different compared to Google.