3 ms·

This seems impressive, but it shows that there is still some way to go when comparing to old school techniques. I don't know how many TPU they used but probably

This seems impressive, but it shows that there is still some way to go when comparing to old school techniques. I don't know how many TPU they used but probably a lot. You can build a 3d key-points map using Slam algorithms real time on a raspberry pi. From there, you render those key-points and descriptors to a virtual screen given the desired camera pose, then you learn a deconvolution from these sparse rendered key-points to image mapping. Alternatively, using more memory, once you have a 3d map, you can save some key-frame with camera poses, when ask the view from a given pose, you pick the k closest poses and interpolate (eventually with a neural net). If now you have some more compute, you can do the previous slam algorithms with dense maps, and interpolate the dense 2.5d point clouds. Their network is probably doing inefficiently a mixture of those different things, trading compute and memory power for flexibility.

zxcvvcxz 8y ago
> You can build a 3d key-points map using Slam algorithms real time on a raspberry pi. Does anyone have links to any open source projects doing this? Preferably with an example video(s) showing results?
- GistNoesis 8y ago
  I used this today for my robot. https://github.com/Alkaid-Benetnash/ORB_SLAM2/ https://github.com/Alkaid-Benetnash/ORB_SLAM2/ (This particular fork can save the map although it needs to be generated on the pi). It works almost out of the box, just need half a day of slow compilation. For use on a raspberry pi model 3B+, 2000 key-points, it runs 1-2 fps at 640x480, 5-6 fps at 320x240. Use 500M for a few rooms and 75% CPU when map building 50% CPU once built. It's not optimized for the pi so you can probably get it to run at least 3 times faster if you are willing to get your hands dirty. For it to work well a 180 degree camera really makes a difference, and run opencv cpp-tutorial-cameracalibration on a chessboard to get the needed extrinsics. There are probably other slam algos in ROS, but I'm not sure how raspberry-pi compatible they are. If you want to try and experiment with neural networks, once you have build your 3d map on powerful computer you can build a neural network to learn the pose from the image. This will allow you to have some constant time, constant memory algorithm for later use on the pi, it will probably be less precise.

zxcvvcxz 8y ago

> You can build a 3d key-points map using Slam algorithms real time on a raspberry pi. Does anyone have links to any open source projects doing this? Preferably with an example video(s) showing results?

GistNoesis 8y ago
I used this today for my robot. https://github.com/Alkaid-Benetnash/ORB_SLAM2/ https://github.com/Alkaid-Benetnash/ORB_SLAM2/ (This particular fork can save the map although it needs to be generated on the pi). It works almost out of the box, just need half a day of slow compilation. For use on a raspberry pi model 3B+, 2000 key-points, it runs 1-2 fps at 640x480, 5-6 fps at 320x240. Use 500M for a few rooms and 75% CPU when map building 50% CPU once built. It's not optimized for the pi so you can probably get it to run at least 3 times faster if you are willing to get your hands dirty. For it to work well a 180 degree camera really makes a difference, and run opencv cpp-tutorial-cameracalibration on a chessboard to get the needed extrinsics. There are probably other slam algos in ROS, but I'm not sure how raspberry-pi compatible they are. If you want to try and experiment with neural networks, once you have build your 3d map on powerful computer you can build a neural network to learn the pose from the image. This will allow you to have some constant time, constant memory algorithm for later use on the pi, it will probably be less precise.