3 ms·

As someone who's worked on visual odometry projects before, I'm extremely interested in how they're able to compute the phone's motion estimations so accurately

As someone who's worked on visual odometry projects before, I'm extremely interested in how they're able to compute the phone's motion estimations so accurately and in real time on a mobile device. Is any of this project open source? Is anyone here familiar with the vision techniques they're using to accomplish this feat? Perhaps there's an FPGA or ASIC designed for visual odometry inside the phone.

siera 13y ago
It uses a Myriad 1 processor from Movidius http://www.movidius.com/tango http://www.movidius.com/tango. It's a custom media coprocessor aimed at mobile devices. I found more information about the processor architecture in these slides : http://www.hotchips.org/wp-content/uploads/hc_archives/hc23/HC23.19.8-Video/HC23.19.811-1TOPS-Media-Moloney-Movidius.pdf http://www.hotchips.org/wp-content/uploads/hc_archives/hc23/... Edit : Video of the talk associated with the slides http://www.youtube.com/watch?v=KiTaPRtN2yM http://www.youtube.com/watch?v=KiTaPRtN2yM
- ulyssesgrant 13y ago
  Thanks for the info. Still very impressed with the speed and accuracy, assuming it works as well as they have presented. To be able to compute visual odometry in an arbitrary scene on a mobile device seems like a really big accomplishment, no?
  - siera 13y ago
    Visual odometry is getting quite accurate. For example, it is used on the Mars Exploration Rovers (Spirit and Opportunity) to improve the position obtained from wheel odometry [1]. As you mentionned, the big accomplishment is doing this in real time on a mobile device. On a typical mobile ARM processor, it would probably be very slow and drain the battery in no time. That's why they are using a custom coprocessor designed specifically for image processing and computer vision. The next challenge is porting the existing visual odometry algorithms on this new processor. It looks like hiDOF did the implementation [2]. [1] http://onlinelibrary.wiley.com/doi/10.1002/rob.20184/abstract http://onlinelibrary.wiley.com/doi/10.1002/rob.20184/abstrac... [2] http://hidof.com/projects/project-tango/ http://hidof.com/projects/project-tango/

siera 13y ago

It uses a Myriad 1 processor from Movidius http://www.movidius.com/tango http://www.movidius.com/tango. It's a custom media coprocessor aimed at mobile devices. I found more information about the processor architecture in these slides : http://www.hotchips.org/wp-content/uploads/hc_archives/hc23/HC23.19.8-Video/HC23.19.811-1TOPS-Media-Moloney-Movidius.pdf http://www.hotchips.org/wp-content/uploads/hc_archives/hc23/... Edit : Video of the talk associated with the slides http://www.youtube.com/watch?v=KiTaPRtN2yM http://www.youtube.com/watch?v=KiTaPRtN2yM

ulyssesgrant 13y ago
Thanks for the info. Still very impressed with the speed and accuracy, assuming it works as well as they have presented. To be able to compute visual odometry in an arbitrary scene on a mobile device seems like a really big accomplishment, no?
- siera 13y ago
  Visual odometry is getting quite accurate. For example, it is used on the Mars Exploration Rovers (Spirit and Opportunity) to improve the position obtained from wheel odometry [1]. As you mentionned, the big accomplishment is doing this in real time on a mobile device. On a typical mobile ARM processor, it would probably be very slow and drain the battery in no time. That's why they are using a custom coprocessor designed specifically for image processing and computer vision. The next challenge is porting the existing visual odometry algorithms on this new processor. It looks like hiDOF did the implementation [2]. [1] http://onlinelibrary.wiley.com/doi/10.1002/rob.20184/abstract http://onlinelibrary.wiley.com/doi/10.1002/rob.20184/abstrac... [2] http://hidof.com/projects/project-tango/ http://hidof.com/projects/project-tango/