I’ve been working to get orb_slam to run on the raspberry pi in real-time. One of the most interesting parts, to me, is the bag of words approach to feature mat

I’ve been working to get orb_slam to run on the raspberry pi in real-time. One of the most interesting parts, to me, is the bag of words approach to feature matching. It’s shockingly reliable for such a simple method. It’s used for initialization when no world model is available, a backup for when the motion model fails to track a new frame, for relocalization and loop closing. Orb_slam wouldn’t work without it, and all it is is a kmeans tree for classifying nearest neighbors. When I was starting out, I was looking for a SLAM system, not just visual odometry, that could realistically be sped up for the pi. The winning factor for orb_slam was that it used sparse simple feature points that could be cheaply classified. After spending almost a year working on this, I have the system running at 10fps on the kitti benchmark, which are sequences of larger images also recorded at 10fps. VGA images can run up to 20fps depending on various trade offs. My present opinion is that orb features are great for mapping, but not robust enough for reliable visual odometry. It wouldn’t be so much of an issue to generate a local point cloud on the pi, but processing the cloud in real time is out of the question. If someone works out how to reliably and cheaply classify point cloud features it would be a game changer.