I once looked into mouse sensors for similar purpose, but the raw frame export capabilities were disappointing. Looking at the datasheet of PMW3360 which claims

I once looked into mouse sensors for similar purpose, but the raw frame export capabilities were disappointing. Looking at the datasheet of PMW3360 which claims to have up to 12000 FPS, but the timing sequence for raw frame dump suggests ~20ms if not 40ms to transfer the whole frame which would give only 50-25FPS. That doesn't necessarily mean the main spec is lying, it just isn't capable of dumping the frames over SPI as fast as it is producing and processing them internally. Frame dumping seems to be more of debug/test feature, so no big surprise it's not capable of fully utilizing the sensor and there is little motivation for sensor manufacturer to improve it. But there is also a chance that the manufacturer has put very conservative timing for it in the datasheet, since it's not the main functionality of sensor. If anyone has explored further how much the timing can be optimized, or maybe there are other mice sensors for with better frame dumping capabilities, I am curious to hear about it as well. With that said even if you remove all the unnecessary delays data transfer over SPI will likely be a limiting factor for very high framerates. Assuming 4MHz SPI and 32x32 8bpp image that would be at most 500FPS. Few hundred FPS would still be nice, and you might push it a bit higher with increasing frequency but it's probably not possible to push it much further than 1000FPS without MIPI or some kind of other parallel data transfer protocol. The ardupilot project in sibling comment seems to be using the mouse movement output directly instead of capturing the frame and then doing optical flow analysis on main CPU. The image dump is only being used to verify camera focus.