If the image you get out of the sensor is processing friendly (such as just the image of the IR leds) then you can do this pretty quickly in software. Some pre

If the image you get out of the sensor is processing friendly (such as just the image of the IR leds) then you can do this pretty quickly in software. Some preprocessing on the camera can help but the source of most of the latency is going to be the image sensor readout. I've actually worked on a real time tracking system like this about 20 years ago.