3 ms·
Something seems weird here. I agree that Python has some overheard, but the time taken should presumably be dominated by the neural network object detection. I
by nl 6y ago
Something seems weird here.
I agree that Python has some overheard, but the time taken should presumably be dominated by the neural network object detection. In TensorFlow that is written in (highly optimised) C, and should be using the NEON instructions on ARM[1].
Notably, DeepStream gives the same performance with the Python and C++[2].
YOLO inference speed is generally higher than a Mobilenet SSD, but you can run YOLO on TensorFlow instead of Darknet[3], or use a NNPACK version of Darknet.
Edit: "I don't need to turn on the noisy Xavier fan." - wait - this isn't on a Raspberry Pi? If you have a GPU on device then there's lots of other things going on.
[1] https://www.tensorflow.org/install/source_rpi https://www.tensorflow.org/install/source_rpi
[2] https://developer.nvidia.com/deepstream-sdk https://developer.nvidia.com/deepstream-sdk (Scroll down for benchmarks)
[3] https://github.com/hunglc007/tensorflow-yolov4-tflite https://github.com/hunglc007/tensorflow-yolov4-tflite
- staycoolboy 6y ago> wait - this isn't on a Raspberry Pi? Literally the first sentence of my post. You and many others seem to forget that I explicitly stated I wanted 4 HD streams at 30fps from my home IP address. > but the time taken should presumably be dominated by the neural network object detection There is a lot more to a pipeline than just inference. The problem is aggregating the video streams, downsampling, submitting for inference, and then activating the Gstreamer element to write the MPEG. Most of this can use the nvidia memory properties on the nv* elements, which is great! However eventually you need to copy out for the Arm core. 4 HD streams is a lot of work for a small Arm core. The basic Gstreamer elements do not use the ACL AFAIK. I did recompile them with ORC optimization, but I'm not too familiar how/if that uses ACL/NEON. And the one that I built is basically just a bus messaging system for the downstream codec pipeline that is flagged by the tracker. Can you point me to the benchmark in your link [#2] that indicates Python & C++ have the same performance? Nvidia does have an advantage in that Tensorflow-gpu supports them natively, but that is just for inference. I only see one table and the comments explicitly state they use the DS SDK and -not- TFlow.