3 ms·

All I see, usually, is some AI YOLO algorithm applied to an offline video. This is the first time that I've seen a "complete" setup. Any info to learn more on

All I see, usually, is some AI YOLO algorithm applied to an offline video. This is the first time that I've seen a "complete" setup. Any info to learn more on applying YOLO and similar models to real time streams (whatever the format)?

hug 2y ago
This repository seems to be exactly what you are asking for. It's YOLO analysis of video frames passed in through Real Time Streaming Protocol.
- _giorgio_ 2y ago
  Yes, probably it's only one reasonably sized, let's say that with a lot of patience you can study it! I'll search for some online resources too. I thought that this topic yolo object recognition would have much more following, instead there are really only a few projects. https://github.com/search?q=yolo+rtsp&type=repositories&s=forks&o=desc https://github.com/search?q=yolo+rtsp&type=repositories&s=fo...
llm_trw 2y ago
Just stream it one frame at a time to the model and eat the latency: https://www.youtube.com/watch?v=IHbJcOex6dk https://www.youtube.com/watch?v=IHbJcOex6dk if you need more hand holding. There's a reason why there's a whole family of models from tiny to huge.
- yeldarb 2y ago
  If you do it naively your video frames will buffer waiting to be consumed causing a memory leak and eventual crash (or quick crash if you’re running on a device with constrained resources). You really need to have a thread consuming the frames and feeding them to a worker that can run on its own clock.
  - llm_trw 2y ago
    That's not how loop devices work on Linux.
  - _giorgio_ 2y ago
    Sorry for the newbie question Under windows, say that I have an RTSP stream (or something similar) Would you use a single python script with which one of this multithreading solutions? 1 import concurrent.futures 2 import multiprocessing 3 import threading
- _giorgio_ 2y ago
  Thanks for the link, but what happens when you have a video stream, be it a usb webcam, or an RTSP stream, and the hardware can't keep up? I'm on windows. Ideally I'd like the frames to be dropped, so the inference is done on the last received frame? Is this a standard behaviour?
yeldarb 2y ago
We’ve got an open source pipeline as part of inference[1] that handles the nuances (multithreading, batching, syncing, reconnecting) of running multiple real time streams (pass in an array of RTSP urls) for CV models like YOLO: https://blog.roboflow.com/vision-models-multiple-streams/ https://blog.roboflow.com/vision-models-multiple-streams/ [1] https://github.com/roboflow/inference https://github.com/roboflow/inference

hug 2y ago

This repository seems to be exactly what you are asking for. It's YOLO analysis of video frames passed in through Real Time Streaming Protocol.

_giorgio_ 2y ago
Yes, probably it's only one reasonably sized, let's say that with a lot of patience you can study it! I'll search for some online resources too. I thought that this topic yolo object recognition would have much more following, instead there are really only a few projects. https://github.com/search?q=yolo+rtsp&type=repositories&s=forks&o=desc https://github.com/search?q=yolo+rtsp&type=repositories&s=fo...

llm_trw 2y ago

Just stream it one frame at a time to the model and eat the latency: https://www.youtube.com/watch?v=IHbJcOex6dk https://www.youtube.com/watch?v=IHbJcOex6dk if you need more hand holding. There's a reason why there's a whole family of models from tiny to huge.

yeldarb 2y ago
If you do it naively your video frames will buffer waiting to be consumed causing a memory leak and eventual crash (or quick crash if you’re running on a device with constrained resources). You really need to have a thread consuming the frames and feeding them to a worker that can run on its own clock.
- llm_trw 2y ago
  That's not how loop devices work on Linux.
- _giorgio_ 2y ago
  Sorry for the newbie question Under windows, say that I have an RTSP stream (or something similar) Would you use a single python script with which one of this multithreading solutions? 1 import concurrent.futures 2 import multiprocessing 3 import threading
_giorgio_ 2y ago
Thanks for the link, but what happens when you have a video stream, be it a usb webcam, or an RTSP stream, and the hardware can't keep up? I'm on windows. Ideally I'd like the frames to be dropped, so the inference is done on the last received frame? Is this a standard behaviour?

yeldarb 2y ago

We’ve got an open source pipeline as part of inference[1] that handles the nuances (multithreading, batching, syncing, reconnecting) of running multiple real time streams (pass in an array of RTSP urls) for CV models like YOLO: https://blog.roboflow.com/vision-models-multiple-streams/ https://blog.roboflow.com/vision-models-multiple-streams/ [1] https://github.com/roboflow/inference https://github.com/roboflow/inference