4 ms·
is there any progress on combining object detection with object segmentation? So instead of boundary boxes we get the true shape of objects? I know segmentation
by singularity2001 4y ago
is there any progress on combining object detection with object segmentation? So instead of boundary boxes we get the true shape of objects? I know segmentation exists, just wondering about integration with Yolo or similar.
- genewitch 4y agoIs this "rotoscoping"?
- klysm 4y agoIn real time?
- yeldarb 4y agoYes, this is called "instance segementation". There's a YOLO-based instance segmentation model called YOLACT.
- singularity2001 4y agoWonderful, thanks. It says backbone: Resnet101-FPN in https://github.com/dbolya/yolact https://github.com/dbolya/yolact ? Anyone else looking for a pip installable solution: I found https://github.com/ayoolaolafenwa/PixelLib https://github.com/ayoolaolafenwa/PixelLib And most current: https://github.com/yeliudev/catnet https://github.com/yeliudev/catnet
- Q6T46nT668w6i3m 4y agoYes, look into instance or panoptic segmentation. The most popular method is a region-based network that jointly regresses bounding box coordinates alongside an object mask and class label.
- singularity2001 4y agoThanks. The next step would be combining it with text-image foundation models such as clip https://github.com/openai/CLIP https://github.com/openai/CLIP so that the model no longer depends on a limited set of predefined labels (coco…), right? Also occlusion inference would be fantastic, so that we can select between the visible parts of the object and the whole shape (behind trees etc). Exciting decade.
- dimatura 4y agoThis task is called instance segmentation and is an active research topic. Mask-RCNN is relatively old these days, but still might be the most popular approach. There also happens to be a few approaches for the task taking inspiration in methodology from YOLO, e.g. YOLACT (which clearly also pays homage in name).