4 ms·
YOLOv5 seems to have one important advantage over v4, which your post helped highlight: Fourth, YOLOv5 is small. Specifically, a weights file for YOLOv5 is 27
by sillysaurusx 6y ago
YOLOv5 seems to have one important advantage over v4, which your post helped highlight:
Fourth, YOLOv5 is small. Specifically, a weights file for YOLOv5 is 27 megabytes. Our weights file for YOLOv4 (with Darknet architecture) is 244 megabytes. YOLOv5 is nearly 90 percent smaller than YOLOv4. This means YOLOv5 can be deployed to embedded devices much more easily.
Naming controversy aside, it's nice to have some model that can get close to the same accuracy at 10% of the size.
Naming it v5 was certainly ... bold ... though. If it can't outperform v4 in any scenario, is it really worthy of the name? (On the other hand, if v5 can beat v4 in inference time or accuracy, that should be highlighted somewhere.)
FWIW I doubt anyone who looks into this will think roboflow had anything to do with the current controversies. You just showed off what someone else made, which is both legit and helpful. It's not like you were the ones that named it v5.
On the other hand... visiting https://models.roboflow.ai/ https://models.roboflow.ai/ does show YOLOv5 as "current SOTA", with some impressive-sounding results:
SIZE: YOLOv5 is about 88% smaller than YOLOv4 (27 MB vs 244 MB)
SPEED: YOLOv5 is about 180% faster than YOLOv4 (140 FPS vs 50 FPS)
ACCURACY: YOLOv5 is roughly as accurate as YOLOv4 on the same task (0.895 mAP vs 0.892 mAP)
Then it links to https://blog.roboflow.ai/yolov5-is-here/ https://blog.roboflow.ai/yolov5-is-here/ but there doesn't seem to be any clear chart showing "here's v5 performance vs v4 performance under these conditions: x, y, z"
Out of curiosity, where did the "180% faster" and 0.895 mAP vs 0.892 mAP numbers come from? Is there some way to reproduce those measurements?
The benchmarks at https://github.com/WongKinYiu/CrossStagePartialNetworks/issues/32#issuecomment-640887979 https://github.com/WongKinYiu/CrossStagePartialNetworks/issu... seem to show different results, with v4 coming out ahead in both accuracy and speed at 736x736 res. I'm not sure if they're using a standard benchmarking script though.
Thanks for gathering together what's currently known. The field does move fast.
- rocauc 6y agoAgreed! Crucially, we're tracking "out of the box" performance, e.g., if a developer grabbed X model and used it on a sample task, how could they expect it to perform? Further research and evaluation is recommended! For size, we measured the sizes of our saved weights files for Darknet YOLOv4 versus the PyTorch YOLOv5 implementation. For inference speed, we checked "out of the box" speed using a Colab Notebook equipped with a Tesla P100. We used the same task[1] for both - e.g. see the YOLOv5 Colab notebook[2]. For Darknet YOLOv4 inference speed, we translated the Darknet weights using the Ultralytics YOLOv3 repo (as we've seen many do for deployments)[3]. (To achieve top YOLOv4 inference speed, one should reconfigure Darknet carefully with OpenCV, CUDA, cuDNN, and carefully monitor batch size.) For accuracy, we evaluated the task above with mAP after quick training (100 epochs) with the smallest YOLOv5s model against the full YOLOv4 model (using recommended 2000*n, n is classes). Our example is a small custom dataset, and should be investigated on e.g. COCO. 90-classes. [1] https://public.roboflow.ai/object-detection/bccd https://public.roboflow.ai/object-detection/bccd [2] https://colab.research.google.com/drive/1gDZ2xcTOgR39tGGs-EZ6i3RTs16wmzZQ https://colab.research.google.com/drive/1gDZ2xcTOgR39tGGs-EZ... [3] https://github.com/ultralytics/yolov3 https://github.com/ultralytics/yolov3
- deleted 6y ago[deleted]
- bArray 6y agoThis is why I have so much doubt. To claim it's better in any meaningful way you need to show it on the same framework, varied datasets, varied input sizes and you should be able to use it in your detection problem and also see some benefits from the previous version. > SIZE: YOLOv5 is about 88% smaller than YOLOv4 (27 MB vs 244 MB) Is that a benefit of Darknet vs TF, YOLOv4 vs YOLOv5, or did you win the NN lottery [1]? > SPEED: YOLOv5 is about 180% faster than YOLOv4 (140 FPS vs 50 FPS) Again, where does this improvement come from? > ACCURACY: YOLOv5 is roughly as accurate as YOLOv4 on the same task (0.895 mAP vs 0.892 mAP) The difference in 0.1% accuracy can be huge, for example the difference between 99.9% and 100% could require an insanely larger neural network. Even much less that 99% accuracy, it seems clear to me that there can still be some limitations on accuracy from neural network size. For example, if you really don't care so much for accuracy, you can really squeeze the network down [2]. [1] https://ai.facebook.com/blog/understanding-the-generalization-of-lottery-tickets-in-neural-networks https://ai.facebook.com/blog/understanding-the-generalizatio... [2] https://arxiv.org/abs/1910.03159 https://arxiv.org/abs/1910.03159