5 ms·

I'm just going to call this out as bullshit. This isn't YOLOv5. I doubt they even did a proper comparison between their model and YOLOv4. Someone asked it to n

by bArray 6y ago

I'm just going to call this out as bullshit. This isn't YOLOv5. I doubt they even did a proper comparison between their model and YOLOv4. Someone asked it to not be called YOLOv5 and their response was just awful [1]. They also blew off a request to publish a blog/paper detailing the network [2]. I filed a ticket to get to the bottom of this with the creators of YOLOv4: https://github.com/AlexeyAB/darknet/issues/5920 https://github.com/AlexeyAB/darknet/issues/5920 [1] https://github.com/ultralytics/yolov5/issues/2 https://github.com/ultralytics/yolov5/issues/2 [2] https://github.com/ultralytics/yolov5/issues/4 https://github.com/ultralytics/yolov5/issues/4

rcpt 6y ago
I love that the response to them is "you can you up,no can no bb" Learned a new phrase today.
- catalogia 6y ago
  Can you explain it? I can't figure out what that means.
  - FriendlyNormie 6y ago
    It means teaching pajeets how to use computers was a mistake.
  - arctangent 6y ago
    Apparently it is Chinese internet slang meaning: "If you can do it, then you go and do it. If you can’t do it, then don’t criticise others." via: http://www.chinesetimeschool.com/zh-cn/articles/chinese-internet-slang-u-can-u-up/ http://www.chinesetimeschool.com/zh-cn/articles/chinese-inte...
  - kbenson 6y ago
    Just found these.[1][2] That is pretty awful, if it's from a dev. Edit: Although as yeldarb explains in a comment here[3], it's probably a bit more complicated than that. 1: https://www.urbandictionary.com/define.php?term=you%20can%20you%20up https://www.urbandictionary.com/define.php?term=you%20can%20... 2: https://www.quora.com/Whats-the-meaning-of-you-can-you-up-no-can-no-bibi https://www.quora.com/Whats-the-meaning-of-you-can-you-up-no... 3: https://news.ycombinator.com/item?id=23478983 https://news.ycombinator.com/item?id=23478983
    - bArray 6y ago
      > Edit: Although as yeldarb explains in a comment here[3], > it's probably a bit more complicated than that. Legally speaking I'm not sure anything wrong was really done here. Morally speaking, it seems quite unethical. AlexeyAB has really been carrying the torch of the Darknet framework and the YOLO neural network for quite some time (with pjreddie effectively handing it over to him). AlexeyAB has been providing support on pjreddie's abandoned repository (e.g. [1]) and actively working on improvements in a fork [2]. If you look at the contributors graphs, he really has been keeping the project alive [3] (vs Darknet by pjreddie [4]). Probably the worse part in my opinion is that they have also seemingly bypassed the open source nature of the project. This is quite damning. [1] https://github.com/pjreddie/darknet/issues/1900 https://github.com/pjreddie/darknet/issues/1900 [2] https://github.com/AlexeyAB/darknet https://github.com/AlexeyAB/darknet [3] https://github.com/AlexeyAB/darknet/graphs/contributors https://github.com/AlexeyAB/darknet/graphs/contributors [4] https://github.com/pjreddie/darknet/graphs/contributors https://github.com/pjreddie/darknet/graphs/contributors
      - kbenson 6y ago
        So, the question I have is whether AlexeyAB got some sort of endorsement from pjreddie, or if they just took over the name by nature of being the most active fork? If it's the latter, ultralytics' actions don't seem quite as bad (although they still feel kind of off-putting, especially with how some of the responses to calls for a name change were formulated). I guess given the info I have now, to me it boils down to whether there's precedent for the next version of the name to be taken by whoever is doing work on it? If the original author never endorsed AlexeyAB (I don't know one way or another), then perhaps AlexeyAB should have changed the name but references or payed homage to YOLO in some way? Eh, this is all starting to feel a bit too close to youtube drama for my liking.
        nerderloo 6y ago
        AlexeyAB seems to have gotten endorsement from pjreddie: https://github.com/AlexeyAB/darknet/issues/5920#issuecomment-642244531 https://github.com/AlexeyAB/darknet/issues/5920#issuecomment...
- whoevercares 6y ago
  你行你上啊不行别bb（bb=trashtalking/non-favorable comments） This is literally trash talking Slang in Chinese, because this field is full of young bloated researchers who forget their last name
  - joshvm 6y ago
    > who forget their last name I've not heard that one before either. Is it a reference to the Dark Tower? ("[he] has forgotten the face of his father") or did Stephen King borrow it from somewhere else?
    - whoevercares 6y ago
      This is an old punchline in China for many years and I doubt it comes from English literature. I guess the meaning is similar (last name ~= name of the father) Edit: obviously I should google dark power first lol.
      - joshvm 6y ago
        Also a slight edit, I wrote name initially. Of course in the books it's "face of his father", but it still sounds similar [1]. To admit to forgetting the face of one's father is to be deeply shameful, to accuse someone of it is insinuating they should be ashamed of themselves. Can you write it in Chinese? [1] https://www.goodreads.com/quotes/12991-i-do-not-aim-with-my-hand-he-who-aims https://www.goodreads.com/quotes/12991-i-do-not-aim-with-my-...
        whoevercares 6y ago
        “不知道自己姓什么了”
kuzee 6y ago
Looks like ultralytics, not roboflow, is the one that named this model v5. Different people/companies.
- bArray 6y ago
  Yep, I updated my GitHub comment with respect to what @josephofiowa said. I made an assumptions when seeing the same PR images/language being used.
sillysaurusx 6y ago
There are some benchmarks here: https://github.com/WongKinYiu/CrossStagePartialNetworks/issues/32#issuecomment-640887979 https://github.com/WongKinYiu/CrossStagePartialNetworks/issu... It's hard to interpret benchmarks in a fair way, but it's sort of sounding like YOLOv4 might be superior to YOLOv5, at least for certain resolutions. Does YOLOv5 outperform YOLOv4 at all? Faster inference time or higher accuracy?
rocauc 6y ago
Hey all - OP here. We're not affiliated with Ultralytics or the other researchers. We're a startup that enables developers to use computer vision without being machine learning experts, and we support a wide array of open source model architectures for teams to try on their data: https://models.roboflow.ai https://models.roboflow.ai Beyond that, we're just fans. We're amazed by how quickly the field is moving and we did some benchmarks that we thought other people might find as exciting as we did. I don't want to take a side in the naming controversy. Our core focus is helping developers get data into any model, regardless of its name!
- sillysaurusx 6y ago
  YOLOv5 seems to have one important advantage over v4, which your post helped highlight: Fourth, YOLOv5 is small. Specifically, a weights file for YOLOv5 is 27 megabytes. Our weights file for YOLOv4 (with Darknet architecture) is 244 megabytes. YOLOv5 is nearly 90 percent smaller than YOLOv4. This means YOLOv5 can be deployed to embedded devices much more easily. Naming controversy aside, it's nice to have some model that can get close to the same accuracy at 10% of the size. Naming it v5 was certainly ... bold ... though. If it can't outperform v4 in any scenario, is it really worthy of the name? (On the other hand, if v5 can beat v4 in inference time or accuracy, that should be highlighted somewhere.) FWIW I doubt anyone who looks into this will think roboflow had anything to do with the current controversies. You just showed off what someone else made, which is both legit and helpful. It's not like you were the ones that named it v5. On the other hand... visiting https://models.roboflow.ai/ https://models.roboflow.ai/ does show YOLOv5 as "current SOTA", with some impressive-sounding results: SIZE: YOLOv5 is about 88% smaller than YOLOv4 (27 MB vs 244 MB) SPEED: YOLOv5 is about 180% faster than YOLOv4 (140 FPS vs 50 FPS) ACCURACY: YOLOv5 is roughly as accurate as YOLOv4 on the same task (0.895 mAP vs 0.892 mAP) Then it links to https://blog.roboflow.ai/yolov5-is-here/ https://blog.roboflow.ai/yolov5-is-here/ but there doesn't seem to be any clear chart showing "here's v5 performance vs v4 performance under these conditions: x, y, z" Out of curiosity, where did the "180% faster" and 0.895 mAP vs 0.892 mAP numbers come from? Is there some way to reproduce those measurements? The benchmarks at https://github.com/WongKinYiu/CrossStagePartialNetworks/issues/32#issuecomment-640887979 https://github.com/WongKinYiu/CrossStagePartialNetworks/issu... seem to show different results, with v4 coming out ahead in both accuracy and speed at 736x736 res. I'm not sure if they're using a standard benchmarking script though. Thanks for gathering together what's currently known. The field does move fast.
quietbritishjim 6y ago
> Someone asked it to not be called YOLOv5 and their response was just awful [1] I don't see any response by them at all. Do you mean the comment by WDNMD0-0? I can't see any reason to believe they're connected to the company, have I missed something?
joshvm 6y ago
I somewhat agree on the naming issue. I don't think yolov5 is semantically very informative. But by the way, if you read the issues from a while back you'll see that AlexeyAB's fork basically scooped them, hence the version bump. Ultralytics probably would have called this Yolov4 otherwise. This repo has been in the works for a while. For history, Ultralytics originally forked the core code from some other Pytorch implementation which was inference-only. Their claim to fame is that they were the first to get training to work in Pytorch. This took a while, probably because there is actually very little documentation for Yolov3 and there was confusion over what the loss function actually ought to be. The darknet repo is totally uncommented C with lots of single letter variable names. AlexeyAB is a Saint. That said, should it be a totally new name? The changes are indeed relatively minor in terms of architecture, it's still yolo underneath (in fact I think the classification/regression head is pretty much unchanged). The v4 release was also quite contentious. Actually their previous models used to be called yolov3-spp-ultralytics. Probably I would have gone with efficient-yolo or something similar. That's no worse than fast/faster rcnn. I disagree on your second point though. Demanding a paper when the author says "we will later" is hardly a blow off. Publishing and writing takes time. The code is open source, the implementation is there. How many times does it happen the other way around? And before we knock Glenn for this, as far as I know, he's running a business, not a research group. Disclosure: I've contributed (in minor ways) to both this repository and Alexey's darknet fork. I use both regularly for work and I would say I'm familiar enough with both codebases. I mostly ignore the benchmarks because performance on coco is meaningless for performance on custom data. I'm not affiliated with either group, in case it's not clear.
- nerderloo 6y ago
  Didn't AlexeyAB endorse YOLOv4 though? Did he also endorse YOLOv5?
  - joshvm 6y ago
    AlexeyAB is the primary author on YOLOv4, and the darknet maintainer, so yes. This is pretty much the official word on the matter: https://github.com/AlexeyAB/darknet/issues/5920#issuecomment-642244531 https://github.com/AlexeyAB/darknet/issues/5920#issuecomment... Despite that, there was still a lot of controversy over the decision to call it v4. See that thread for the discussion on v5 and you can make your own judgement.

by bArray 6y ago

rcpt 6y ago

I love that the response to them is "you can you up,no can no bb" Learned a new phrase today.

catalogia 6y ago
Can you explain it? I can't figure out what that means.
- FriendlyNormie 6y ago
  It means teaching pajeets how to use computers was a mistake.
- arctangent 6y ago
  Apparently it is Chinese internet slang meaning: "If you can do it, then you go and do it. If you can’t do it, then don’t criticise others." via: http://www.chinesetimeschool.com/zh-cn/articles/chinese-internet-slang-u-can-u-up/ http://www.chinesetimeschool.com/zh-cn/articles/chinese-inte...
- kbenson 6y ago
  Just found these.[1][2] That is pretty awful, if it's from a dev. Edit: Although as yeldarb explains in a comment here[3], it's probably a bit more complicated than that. 1: https://www.urbandictionary.com/define.php?term=you%20can%20you%20up https://www.urbandictionary.com/define.php?term=you%20can%20... 2: https://www.quora.com/Whats-the-meaning-of-you-can-you-up-no-can-no-bibi https://www.quora.com/Whats-the-meaning-of-you-can-you-up-no... 3: https://news.ycombinator.com/item?id=23478983 https://news.ycombinator.com/item?id=23478983
  - bArray 6y ago
    > Edit: Although as yeldarb explains in a comment here[3], > it's probably a bit more complicated than that. Legally speaking I'm not sure anything wrong was really done here. Morally speaking, it seems quite unethical. AlexeyAB has really been carrying the torch of the Darknet framework and the YOLO neural network for quite some time (with pjreddie effectively handing it over to him). AlexeyAB has been providing support on pjreddie's abandoned repository (e.g. [1]) and actively working on improvements in a fork [2]. If you look at the contributors graphs, he really has been keeping the project alive [3] (vs Darknet by pjreddie [4]). Probably the worse part in my opinion is that they have also seemingly bypassed the open source nature of the project. This is quite damning. [1] https://github.com/pjreddie/darknet/issues/1900 https://github.com/pjreddie/darknet/issues/1900 [2] https://github.com/AlexeyAB/darknet https://github.com/AlexeyAB/darknet [3] https://github.com/AlexeyAB/darknet/graphs/contributors https://github.com/AlexeyAB/darknet/graphs/contributors [4] https://github.com/pjreddie/darknet/graphs/contributors https://github.com/pjreddie/darknet/graphs/contributors
    - kbenson 6y ago
      So, the question I have is whether AlexeyAB got some sort of endorsement from pjreddie, or if they just took over the name by nature of being the most active fork? If it's the latter, ultralytics' actions don't seem quite as bad (although they still feel kind of off-putting, especially with how some of the responses to calls for a name change were formulated). I guess given the info I have now, to me it boils down to whether there's precedent for the next version of the name to be taken by whoever is doing work on it? If the original author never endorsed AlexeyAB (I don't know one way or another), then perhaps AlexeyAB should have changed the name but references or payed homage to YOLO in some way? Eh, this is all starting to feel a bit too close to youtube drama for my liking.
      - nerderloo 6y ago
        AlexeyAB seems to have gotten endorsement from pjreddie: https://github.com/AlexeyAB/darknet/issues/5920#issuecomment-642244531 https://github.com/AlexeyAB/darknet/issues/5920#issuecomment...
whoevercares 6y ago
你行你上啊不行别bb（bb=trashtalking/non-favorable comments） This is literally trash talking Slang in Chinese, because this field is full of young bloated researchers who forget their last name
- joshvm 6y ago
  > who forget their last name I've not heard that one before either. Is it a reference to the Dark Tower? ("[he] has forgotten the face of his father") or did Stephen King borrow it from somewhere else?
  - whoevercares 6y ago
    This is an old punchline in China for many years and I doubt it comes from English literature. I guess the meaning is similar (last name ~= name of the father) Edit: obviously I should google dark power first lol.
    - joshvm 6y ago
      Also a slight edit, I wrote name initially. Of course in the books it's "face of his father", but it still sounds similar [1]. To admit to forgetting the face of one's father is to be deeply shameful, to accuse someone of it is insinuating they should be ashamed of themselves. Can you write it in Chinese? [1] https://www.goodreads.com/quotes/12991-i-do-not-aim-with-my-hand-he-who-aims https://www.goodreads.com/quotes/12991-i-do-not-aim-with-my-...
      - whoevercares 6y ago
        “不知道自己姓什么了”

kuzee 6y ago

Looks like ultralytics, not roboflow, is the one that named this model v5. Different people/companies.

bArray 6y ago
Yep, I updated my GitHub comment with respect to what @josephofiowa said. I made an assumptions when seeing the same PR images/language being used.

sillysaurusx 6y ago

There are some benchmarks here: https://github.com/WongKinYiu/CrossStagePartialNetworks/issues/32#issuecomment-640887979 https://github.com/WongKinYiu/CrossStagePartialNetworks/issu... It's hard to interpret benchmarks in a fair way, but it's sort of sounding like YOLOv4 might be superior to YOLOv5, at least for certain resolutions. Does YOLOv5 outperform YOLOv4 at all? Faster inference time or higher accuracy?

rocauc 6y ago

Hey all - OP here. We're not affiliated with Ultralytics or the other researchers. We're a startup that enables developers to use computer vision without being machine learning experts, and we support a wide array of open source model architectures for teams to try on their data: https://models.roboflow.ai https://models.roboflow.ai Beyond that, we're just fans. We're amazed by how quickly the field is moving and we did some benchmarks that we thought other people might find as exciting as we did. I don't want to take a side in the naming controversy. Our core focus is helping developers get data into any model, regardless of its name!

sillysaurusx 6y ago
YOLOv5 seems to have one important advantage over v4, which your post helped highlight: Fourth, YOLOv5 is small. Specifically, a weights file for YOLOv5 is 27 megabytes. Our weights file for YOLOv4 (with Darknet architecture) is 244 megabytes. YOLOv5 is nearly 90 percent smaller than YOLOv4. This means YOLOv5 can be deployed to embedded devices much more easily. Naming controversy aside, it's nice to have some model that can get close to the same accuracy at 10% of the size. Naming it v5 was certainly ... bold ... though. If it can't outperform v4 in any scenario, is it really worthy of the name? (On the other hand, if v5 can beat v4 in inference time or accuracy, that should be highlighted somewhere.) FWIW I doubt anyone who looks into this will think roboflow had anything to do with the current controversies. You just showed off what someone else made, which is both legit and helpful. It's not like you were the ones that named it v5. On the other hand... visiting https://models.roboflow.ai/ https://models.roboflow.ai/ does show YOLOv5 as "current SOTA", with some impressive-sounding results: SIZE: YOLOv5 is about 88% smaller than YOLOv4 (27 MB vs 244 MB) SPEED: YOLOv5 is about 180% faster than YOLOv4 (140 FPS vs 50 FPS) ACCURACY: YOLOv5 is roughly as accurate as YOLOv4 on the same task (0.895 mAP vs 0.892 mAP) Then it links to https://blog.roboflow.ai/yolov5-is-here/ https://blog.roboflow.ai/yolov5-is-here/ but there doesn't seem to be any clear chart showing "here's v5 performance vs v4 performance under these conditions: x, y, z" Out of curiosity, where did the "180% faster" and 0.895 mAP vs 0.892 mAP numbers come from? Is there some way to reproduce those measurements? The benchmarks at https://github.com/WongKinYiu/CrossStagePartialNetworks/issues/32#issuecomment-640887979 https://github.com/WongKinYiu/CrossStagePartialNetworks/issu... seem to show different results, with v4 coming out ahead in both accuracy and speed at 736x736 res. I'm not sure if they're using a standard benchmarking script though. Thanks for gathering together what's currently known. The field does move fast.

quietbritishjim 6y ago

> Someone asked it to not be called YOLOv5 and their response was just awful [1] I don't see any response by them at all. Do you mean the comment by WDNMD0-0? I can't see any reason to believe they're connected to the company, have I missed something?

joshvm 6y ago

I somewhat agree on the naming issue. I don't think yolov5 is semantically very informative. But by the way, if you read the issues from a while back you'll see that AlexeyAB's fork basically scooped them, hence the version bump. Ultralytics probably would have called this Yolov4 otherwise. This repo has been in the works for a while. For history, Ultralytics originally forked the core code from some other Pytorch implementation which was inference-only. Their claim to fame is that they were the first to get training to work in Pytorch. This took a while, probably because there is actually very little documentation for Yolov3 and there was confusion over what the loss function actually ought to be. The darknet repo is totally uncommented C with lots of single letter variable names. AlexeyAB is a Saint. That said, should it be a totally new name? The changes are indeed relatively minor in terms of architecture, it's still yolo underneath (in fact I think the classification/regression head is pretty much unchanged). The v4 release was also quite contentious. Actually their previous models used to be called yolov3-spp-ultralytics. Probably I would have gone with efficient-yolo or something similar. That's no worse than fast/faster rcnn. I disagree on your second point though. Demanding a paper when the author says "we will later" is hardly a blow off. Publishing and writing takes time. The code is open source, the implementation is there. How many times does it happen the other way around? And before we knock Glenn for this, as far as I know, he's running a business, not a research group. Disclosure: I've contributed (in minor ways) to both this repository and Alexey's darknet fork. I use both regularly for work and I would say I'm familiar enough with both codebases. I mostly ignore the benchmarks because performance on coco is meaningless for performance on custom data. I'm not affiliated with either group, in case it's not clear.

nerderloo 6y ago
Didn't AlexeyAB endorse YOLOv4 though? Did he also endorse YOLOv5?
- joshvm 6y ago
  AlexeyAB is the primary author on YOLOv4, and the darknet maintainer, so yes. This is pretty much the official word on the matter: https://github.com/AlexeyAB/darknet/issues/5920#issuecomment-642244531 https://github.com/AlexeyAB/darknet/issues/5920#issuecomment... Despite that, there was still a lot of controversy over the decision to call it v4. See that thread for the discussion on v5 and you can make your own judgement.