2 ms·

Though I haven't effort into analyzing or tweaking the training, some of the perditions does look like it points at the shadow instead of the tip. This syntheti

Though I haven't effort into analyzing or tweaking the training, some of the perditions does look like it points at the shadow instead of the tip. This synthetic dataset is tiny (not even 100k) - but still larger than what I could find. I think there's a lot of room for improvement. What do you think is the most important requirement?

godelski 3y ago
That's all fair, and I'm a big believer in context for proper evaluation. My issue is really about the presentation. There's a pervasive issue in research that exaggeration compounds as if one work exaggerates its effectiveness that subsequent works have a higher bar to meet to show improvements and the frequent result is additional exaggeration (even in just small amounts). > What do you think is the most important requirement? Data quality and augmentation. I looked at the dataset[0] and the quality is very poor. There is very low diversity and it appears that it is composed of a few images and increased through augmentation. Augmentation should not be handled through dataset generation but dataset processing. Torch will even move bounding box keypoints when cropping or resizing images. Data quality is one of the most important factors and I know it is common to just get labels to Turk or cheap services but you absolutely get what you pay for. [0] Please don't harvest emails and usernames to require downloading. I created a dummy account for this because that's absolutely not acceptable. DO NOT SEND PEOPLE SPAM. DO NOT HARVEST EMAILS AND SELL THOSE EMAILS. I happily flag emails that come to my account through means like this until Google identifies all emails from that sender as spam. DO NOT DO THIS. It will just make people hate you and build a devoted base against you. There is no justification for this. You should absolutely rescind the gate and delete all emails you harvested. Your licensing is already an agreement. Don't be shady.

godelski 3y ago

That's all fair, and I'm a big believer in context for proper evaluation. My issue is really about the presentation. There's a pervasive issue in research that exaggeration compounds as if one work exaggerates its effectiveness that subsequent works have a higher bar to meet to show improvements and the frequent result is additional exaggeration (even in just small amounts). > What do you think is the most important requirement? Data quality and augmentation. I looked at the dataset[0] and the quality is very poor. There is very low diversity and it appears that it is composed of a few images and increased through augmentation. Augmentation should not be handled through dataset generation but dataset processing. Torch will even move bounding box keypoints when cropping or resizing images. Data quality is one of the most important factors and I know it is common to just get labels to Turk or cheap services but you absolutely get what you pay for. [0] Please don't harvest emails and usernames to require downloading. I created a dummy account for this because that's absolutely not acceptable. DO NOT SEND PEOPLE SPAM. DO NOT HARVEST EMAILS AND SELL THOSE EMAILS. I happily flag emails that come to my account through means like this until Google identifies all emails from that sender as spam. DO NOT DO THIS. It will just make people hate you and build a devoted base against you. There is no justification for this. You should absolutely rescind the gate and delete all emails you harvested. Your licensing is already an agreement. Don't be shady.