4 ms·

Suppose someone wanted to train a model to identify which decade a photo was taken in. What would be the current SOTA architecture for that type of task? (Suppo

Suppose someone wanted to train a model to identify which decade a photo was taken in. What would be the current SOTA architecture for that type of task? (Suppose also that you had a few million labeled examples.) I like yolo because it’s a production grade object defector. It seems harder to find a production grade classifier. One amusing but dumb idea would be to use yolo for this: train the model on “photo from 1930,” “photo from 1940,” etc, where the bounding boxes cover the entire photo. But I’m curious what the professional solution might be.

sangfroid_bio 6y ago
Depends on whether you are looking for effectiveness vs efficiency. YOLO et al. are optimised for general feature extraction. They are ridiculously good if you need to build something quickly. If you are looking for the most efficient way e.g. lowest processing power per image then use a network that is optimised for the particular feature you are trying to extract.
- sillysaurusx 6y ago
  Highest accuracy. I was mainly wondering which model ML engineers reach for circa 2020 for classification tasks. Searching for classifiers brings up article after article presenting toy classifiers, but very little with a production focus.
  - dontreact 6y ago
    I've seen people get good results with Efficientnet, and it scales to be quite large. In general, I look at whatever the best state of the art is on imagenet, but then look for implementations that look easy to use among the top few.
rocauc 6y ago
I am guilty of using your dumb idea with satisfactory performance. I'd recommend EfficientNet (or one of its may variations) for an off-the-shelf SOTA classifier. https://paperswithcode.com/sota/image-classification-on-imagenet https://paperswithcode.com/sota/image-classification-on-imag...
modeless 6y ago
Easy, just use an ImageNet classifier architecture with one category for each decade. No need to bother with object detection at all. You could fine-tune or train from scratch. Image classification is probably the single most-researched task with the widest variety of models available. You could select an architecture based on ease of implementation, efficiency, absolute accuracy, or any combination. Papers With Code has great lists of state-of-the-art models. Take your pick: https://paperswithcode.com/sota/image-classification-on-imagenet https://paperswithcode.com/sota/image-classification-on-imag...
ericjang 6y ago
Out of curiosity, what would such a model be used for? I worry that as ML is becoming more popular and powerful, people are jumping to use it on problems from which the answer cannot possibly be identified accurately from the inputs alone. This model's predictions would entirely based on historical trends of what images looked like, rather than something objective like carbon dating. I am not discouraging someone from building such a model, but it would be really helpful to know the context for which such a model is being developed. If it's just a hobby investigation, it would be cool to see how "predictable" dates are from images. I could even see it being used in forensics to provide a "first guess" as to when an image occurred, and helping with triaging of evidence. However, things become deeply problematic if the result from the image is fed to people as "ground truth" simply because the model was found to be accurate on a validation dataset. I certainly wouldn't want this model to be used to determine whether a suspect is innocent / guilty, or to be used naively by museums to date photographs.
- theplague42 6y ago
  For example, creating a scrapbook of somebody's life from scanned photographs that lack any metadata.
- taneq 6y ago
  Could be useful for identifying faked historical photos?
site-packages1 6y ago
I would probably first start with examples from different eras and try to build a statistical classifier on it without resorting to detectors. Those seem like big overkill for such a task.
nl 6y ago
Yolo isn't really suitable for classification - it's an object detector. For maximum accuracy there are high-quality well tested ResNet128 or ResNet152 implementations for PyTorch[1] and TensorFlow[2] that most people would use as a basis for classification tasks where the highest accuracy is needed. Lower quality ResNets (eg ResNet50) run a lot faster. Note that in this post, PP-YOLO replaces the older YOLO backbone with a ResNet50 (ResNet50-vd-dcn to be precise) backbone. EfficientNet is another good option[3], as is SE-ResNet. For this specific task it's not entirely clear what/how the classification is supposed to work though. The source of the photos is really, really important: if they are physical photos then the scanner used is important. And things like different film have different tone, and storage of physical photos matters a lot. [1] https://pytorch.org/hub/pytorch_vision_resnet/ https://pytorch.org/hub/pytorch_vision_resnet/ [2] https://tfhub.dev/google/imagenet/resnet_v2_152/feature_vector/4 https://tfhub.dev/google/imagenet/resnet_v2_152/feature_vect... [3] https://tfhub.dev/google/collections/efficientnet/1 https://tfhub.dev/google/collections/efficientnet/1
p1esk 6y ago
It depends on if you're OK with the classifier overfitting on potentially meaningless details (like a shade of color of the paper most commonly used in that decade, etc) or if you actually want to classify the images based on content (style of clothes, etc).

sangfroid_bio 6y ago

Depends on whether you are looking for effectiveness vs efficiency. YOLO et al. are optimised for general feature extraction. They are ridiculously good if you need to build something quickly. If you are looking for the most efficient way e.g. lowest processing power per image then use a network that is optimised for the particular feature you are trying to extract.

sillysaurusx 6y ago
Highest accuracy. I was mainly wondering which model ML engineers reach for circa 2020 for classification tasks. Searching for classifiers brings up article after article presenting toy classifiers, but very little with a production focus.
- dontreact 6y ago
  I've seen people get good results with Efficientnet, and it scales to be quite large. In general, I look at whatever the best state of the art is on imagenet, but then look for implementations that look easy to use among the top few.

rocauc 6y ago

I am guilty of using your dumb idea with satisfactory performance. I'd recommend EfficientNet (or one of its may variations) for an off-the-shelf SOTA classifier. https://paperswithcode.com/sota/image-classification-on-imagenet https://paperswithcode.com/sota/image-classification-on-imag...

modeless 6y ago

Easy, just use an ImageNet classifier architecture with one category for each decade. No need to bother with object detection at all. You could fine-tune or train from scratch. Image classification is probably the single most-researched task with the widest variety of models available. You could select an architecture based on ease of implementation, efficiency, absolute accuracy, or any combination. Papers With Code has great lists of state-of-the-art models. Take your pick: https://paperswithcode.com/sota/image-classification-on-imagenet https://paperswithcode.com/sota/image-classification-on-imag...

ericjang 6y ago

Out of curiosity, what would such a model be used for? I worry that as ML is becoming more popular and powerful, people are jumping to use it on problems from which the answer cannot possibly be identified accurately from the inputs alone. This model's predictions would entirely based on historical trends of what images looked like, rather than something objective like carbon dating. I am not discouraging someone from building such a model, but it would be really helpful to know the context for which such a model is being developed. If it's just a hobby investigation, it would be cool to see how "predictable" dates are from images. I could even see it being used in forensics to provide a "first guess" as to when an image occurred, and helping with triaging of evidence. However, things become deeply problematic if the result from the image is fed to people as "ground truth" simply because the model was found to be accurate on a validation dataset. I certainly wouldn't want this model to be used to determine whether a suspect is innocent / guilty, or to be used naively by museums to date photographs.

theplague42 6y ago
For example, creating a scrapbook of somebody's life from scanned photographs that lack any metadata.
taneq 6y ago
Could be useful for identifying faked historical photos?

site-packages1 6y ago

I would probably first start with examples from different eras and try to build a statistical classifier on it without resorting to detectors. Those seem like big overkill for such a task.

nl 6y ago

Yolo isn't really suitable for classification - it's an object detector. For maximum accuracy there are high-quality well tested ResNet128 or ResNet152 implementations for PyTorch[1] and TensorFlow[2] that most people would use as a basis for classification tasks where the highest accuracy is needed. Lower quality ResNets (eg ResNet50) run a lot faster. Note that in this post, PP-YOLO replaces the older YOLO backbone with a ResNet50 (ResNet50-vd-dcn to be precise) backbone. EfficientNet is another good option[3], as is SE-ResNet. For this specific task it's not entirely clear what/how the classification is supposed to work though. The source of the photos is really, really important: if they are physical photos then the scanner used is important. And things like different film have different tone, and storage of physical photos matters a lot. [1] https://pytorch.org/hub/pytorch_vision_resnet/ https://pytorch.org/hub/pytorch_vision_resnet/ [2] https://tfhub.dev/google/imagenet/resnet_v2_152/feature_vector/4 https://tfhub.dev/google/imagenet/resnet_v2_152/feature_vect... [3] https://tfhub.dev/google/collections/efficientnet/1 https://tfhub.dev/google/collections/efficientnet/1

p1esk 6y ago

It depends on if you're OK with the classifier overfitting on potentially meaningless details (like a shade of color of the paper most commonly used in that decade, etc) or if you actually want to classify the images based on content (style of clothes, etc).