3 ms·

Ask HN: Object Recognition for Photo Metadata

3 points by hyperific 4y ago

I have a lot of untagged photos on my hard drive, loosely organized in folders. My image collection is large enough now that the folder structure is getting too cumbersome. I know a lot of photographers use metadata tags but at this point my library is so large that it'd take me months to tag everything. I'm looking for a program that will iterate over my photos, detect from a wide variety of object types and subsequently append those object labels to the image's metadata as tags. Any ideas?

version_five 4y ago
If you have some software experience, look up detectron2 from Facebook. What you are describing would be very easy to do with their toolkit - they have a notebook that can get you started. But it assumes you can program in python
- hyperific 4y ago
  I'll check that out, thanks!
- hyperific 4y ago
  I've messed with Yolo a little bit for another project. Do you think that could be fruitful? Ideally I'd like to use something that's able to discern between fairly similar things. Like "hummingbird" vs "bird" or "man" / "woman" versus "person".
  - version_five 4y ago
    From what I understand, what's most important to you is having a model that's already trained on something, rather than the architecture. Yolo is probably fine, as would be some of the older ones. You should be able to find a model that's been pretrained on COCO - you can look at see what classes are included. I don't know if there are other broadly trained models available that will serve your purpose. What I'd do is just run your picture through a COCO trained object detection model and see if the annotations do what you want. Though backing up a bit, there are also image captioning models that may better do what you want to do for organizing your photos. I'm not really familiar with any - though I did come across BLIP the other day but I haven't used it: https://github.com/salesforce/BLIP https://github.com/salesforce/BLIP This may be a better way to get at what you want
RobotToaster 4y ago
Depending on how they were taken, it may not need to be that complicated. Most phones for instance embed GPS coordinates in photos metadata, which can be accessed by some photo organisation software. I'd be interested if anyone does know of anything that can do what you ask though, I know lightroom can recognise faces, but more organisation is always good.
- hyperific 4y ago
  Trouble is most of the photos were taken with a DSLR sans GPS module. I _do_ have plenty of metadata about the camera, lens, and exposure settings so that's something.
mikewarot 4y ago
I know you've probably already thought of this, but be sure to give the tags a signature, so you can revise them later when a newer, better model comes along, OR you do manual corrections.
- hyperific 4y ago
  I hadn't thought of that but that's a great idea. I see that being useful as I experiment with pre-trained vs custom models too.
tough 4y ago
Maybe something like facebookresearch/detic You can try it out in replicate: https://replicate.com/facebookresearch/detic https://replicate.com/facebookresearch/detic If you're not a software engineer, would you pay for this as an app? Might need to build it over the weekend lol
- hyperific 4y ago
  Ooo Detic looks like what I'm looking for. Being able to differentiate between subclasses of objects is key. If I was still working as a professional photographer (and if I didn't know anything about programming) I'd pay for this kind of thing for sure. I have enough experience in Python to know it's possible but I didn't want to reinvent the wheel, so to speak.
robertmp 4y ago
kenan01 4y ago
kenan01 4y ago
Calvin Davis Jr. https://open.spotify.com/artist/4Qh8jIwyRd1ejIHdQCBrG4?utm_source=samsung&utm_medium=keyboard https://open.spotify.com/artist/4Qh8jIwyRd1ejIHdQCBrG4?utm_s...

3 points by hyperific 4y ago

version_five 4y ago

If you have some software experience, look up detectron2 from Facebook. What you are describing would be very easy to do with their toolkit - they have a notebook that can get you started. But it assumes you can program in python

hyperific 4y ago
I'll check that out, thanks!
hyperific 4y ago
I've messed with Yolo a little bit for another project. Do you think that could be fruitful? Ideally I'd like to use something that's able to discern between fairly similar things. Like "hummingbird" vs "bird" or "man" / "woman" versus "person".
- version_five 4y ago
  From what I understand, what's most important to you is having a model that's already trained on something, rather than the architecture. Yolo is probably fine, as would be some of the older ones. You should be able to find a model that's been pretrained on COCO - you can look at see what classes are included. I don't know if there are other broadly trained models available that will serve your purpose. What I'd do is just run your picture through a COCO trained object detection model and see if the annotations do what you want. Though backing up a bit, there are also image captioning models that may better do what you want to do for organizing your photos. I'm not really familiar with any - though I did come across BLIP the other day but I haven't used it: https://github.com/salesforce/BLIP https://github.com/salesforce/BLIP This may be a better way to get at what you want

RobotToaster 4y ago

Depending on how they were taken, it may not need to be that complicated. Most phones for instance embed GPS coordinates in photos metadata, which can be accessed by some photo organisation software. I'd be interested if anyone does know of anything that can do what you ask though, I know lightroom can recognise faces, but more organisation is always good.

hyperific 4y ago
Trouble is most of the photos were taken with a DSLR sans GPS module. I _do_ have plenty of metadata about the camera, lens, and exposure settings so that's something.

mikewarot 4y ago

I know you've probably already thought of this, but be sure to give the tags a signature, so you can revise them later when a newer, better model comes along, OR you do manual corrections.

hyperific 4y ago
I hadn't thought of that but that's a great idea. I see that being useful as I experiment with pre-trained vs custom models too.

tough 4y ago

Maybe something like facebookresearch/detic You can try it out in replicate: https://replicate.com/facebookresearch/detic https://replicate.com/facebookresearch/detic If you're not a software engineer, would you pay for this as an app? Might need to build it over the weekend lol

hyperific 4y ago
Ooo Detic looks like what I'm looking for. Being able to differentiate between subclasses of objects is key. If I was still working as a professional photographer (and if I didn't know anything about programming) I'd pay for this kind of thing for sure. I have enough experience in Python to know it's possible but I didn't want to reinvent the wheel, so to speak.

robertmp 4y ago

kenan01 4y ago

Calvin Davis Jr. https://open.spotify.com/artist/4Qh8jIwyRd1ejIHdQCBrG4?utm_source=samsung&utm_medium=keyboard https://open.spotify.com/artist/4Qh8jIwyRd1ejIHdQCBrG4?utm_s...