12 ms·

Just to add context - I do have backups, but they're spread across a myriad of different hard drives. I had used Amazon Photos to consolidate, and benefit from

Just to add context - I do have backups, but they're spread across a myriad of different hard drives. I had used Amazon Photos to consolidate, and benefit from the facial recognition to be able to quickly locate pictures of family members quickly. I think that I'll give DigiKam another go, against the NAS loaded up. Thanks for the comments!

capableweb 4y ago
> I think that I'll give DigiKam another go, against the NAS loaded up. I'm also looking for something FOSS that can do basic face recognition + maybe even more, but last time I checked DigiKam's detection didn't work so well, or maybe I got spoiled by the detection in Google Photos. If you do give it a go, would be nice if you reported back on your experience :)
- komali2 4y ago
  FOSS photo software with face recognition (note that some of this is off-premise face recognition, sometimes requiring an API key you get on your own) https://damselfly.info https://damselfly.info https://github.com/LibrePhotos/librephotos https://github.com/LibrePhotos/librephotos https://photoview.github.io/ https://photoview.github.io/ I may have missed some: https://github.com/awesome-selfhosted/awesome-selfhosted#photo-and-video-galleries https://github.com/awesome-selfhosted/awesome-selfhosted#pho...
  - capableweb 4y ago
    What I found in my previous search is that either they're really good at managing large collections of photos, or they're really good at doing various AI/recognition, but none of what I've found have been good at both. Hence my interest in DigiKam as their management features for large collections is second to none.
    - CWuestefeld 4y ago
      Isn't that OK? You should be able to find some reasonable workflow around using the recognizer to tag the photo in its EXIF/IPTC metadata, and then load that into the organizer.
      - capableweb 4y ago
        It's not not OK :) Just not what I'm looking for, I spend enough time professionally with writing glue code, that I'm looking for a good out-of-the-box experience for something like this.
    - specto 4y ago
      I think immich will get there eventually
- wh33zle 4y ago
  Not sure if it has been mentioned already but photoprism added face recognition a few releases ago and it is working well for me: https://github.com/photoprism/photoprism https://github.com/photoprism/photoprism The quality of this software is overall extremely good. It is a solo-developer as well so you might want to consider sponsoring them if you end up using it. I am not affiliated, just a happy user :)
  - thedanbob 4y ago
    Seconding photoprism. The face recognition was pretty meh when it was first introduced, more of a gimmick than anything. But (I assume) a recent update pushed it into "wow, this is actually really good" territory.
    - domh 4y ago
      Thirding photoprism, got it hosted on a raspberry pi and made it accessible outside of my home via Tailscale. Seems to work fairly well... though I could probably make it faster with an SSD vs an HDD over USB.
  - Cyph0n 4y ago
    Photoprism is absolutely amazing. It was the quickest donation I’ve ever made to an OSS project. I have a Photoprism instance running on my home server backed by a raidz1 ZFS pool (1st backup). Photos are periodically synced to a Backblaze bucket with versioning enabled (2nd backup). The source of most photos is an iOS device with iCloud enabled (main copy). I rely on PhotoSync to periodically sync from the iOS device to Photoprism.
  - philsnow 4y ago
    To photoprism users: do you know how well it handles identifying photos of kids over the span of years? Google photos absolutely blows me away with its ~1% false positive rate (that is, given a photo, correctly identifying with kid(s) are in it) identifying each of my kids, nieces, and nephews. I don’t really know what its false negative rate (a false negative would be an uploaded photo of John but that doesn’t get tagged as having John in it) is, though. I can search for each of my kids and see photos of them going back to their birth.
- mavhc 4y ago
  How does the face recognition compare to https://www.photoprism.app/ https://www.photoprism.app/
- raffraffraff 4y ago
  There's a difference between face detection and face recognition. Last I checked (less than a year ago) it could find faces, but was abysmal at recognizing them. I spend a few days on it, tweaking this and that, taking to the devs. It was crap.
  - thedanbob 4y ago
    It was pretty bad until just recently (a couple of months ago?) when I noticed it was incredibly improved, might want to try it again.
- memoryoak 4y ago
  [flagged]
  - deleted 4y ago
    [deleted]
bayindirh 4y ago
Digikam is the bees knees in digital asset management. I manage my personal photo archive with that, and it punches way above its weight.
- tastemykungfu 4y ago
  Except for it's horrible face recognition right?.. ugh.
  - bayindirh 4y ago
    I'm away from my desktop computer, but it wasn't half bad when I last updated its DB. I don't take much portraits, so I don't use that feature much, but give it a look when I can and update you.
  - yread 4y ago
    Latest release failed the tests (with some memory corruption yikes) even. Doesn't inspire a lot of confidence that it won't eat your photos. But the v7 uses DNN from OpenCV for face detection that supposedly has 91% accuracy on CPLFW which is pretty good! So how come people here say it sucks?
    - bayindirh 4y ago
      Because many people test it once, and assume that since it's FOSS, the feature is not evolved over time.
- foobarbecue 4y ago
  "knees bees" ... maybe I'm missing a joke, but I always said "bees knees" in this context as a play on "business"
  - bayindirh 4y ago
    Fixed it, thanks. Sometimes my mind to finger interface changes packets on the way. I guess my nervous system uses UDP.
8fingerlouie 4y ago
Not that it helps you now, but i also keep all our family photos in the cloud (iCloud in my case), but at the same time i have a small ARM machine at home that keeps a mirror of the iCloud data. That ARM machine also has the responsibility of making backups, local to a USB drive, as well as to another cloud. Not mirrors, but proper versioned backups (as in Restic, Borg, Arq, Duplicacy, Kopia, etc). I also maintain a couple of USB drives with yearly updated mirrors of the entire photo library. The drives are stored at geographically different locations, and surface scanned, updated and rotated yearly. And finally, as a "last ditch recovery", i maintain an archive of M-disc Blu-Ray discs that contain a complete copy of our family photo library. Every year i make an identical set of discs containing the past years photos, and these sets are stored alongside the USB drives. I don't bother archiving documents as everything that is important is stored on government servers anyway, or exists in hardcopy. Also, if every step in my normal 3-2-1 backup scheme has failed and i need to recover from the archive, i probably have bigger issues than retrieving my budget for this years finances.
- password4321 4y ago
  > I don't bother archiving documents Isn't that 2 or 3 orders of magnitude less space though? I do understand where you're coming from for sure.
  - Arch-TK 4y ago
    You have to fuck around with the storage format because by default scanning sofware, even when scanning documents, will basically just take poorly compressed high resolution images. Maybe there's some service which sorts that out for you but so far I have a little webapp which stores an index of paper copies. Once I figure out how to automate scanning, compressing and OCRing documents then I think it will be worth storing digital copies. But for now it works.
    - ghaff 4y ago
      And don't forget adding metadata. I do scan some paper but there's increasingly less of it and I'm likely to need future access to so little of it that it's mostly not worth the trouble.
zamalek 4y ago
This is my backup method: * A nextcloud+recognize (for face recog) instance running on Hetzner. $20/month (I co-host other services on the VM). This is attached to a Hetzner storage box. * Syncthing pushes all of nextcloud to a zfs.rent (great service BTW) machine, I purchased my disks up front, so it's $cheap/month. ZFS snapshots are taken. * Local RPi NAS. Syncthing up to the other two. About $40/month in total. How valuable are 240k photos to you? Uploading photos is a matter of sending them to Nextcloud. Syncthing does the rest.
mikewarot 4y ago
Digikam is less crashy than it used to be. Its slowly going through my 650,000+ photos and finding faces. I started this past Sunday. I expect to be done in a week or two.
- raffraffraff 4y ago
  Finding faces or identifying faces? Because I tried the latter and it was horrendously bad.
  - mikewarot 4y ago
    Right now, it appears to be doing both. I really miss Picassa... if it wasn't for a bug in the last version, where it sometimes swapped face tags, I'd still be using it.
indymike 4y ago
Can second using DigiKam. It's a great app.
didgetmaster 4y ago
I'm glad you haven't really 'lost' the photos since you have copies elsewhere. It is surprising how many people these days rely on some cloud service to store and preserve their only copy of some important and unreproducible data. You are right about file systems doing a lousy job of helping you organize and locate photos, especially when they are spread across many different HDD and SDD drives attached to various computers. They are equally bad at organizing other forms of content (documents, videos, logs, music, software, etc.) File systems were invented decades ago when the biggest hard drives could only hold a few thousand files at most, and drives were also so expensive that most people only had one of them. Traditional file systems are antiquated and need to be replaced with something better.
Helmut10001 4y ago
Self-host and use Photoview [1], it has face recognition. I use this since 2 years with 5TB of photos (on ZFS) and it works flawlessly. It is worth it - everything under your control and you decide when to migrate. My phone's photos are also uploaded automatically to a Nextcloud folder, which is read-mounted to Photoview and scanned automatically. [1]:https://github.com/photoview/photoview https://github.com/photoview/photoview
34679 4y ago
I'm curious if you've asked your family members whether they're ok with having photos of themselves being used to train Amazon's facial recognition model? I know it's a common practice and Amazon isn't alone, yet I feel people should be more aware of what's happening.
varispeed 4y ago
I would be careful with hard drives. As I learned couple of days ago when I got my 10 year old hard drive from storage, I was unable to read most of the data as the drive would start to intermittently disconnect from the computer. Most likely the drive is okay, but the board has developed an issue - maybe capacitors gone bad or something. Now, I am yet to check if it is the one that has hardware encryption (I supposed to get rid of all of them and copy data to alternative drives) - there are drives that encrypt the data by default and if you don't set up the key, the encryption still takes place with some default key. If the board dies for some reason, then you won't ever be able to decrypt the data - even if technically was unencrypted, even by swapping the board from a working hard drive. This happened to me once and I lost 3TB of important data couple of years ago. That being said - I think the best additional backup is to store important things on BluRay - producers estimate they should last 80-100 years and so far never had a bad disk, even those burned years ago.
- brokenmachine 4y ago
  I recommend ddrescue for data recovery, worked really well when I needed it. It will resume when you hook the drive back up, and work around dead sectors.

capableweb 4y ago

> I think that I'll give DigiKam another go, against the NAS loaded up. I'm also looking for something FOSS that can do basic face recognition + maybe even more, but last time I checked DigiKam's detection didn't work so well, or maybe I got spoiled by the detection in Google Photos. If you do give it a go, would be nice if you reported back on your experience :)

komali2 4y ago
FOSS photo software with face recognition (note that some of this is off-premise face recognition, sometimes requiring an API key you get on your own) https://damselfly.info https://damselfly.info https://github.com/LibrePhotos/librephotos https://github.com/LibrePhotos/librephotos https://photoview.github.io/ https://photoview.github.io/ I may have missed some: https://github.com/awesome-selfhosted/awesome-selfhosted#photo-and-video-galleries https://github.com/awesome-selfhosted/awesome-selfhosted#pho...
- capableweb 4y ago
  What I found in my previous search is that either they're really good at managing large collections of photos, or they're really good at doing various AI/recognition, but none of what I've found have been good at both. Hence my interest in DigiKam as their management features for large collections is second to none.
  - CWuestefeld 4y ago
    Isn't that OK? You should be able to find some reasonable workflow around using the recognizer to tag the photo in its EXIF/IPTC metadata, and then load that into the organizer.
    - capableweb 4y ago
      It's not not OK :) Just not what I'm looking for, I spend enough time professionally with writing glue code, that I'm looking for a good out-of-the-box experience for something like this.
  - specto 4y ago
    I think immich will get there eventually
wh33zle 4y ago
Not sure if it has been mentioned already but photoprism added face recognition a few releases ago and it is working well for me: https://github.com/photoprism/photoprism https://github.com/photoprism/photoprism The quality of this software is overall extremely good. It is a solo-developer as well so you might want to consider sponsoring them if you end up using it. I am not affiliated, just a happy user :)
- thedanbob 4y ago
  Seconding photoprism. The face recognition was pretty meh when it was first introduced, more of a gimmick than anything. But (I assume) a recent update pushed it into "wow, this is actually really good" territory.
  - domh 4y ago
    Thirding photoprism, got it hosted on a raspberry pi and made it accessible outside of my home via Tailscale. Seems to work fairly well... though I could probably make it faster with an SSD vs an HDD over USB.
- Cyph0n 4y ago
  Photoprism is absolutely amazing. It was the quickest donation I’ve ever made to an OSS project. I have a Photoprism instance running on my home server backed by a raidz1 ZFS pool (1st backup). Photos are periodically synced to a Backblaze bucket with versioning enabled (2nd backup). The source of most photos is an iOS device with iCloud enabled (main copy). I rely on PhotoSync to periodically sync from the iOS device to Photoprism.
- philsnow 4y ago
  To photoprism users: do you know how well it handles identifying photos of kids over the span of years? Google photos absolutely blows me away with its ~1% false positive rate (that is, given a photo, correctly identifying with kid(s) are in it) identifying each of my kids, nieces, and nephews. I don’t really know what its false negative rate (a false negative would be an uploaded photo of John but that doesn’t get tagged as having John in it) is, though. I can search for each of my kids and see photos of them going back to their birth.
mavhc 4y ago
How does the face recognition compare to https://www.photoprism.app/ https://www.photoprism.app/
raffraffraff 4y ago
There's a difference between face detection and face recognition. Last I checked (less than a year ago) it could find faces, but was abysmal at recognizing them. I spend a few days on it, tweaking this and that, taking to the devs. It was crap.
- thedanbob 4y ago
  It was pretty bad until just recently (a couple of months ago?) when I noticed it was incredibly improved, might want to try it again.
memoryoak 4y ago
[flagged]
- deleted 4y ago
  [deleted]

bayindirh 4y ago

Digikam is the bees knees in digital asset management. I manage my personal photo archive with that, and it punches way above its weight.

tastemykungfu 4y ago
Except for it's horrible face recognition right?.. ugh.
- bayindirh 4y ago
  I'm away from my desktop computer, but it wasn't half bad when I last updated its DB. I don't take much portraits, so I don't use that feature much, but give it a look when I can and update you.
- yread 4y ago
  Latest release failed the tests (with some memory corruption yikes) even. Doesn't inspire a lot of confidence that it won't eat your photos. But the v7 uses DNN from OpenCV for face detection that supposedly has 91% accuracy on CPLFW which is pretty good! So how come people here say it sucks?
  - bayindirh 4y ago
    Because many people test it once, and assume that since it's FOSS, the feature is not evolved over time.
foobarbecue 4y ago
"knees bees" ... maybe I'm missing a joke, but I always said "bees knees" in this context as a play on "business"
- bayindirh 4y ago
  Fixed it, thanks. Sometimes my mind to finger interface changes packets on the way. I guess my nervous system uses UDP.

8fingerlouie 4y ago

Not that it helps you now, but i also keep all our family photos in the cloud (iCloud in my case), but at the same time i have a small ARM machine at home that keeps a mirror of the iCloud data. That ARM machine also has the responsibility of making backups, local to a USB drive, as well as to another cloud. Not mirrors, but proper versioned backups (as in Restic, Borg, Arq, Duplicacy, Kopia, etc). I also maintain a couple of USB drives with yearly updated mirrors of the entire photo library. The drives are stored at geographically different locations, and surface scanned, updated and rotated yearly. And finally, as a "last ditch recovery", i maintain an archive of M-disc Blu-Ray discs that contain a complete copy of our family photo library. Every year i make an identical set of discs containing the past years photos, and these sets are stored alongside the USB drives. I don't bother archiving documents as everything that is important is stored on government servers anyway, or exists in hardcopy. Also, if every step in my normal 3-2-1 backup scheme has failed and i need to recover from the archive, i probably have bigger issues than retrieving my budget for this years finances.

password4321 4y ago
> I don't bother archiving documents Isn't that 2 or 3 orders of magnitude less space though? I do understand where you're coming from for sure.
- Arch-TK 4y ago
  You have to fuck around with the storage format because by default scanning sofware, even when scanning documents, will basically just take poorly compressed high resolution images. Maybe there's some service which sorts that out for you but so far I have a little webapp which stores an index of paper copies. Once I figure out how to automate scanning, compressing and OCRing documents then I think it will be worth storing digital copies. But for now it works.
  - ghaff 4y ago
    And don't forget adding metadata. I do scan some paper but there's increasingly less of it and I'm likely to need future access to so little of it that it's mostly not worth the trouble.

zamalek 4y ago

This is my backup method: * A nextcloud+recognize (for face recog) instance running on Hetzner. $20/month (I co-host other services on the VM). This is attached to a Hetzner storage box. * Syncthing pushes all of nextcloud to a zfs.rent (great service BTW) machine, I purchased my disks up front, so it's $cheap/month. ZFS snapshots are taken. * Local RPi NAS. Syncthing up to the other two. About $40/month in total. How valuable are 240k photos to you? Uploading photos is a matter of sending them to Nextcloud. Syncthing does the rest.

mikewarot 4y ago

Digikam is less crashy than it used to be. Its slowly going through my 650,000+ photos and finding faces. I started this past Sunday. I expect to be done in a week or two.

raffraffraff 4y ago
Finding faces or identifying faces? Because I tried the latter and it was horrendously bad.
- mikewarot 4y ago
  Right now, it appears to be doing both. I really miss Picassa... if it wasn't for a bug in the last version, where it sometimes swapped face tags, I'd still be using it.

indymike 4y ago

Can second using DigiKam. It's a great app.

didgetmaster 4y ago

I'm glad you haven't really 'lost' the photos since you have copies elsewhere. It is surprising how many people these days rely on some cloud service to store and preserve their only copy of some important and unreproducible data. You are right about file systems doing a lousy job of helping you organize and locate photos, especially when they are spread across many different HDD and SDD drives attached to various computers. They are equally bad at organizing other forms of content (documents, videos, logs, music, software, etc.) File systems were invented decades ago when the biggest hard drives could only hold a few thousand files at most, and drives were also so expensive that most people only had one of them. Traditional file systems are antiquated and need to be replaced with something better.

Helmut10001 4y ago

Self-host and use Photoview [1], it has face recognition. I use this since 2 years with 5TB of photos (on ZFS) and it works flawlessly. It is worth it - everything under your control and you decide when to migrate. My phone's photos are also uploaded automatically to a Nextcloud folder, which is read-mounted to Photoview and scanned automatically. [1]:https://github.com/photoview/photoview https://github.com/photoview/photoview

34679 4y ago

I'm curious if you've asked your family members whether they're ok with having photos of themselves being used to train Amazon's facial recognition model? I know it's a common practice and Amazon isn't alone, yet I feel people should be more aware of what's happening.

varispeed 4y ago

I would be careful with hard drives. As I learned couple of days ago when I got my 10 year old hard drive from storage, I was unable to read most of the data as the drive would start to intermittently disconnect from the computer. Most likely the drive is okay, but the board has developed an issue - maybe capacitors gone bad or something. Now, I am yet to check if it is the one that has hardware encryption (I supposed to get rid of all of them and copy data to alternative drives) - there are drives that encrypt the data by default and if you don't set up the key, the encryption still takes place with some default key. If the board dies for some reason, then you won't ever be able to decrypt the data - even if technically was unencrypted, even by swapping the board from a working hard drive. This happened to me once and I lost 3TB of important data couple of years ago. That being said - I think the best additional backup is to store important things on BluRay - producers estimate they should last 80-100 years and so far never had a bad disk, even those burned years ago.

brokenmachine 4y ago
I recommend ddrescue for data recovery, worked really well when I needed it. It will resume when you hook the drive back up, and work around dead sectors.