3 ms·

My concern with backing up iCloud Photos with anything but Apple Photos is that there are some proprietary formats like Live Photos and slow mo video for which

My concern with backing up iCloud Photos with anything but Apple Photos is that there are some proprietary formats like Live Photos and slow mo video for which exports are lossy. Also, Apple Photos stores all edits non-destructively, so 'flattening' the edits into a single file for export is also a lossy operation. It seems like an obvious improvement for Time Machine to support full backups while using optimized storage on the primary system.

lostlogin 9mo ago
Could the first obvious improvement please be its speed? My god. The local Time Machine backup is slower on a 10gb network than Backblaze over the Internet. It isn’t even close.
- dangus 9mo ago
  Asking for anything out of Time Machine is a lost cause. It’s essentially a completed and legacy product. I migrated to Linux + Pika Backup. For photos I use Ente Photos with their managed cloud storage plus a continuous export to my NAS. Ente is surprisingly well integrated with iOS, you really don’t need to use Apple’s solution. It automatically backs up photos I take in the background.
- deleted 9mo ago
  [deleted]
- varenc 9mo ago
  I backup ~3-4GB a day with Time Machine to my local NAS and it takes less 10 minutes. Albeit it should take 30 seconds if it was maxing out the network speed.
  - lostlogin 9mo ago
    I’ve done the sudo sysctl debug.lowpri_throttle_enabled=0. But it’s still slow. 10 minutes is great, and my changes wouldn’t seem as extensive as yours. I need to dig deeper.
    - varenc 9mo ago
      A huge chunk of that 3-4GB is large files that have minor changes. Time Machine doesn't have any sort of delta support so backups the entire file again, like my local Messages or Contacts database. But I think slowness is caused by file count, not file size, so even though it's backing up 3GB+, the total number of files changes isn't that high. (I suspect). I also use a modified version of this script[0] to identify everything that changed in the most recent Time Machine backup. This is hugely helpful and lets me find unimportant things that are the source of lots of unimportant changes which I can then exclude with `tmux addexclusion -p <PATH>`. For example I exclude 'node_modules' folders for anything that gets regularly updated. This removes ~10k files that would otherwise be wastefully backed up. Speeding it up is much more about reducing file count than total size. [0] https://github.com/nlfiedler/timedog https://github.com/nlfiedler/timedog (though its woefully out of date, and I had to make some hacky patches to it to get it working)
- rgovostes 9mo ago
  I reinstalled my system and attempted for weeks to get Time Machine to complete a first backup. Every time I started it, the progress bar would fill up about 60% and then stall, and eventually kernel panic if the system was left idle for hours. Never happened before I reinstalled, though I have had it randomly decide the backup is corrupt and it has to start over. macOS deserves a better first-party backup feature.
  - lostlogin 9mo ago
    > I have had it randomly decide the backup is corrupt and it has to start over. I have hit this too many times.
dangus 9mo ago
In my experience migrating to another provider from iCloud, this hasn’t been a significant issue. Live Photos in particular are not really proprietary in the sense that they’re implemented in an extremely simple way that basically every photo tool understands. ~~Slow motion videos are also not proprietary, they’re just a plain video file.~~ <<< edit: I think I’m wrong about slow motion
- rgovostes 9mo ago
  > Slow motion videos are also not proprietary, they’re just a plain video file. I haven't looked into the implementation details, but Photos lets you adjust the section of the video that is played back in slow motion. I thought if you share a slow-mo video, it gets re-encoded to bake this in (i.e., one second at 240fps gets exported as four seconds at 60fps).
fn-mote 9mo ago
How about just turning off optimized storage and letting Time Machine do its job?
- rgovostes 9mo ago
  I have a 1.2T photo library. Carrying that around on my MacBook requires an $800 upgrade to the SSD. Time Machine's job is to back up my data, it's not strictly to make a 1:1 copy of local storage. It should back up my cloud data too.
  - mrexroad 9mo ago
    My library is large too (roughly a third larger). After years of far more complicated storage/backup solutions, I settled w/ a second Photos library on an external hdd w/ optimize storage disabled. I plug the drive in and open this library every so often to update and then duplicate the drive for an off-site copy. Day to day, I use a Photos library on my primary drive with optimize storage enabled. I’ve found unreasonable value in being able to search through hundreds of thousands of photos from my phone, so I went all-in on Photos.app. Though one enabling factor is that my photography workflow has drastic simplified in recent years to doing very little post (except for astrophotography, which I try and keep wip out of Photos.app anyway).
    - nik_0_0 9mo ago
      I had tried this but found it a little bit weird - switching back and forth on the same device between the 'hard drive w/ full files' library and the 'primary drive optimize storage' didn't really seem easy. IIRC Photos.app will not even open if the default library you are pointing at is not there (i.e drive was unplugged). Are you able to just open up the library file directly and it will work as expected? I also recall when changing Photos.app back to the HDD Library it did a ~2h 'rebuild' session before it even started downloading the new photos, but maybe thats acceptable with the 'every so often' approach.
SchemaLoad 9mo ago
The most annoying thing for me is if you set the date for a photo, it gets stored externally rather than modifying the photo metadata. So when you switch platform, every photo which didn't originally have a captured at date ends up reset to the current day every time you move. For edits, I don't care too much about just baking them in since it's unlikely I'm going back to old photos and want to undo the crop.
kimos 9mo ago
I’ve used this tool for years and it’s great. But it really saves just the raw data. You’d never get it back in to Apple Photos as nice as when you pulled it out. Metadata is missing. Live Photos come out as an image and a similarly named video. But I treat it as the emergency backup. If some Apple DC burns down or they ban my Apple ID for some reason, at least the photos still exist.
jaffa2 9mo ago
This. I don't understand why Apple don't have another checkbox beside the Download Originals to this Mac that reads 'Store Backup of Original Photos on Timemachine' This is all that's needed to solve the issue. I actually bought a Mac Studio, and a USB disk, just to be able to download originals of my photos for local backup, since a MBP is effetively a mobil limited device just like an iphone. "I actually bought a Mac Studio"... "I don't understand why Apple don't " ... wait a minute

lostlogin 9mo ago

Could the first obvious improvement please be its speed? My god. The local Time Machine backup is slower on a 10gb network than Backblaze over the Internet. It isn’t even close.

dangus 9mo ago
Asking for anything out of Time Machine is a lost cause. It’s essentially a completed and legacy product. I migrated to Linux + Pika Backup. For photos I use Ente Photos with their managed cloud storage plus a continuous export to my NAS. Ente is surprisingly well integrated with iOS, you really don’t need to use Apple’s solution. It automatically backs up photos I take in the background.
deleted 9mo ago
[deleted]
varenc 9mo ago
I backup ~3-4GB a day with Time Machine to my local NAS and it takes less 10 minutes. Albeit it should take 30 seconds if it was maxing out the network speed.
- lostlogin 9mo ago
  I’ve done the sudo sysctl debug.lowpri_throttle_enabled=0. But it’s still slow. 10 minutes is great, and my changes wouldn’t seem as extensive as yours. I need to dig deeper.
  - varenc 9mo ago
    A huge chunk of that 3-4GB is large files that have minor changes. Time Machine doesn't have any sort of delta support so backups the entire file again, like my local Messages or Contacts database. But I think slowness is caused by file count, not file size, so even though it's backing up 3GB+, the total number of files changes isn't that high. (I suspect). I also use a modified version of this script[0] to identify everything that changed in the most recent Time Machine backup. This is hugely helpful and lets me find unimportant things that are the source of lots of unimportant changes which I can then exclude with `tmux addexclusion -p <PATH>`. For example I exclude 'node_modules' folders for anything that gets regularly updated. This removes ~10k files that would otherwise be wastefully backed up. Speeding it up is much more about reducing file count than total size. [0] https://github.com/nlfiedler/timedog https://github.com/nlfiedler/timedog (though its woefully out of date, and I had to make some hacky patches to it to get it working)
rgovostes 9mo ago
I reinstalled my system and attempted for weeks to get Time Machine to complete a first backup. Every time I started it, the progress bar would fill up about 60% and then stall, and eventually kernel panic if the system was left idle for hours. Never happened before I reinstalled, though I have had it randomly decide the backup is corrupt and it has to start over. macOS deserves a better first-party backup feature.
- lostlogin 9mo ago
  > I have had it randomly decide the backup is corrupt and it has to start over. I have hit this too many times.

dangus 9mo ago

In my experience migrating to another provider from iCloud, this hasn’t been a significant issue. Live Photos in particular are not really proprietary in the sense that they’re implemented in an extremely simple way that basically every photo tool understands. ~~Slow motion videos are also not proprietary, they’re just a plain video file.~~ <<< edit: I think I’m wrong about slow motion

rgovostes 9mo ago
> Slow motion videos are also not proprietary, they’re just a plain video file. I haven't looked into the implementation details, but Photos lets you adjust the section of the video that is played back in slow motion. I thought if you share a slow-mo video, it gets re-encoded to bake this in (i.e., one second at 240fps gets exported as four seconds at 60fps).

fn-mote 9mo ago

How about just turning off optimized storage and letting Time Machine do its job?

rgovostes 9mo ago
I have a 1.2T photo library. Carrying that around on my MacBook requires an $800 upgrade to the SSD. Time Machine's job is to back up my data, it's not strictly to make a 1:1 copy of local storage. It should back up my cloud data too.
- mrexroad 9mo ago
  My library is large too (roughly a third larger). After years of far more complicated storage/backup solutions, I settled w/ a second Photos library on an external hdd w/ optimize storage disabled. I plug the drive in and open this library every so often to update and then duplicate the drive for an off-site copy. Day to day, I use a Photos library on my primary drive with optimize storage enabled. I’ve found unreasonable value in being able to search through hundreds of thousands of photos from my phone, so I went all-in on Photos.app. Though one enabling factor is that my photography workflow has drastic simplified in recent years to doing very little post (except for astrophotography, which I try and keep wip out of Photos.app anyway).
  - nik_0_0 9mo ago
    I had tried this but found it a little bit weird - switching back and forth on the same device between the 'hard drive w/ full files' library and the 'primary drive optimize storage' didn't really seem easy. IIRC Photos.app will not even open if the default library you are pointing at is not there (i.e drive was unplugged). Are you able to just open up the library file directly and it will work as expected? I also recall when changing Photos.app back to the HDD Library it did a ~2h 'rebuild' session before it even started downloading the new photos, but maybe thats acceptable with the 'every so often' approach.

SchemaLoad 9mo ago

The most annoying thing for me is if you set the date for a photo, it gets stored externally rather than modifying the photo metadata. So when you switch platform, every photo which didn't originally have a captured at date ends up reset to the current day every time you move. For edits, I don't care too much about just baking them in since it's unlikely I'm going back to old photos and want to undo the crop.

kimos 9mo ago

I’ve used this tool for years and it’s great. But it really saves just the raw data. You’d never get it back in to Apple Photos as nice as when you pulled it out. Metadata is missing. Live Photos come out as an image and a similarly named video. But I treat it as the emergency backup. If some Apple DC burns down or they ban my Apple ID for some reason, at least the photos still exist.

jaffa2 9mo ago

This. I don't understand why Apple don't have another checkbox beside the Download Originals to this Mac that reads 'Store Backup of Original Photos on Timemachine' This is all that's needed to solve the issue. I actually bought a Mac Studio, and a USB disk, just to be able to download originals of my photos for local backup, since a MBP is effetively a mobil limited device just like an iphone. "I actually bought a Mac Studio"... "I don't understand why Apple don't " ... wait a minute