3 ms·

If I'm offline for some period of time and upload photos to my laptop, an unreliable filesystem has every chance to corrupt my data before it gets uploaded to t

If I'm offline for some period of time and upload photos to my laptop, an unreliable filesystem has every chance to corrupt my data before it gets uploaded to the cloud when I eventually reconnect. The "cloud" option itself simplifies the backup process and could potentially reduce the window of a chance for corruption, but it does not eliminate the need for a reliable, local filesystem.

uptown 12y ago
Wouldn't you still have the copy on the device or memory card you took the photo from as an alternate source to restore from? Just avoid wiping that until you have two other copies.
- cbhl 12y ago
  If the device is an iPhone[0], then there is only one copy on one filesystem on one memory card. Granted, if your iPhone is offline for extended periods of time, you're also doing it wrong. But maybe you have an iPod Touch instead. [0] For the last few years, the newest iPhone is the #1 most common camera model found in EXIF data on Flickr, http://petapixel.com/2015/01/09/popular-cameras-flickr-2014/ http://petapixel.com/2015/01/09/popular-cameras-flickr-2014/. IIRC, Facebook saw similar results for photo uploads.
- vacri 12y ago
  This seems terrifying if the filesystem is that bad that you have to do this to guard against corruption. And isn't half the point of offloading images from a card to clear up the card for reuse?
  - epochwolf 12y ago
    I don't clear photos off SD Cards until the files are in my offsite backup.
abalone 12y ago
I don't accept the premise that filesystems are "unreliable" because you experienced a case of data loss. Hard drives can and regularly do experience hardware failures even with the best possible filesystem software. Camera memory cards can experience data loss before you get around to uploading. Reliability is like security; it's a matter of degrees not an absolute measure. That is why a good, well-integrated offsite backup solution that "reduces the window of a chance for corruption" is a much more significant impact on data security than anything we could hope to incrementally improve in the filesystem software layer. And Apple's solution is even better than you describe. Photos taken from Apple cameras (which are the vast bulk of photos for most customers) are automatically uploaded directly to the cloud as soon as the device joins wifi.
- binarycrusader 12y ago
  And you seem to be completely ignoring the fact that a filesystem that can't even guarantee that it wrote the data to an intermediate storage device correctly is fundamentally unreliable. That is the core issue. Yes, well-integrated offline backup solutions mitigate some aspects of the problem, but when your filesystem can't even guarantee that it did its job correctly, it doesn't matter how well-integrated your "backup" solutions are because the data may already be corrupt. Nevermind that the "well-integrated" backup solution you are describing here, because of the unreliable filesystem, can't even guarantee that it read the data from disk correctly to a reasonable degree. This is not some anecdotal "Mac OS X burned me" story -- this is a fundamental, architectural issue. Data loss is real -- and so are my concerns about HFS+.
  - millstone 12y ago
    Why do you think HFS+ can't guarantee that it wrote data to a storage device? From what I know (F_FULLFSYNC, etc) it does have that capability.
    - MCRed 12y ago
      And why isn't he giving more details? He's being kinda broad without being specific enough. At this point my guess is he's been running HFS+ without journaling turned on (you can format drives this way) and experienced data loss as a result because the filesystems ability to correct itself was disabled. This would look like what he's describing.

uptown 12y ago

Wouldn't you still have the copy on the device or memory card you took the photo from as an alternate source to restore from? Just avoid wiping that until you have two other copies.

cbhl 12y ago
If the device is an iPhone[0], then there is only one copy on one filesystem on one memory card. Granted, if your iPhone is offline for extended periods of time, you're also doing it wrong. But maybe you have an iPod Touch instead. [0] For the last few years, the newest iPhone is the #1 most common camera model found in EXIF data on Flickr, http://petapixel.com/2015/01/09/popular-cameras-flickr-2014/ http://petapixel.com/2015/01/09/popular-cameras-flickr-2014/. IIRC, Facebook saw similar results for photo uploads.
vacri 12y ago
This seems terrifying if the filesystem is that bad that you have to do this to guard against corruption. And isn't half the point of offloading images from a card to clear up the card for reuse?
- epochwolf 12y ago
  I don't clear photos off SD Cards until the files are in my offsite backup.

abalone 12y ago

I don't accept the premise that filesystems are "unreliable" because you experienced a case of data loss. Hard drives can and regularly do experience hardware failures even with the best possible filesystem software. Camera memory cards can experience data loss before you get around to uploading. Reliability is like security; it's a matter of degrees not an absolute measure. That is why a good, well-integrated offsite backup solution that "reduces the window of a chance for corruption" is a much more significant impact on data security than anything we could hope to incrementally improve in the filesystem software layer. And Apple's solution is even better than you describe. Photos taken from Apple cameras (which are the vast bulk of photos for most customers) are automatically uploaded directly to the cloud as soon as the device joins wifi.

binarycrusader 12y ago
And you seem to be completely ignoring the fact that a filesystem that can't even guarantee that it wrote the data to an intermediate storage device correctly is fundamentally unreliable. That is the core issue. Yes, well-integrated offline backup solutions mitigate some aspects of the problem, but when your filesystem can't even guarantee that it did its job correctly, it doesn't matter how well-integrated your "backup" solutions are because the data may already be corrupt. Nevermind that the "well-integrated" backup solution you are describing here, because of the unreliable filesystem, can't even guarantee that it read the data from disk correctly to a reasonable degree. This is not some anecdotal "Mac OS X burned me" story -- this is a fundamental, architectural issue. Data loss is real -- and so are my concerns about HFS+.
- millstone 12y ago
  Why do you think HFS+ can't guarantee that it wrote data to a storage device? From what I know (F_FULLFSYNC, etc) it does have that capability.
  - MCRed 12y ago
    And why isn't he giving more details? He's being kinda broad without being specific enough. At this point my guess is he's been running HFS+ without journaling turned on (you can format drives this way) and experienced data loss as a result because the filesystems ability to correct itself was disabled. This would look like what he's describing.