4 ms·

IOS5 Breaks Data Storage and Crushes My Soul

19 points by dsil 15y ago

slillibri 15y ago
Not to trivialize the problem, but I can't be the only one to think "Hey, maybe it's not a good idea to store permanent data in a directory called Caches".
- dbattaglia 15y ago
  That was exactly what I was thinking while reading this. Have you ever read any of Raymond Chen's "oldnewthing" blog? He talks a lot about app compatibility between Windows versions, and how they are forced to not change certain undocumented behaviors because developers end up relying on them regardless of the documentation. I didn't read the docs for iOS "caches" storage but I have a feeling this is another case of Apple's "break clients for a better OS" view on app-compat, compared to the MS's "buggy-er OS that still runs Windows 1.0 apps" view (I'm primarily a Windows dev, writing this from my iMac, so I'm not really biased either way, just an observation/guess).
zrgiu_ 15y ago
Maybe i'm not understanding this correctly (I'm not an iPhone dev), but why don't you just store the data in the Documents folder ? Or maybe make it the user's choice: 1. Save data in the Documents folder - > slower itunes backup or 2. Save data in the Caches folder -> Arbitrarily deletion of data
- saurik 15y ago
  The whole point of this conversation is that a real class of problems exist (think "offline Google Maps": download all of France to your phone, so you can have driving directions while you don't have data roaming on your trip) that wants option 3: not backed up (as you just downloaded a gigabyte of data that you can download again any time you want), but guaranteed to be available when offline (as the whole point was to accidentally have your data get deleted while you are travelling through Germany on your way to France).
faizanaziz 15y ago
Apple wants Cache (stuff that can be re downloaded and not core to the experience, just to improve performance) to actually be deletable when space is low. A valid point. Developers want a place to store app information that is critical and sometimes a very important part of the app(example main feature of instapaper is to view offline content). This should not be backed up since its redundant and when the phone needs to be restored(rarely) this data can be re-downloaded. Also backing up will be slow and destroy user experience. Solution make another location which is used for storing the data(not backed up) so then all are happy. Maybe you can have data from hidden directories not backed up inside the Documents directory. Then Apple can delete caches in apps like browsers. User critical data like settings can be backed up and app critical data that need not be backed up can also exist
- Turing_Machine 15y ago
  In the Stack Overflow article linked from the parent, the dev describes the scenario this way: "No, users explicitly download sets of map tiles, that they will need when they are in the forest, or flying their airplane, or sailing their boat." If it's that important, it SHOULD go in a folder that's backed up/synced to the cloud or so it seems to me. What happens when users conditioned to expect that all their documents will automagically appear on all their Apple products discover that the maps they set up on their iPad at home aren't on on their iPhone when they get out in the field? Food for thought.
  - andrewljohnson 15y ago
    No, users shouldn't have to back up 5 gigs of maps to their 5 gig iCloud storage.
    - Turing_Machine 15y ago
      Making an assertion isn't answering the question.
      - Turing_Machine 15y ago
kbob 15y ago
As a user, I can't imagine a class of data that is so important it should never be reclaimed to save space, and is so worthless it should not be backed up. The important use case is what should happen when you lose your phone/tablet and buy a replacement. Or wipe it. Should the data be restored or should it be lazily regenerated/redownloaded? For map cache data, re-download it. Otherwise, the map app runs fast at the expense of all the other apps on the device. Data that has been preloaded to read offline (e.g., Instapaper) should be stored in Documents. If I lost my iPad, I would not want to have to manually reload a bunch of articles when I got a replacement. I understand that IOS5 has gratuitously broken a lot of apps, and the new regime may have been communicated badly. (May have. I'm not an IOS developer; I don't know what the developer documentation says.) But IOS is user-centric, not dev-centric.
- andrewljohnson 15y ago
  Imagine I have a Flight Charts application. The app automatically downloads charts for the entire US, and when the charts are soon to be updated, it also grabs them ahead of time so it can update the charts when the cycle changes, without needing to spend hours downloading new charts. Now imagine when they go to fly and they turn the app on, half of the charts disappear. Are you suggesting I architect the app to go ahead and download the new maps when this happens? Because if that's your suggestion, the Air Force and commercial airlines no longer wants to use my app. Those charts darn well better be on board when they expect them to be. They need to be able to reliably have both gigs of current charts, and gigs of upcoming charts, ready to fly. They might fly to China or elsewhere they won't be able to update, enroute.
  - kbob 15y ago
    You have successfully argued that the data is too valuable to purge. You haven't argued that it's not worth backing up. In this case, it sounds like it should be stored in Documents and backed up. (Are you the original article's author? If so, sorry I misunderstood the lifetime of your mapping data.)
    - 15y ago
deleted 15y ago
[deleted]
- andrewljohnson 15y ago
  Yes, people have complained. And yes, we'll move them to Documents if that's the best option - did you read the whole blog post?
  - deleted 15y ago
    [deleted]
  - deleted 15y ago
    [deleted]
    - andrewljohnson 15y ago
      So, sorry to be snide, but that's why I thought you hadn't read it though. I noted we plan to do just that - move everything to Documents and then have the user click off syncing. We don't think that's a very elegant solution though, and we can't even provide a button or something to effect the change. We literally have to instruct the user how to drill down into a Setting 5 clicks deep.
      - deleted 15y ago
        [deleted]
andrewljohnson 15y ago
I read all the comments in this thread, and it surprises me that so many people have a hard time imagining a case where the user wants to store several gigabytes of data locally - maybe 10 gigs - but not have that data backed up to a cloud.
Hisoka 15y ago
Probably not a practical solution but could you save some of it in the user preferences? If it's an image, you could probably base64 encode it and save it there. I know it's not meant to save tons of data though so probably not a feasible solution. Or how about encoding it and putting it in a Sqlite database?
flocial 15y ago
The bottom line is, this change is mainly to save Apple the bandwidth and storage costs associated with iCloud backups. It's a straightforward solution to offer developers a programmatic way to opt-out. With maps or news readers, I might want to keep a local copy indefinitely on my iPhone but do I want these constantly backed up? No. I wouldn't mind re-downloading them when I upgrade my phone. But do I want them deleted arbitrarily when storage runs low? No because I have no control over it. Just imagine your browser cache being cleared by the OS at random moments when other applications trigger an arbitrary file limit and causing errors at random moments. If Apple could offer a way for users to allocate a fixed amount of space for certain apps and opt out of syncing certain data per app, it wouldn't be a problem. What's more this change suddenly happened. There was no bright red warning sign going into the release, these developers are finding out the hard way.

19 points by dsil 15y ago

Not to trivialize the problem, but I can't be the only one to think "Hey, maybe it's not a good idea to store permanent data in a directory called Caches".

dbattaglia 15y ago
That was exactly what I was thinking while reading this. Have you ever read any of Raymond Chen's "oldnewthing" blog? He talks a lot about app compatibility between Windows versions, and how they are forced to not change certain undocumented behaviors because developers end up relying on them regardless of the documentation. I didn't read the docs for iOS "caches" storage but I have a feeling this is another case of Apple's "break clients for a better OS" view on app-compat, compared to the MS's "buggy-er OS that still runs Windows 1.0 apps" view (I'm primarily a Windows dev, writing this from my iMac, so I'm not really biased either way, just an observation/guess).

zrgiu_ 15y ago

Maybe i'm not understanding this correctly (I'm not an iPhone dev), but why don't you just store the data in the Documents folder ? Or maybe make it the user's choice: 1. Save data in the Documents folder - > slower itunes backup or 2. Save data in the Caches folder -> Arbitrarily deletion of data

saurik 15y ago
The whole point of this conversation is that a real class of problems exist (think "offline Google Maps": download all of France to your phone, so you can have driving directions while you don't have data roaming on your trip) that wants option 3: not backed up (as you just downloaded a gigabyte of data that you can download again any time you want), but guaranteed to be available when offline (as the whole point was to accidentally have your data get deleted while you are travelling through Germany on your way to France).

faizanaziz 15y ago

Apple wants Cache (stuff that can be re downloaded and not core to the experience, just to improve performance) to actually be deletable when space is low. A valid point. Developers want a place to store app information that is critical and sometimes a very important part of the app(example main feature of instapaper is to view offline content). This should not be backed up since its redundant and when the phone needs to be restored(rarely) this data can be re-downloaded. Also backing up will be slow and destroy user experience. Solution make another location which is used for storing the data(not backed up) so then all are happy. Maybe you can have data from hidden directories not backed up inside the Documents directory. Then Apple can delete caches in apps like browsers. User critical data like settings can be backed up and app critical data that need not be backed up can also exist

Turing_Machine 15y ago
In the Stack Overflow article linked from the parent, the dev describes the scenario this way: "No, users explicitly download sets of map tiles, that they will need when they are in the forest, or flying their airplane, or sailing their boat." If it's that important, it SHOULD go in a folder that's backed up/synced to the cloud or so it seems to me. What happens when users conditioned to expect that all their documents will automagically appear on all their Apple products discover that the maps they set up on their iPad at home aren't on on their iPhone when they get out in the field? Food for thought.
- andrewljohnson 15y ago
  No, users shouldn't have to back up 5 gigs of maps to their 5 gig iCloud storage.
  - Turing_Machine 15y ago
    Making an assertion isn't answering the question.
    - Turing_Machine 15y ago

kbob 15y ago

As a user, I can't imagine a class of data that is so important it should never be reclaimed to save space, and is so worthless it should not be backed up. The important use case is what should happen when you lose your phone/tablet and buy a replacement. Or wipe it. Should the data be restored or should it be lazily regenerated/redownloaded? For map cache data, re-download it. Otherwise, the map app runs fast at the expense of all the other apps on the device. Data that has been preloaded to read offline (e.g., Instapaper) should be stored in Documents. If I lost my iPad, I would not want to have to manually reload a bunch of articles when I got a replacement. I understand that IOS5 has gratuitously broken a lot of apps, and the new regime may have been communicated badly. (May have. I'm not an IOS developer; I don't know what the developer documentation says.) But IOS is user-centric, not dev-centric.

andrewljohnson 15y ago
Imagine I have a Flight Charts application. The app automatically downloads charts for the entire US, and when the charts are soon to be updated, it also grabs them ahead of time so it can update the charts when the cycle changes, without needing to spend hours downloading new charts. Now imagine when they go to fly and they turn the app on, half of the charts disappear. Are you suggesting I architect the app to go ahead and download the new maps when this happens? Because if that's your suggestion, the Air Force and commercial airlines no longer wants to use my app. Those charts darn well better be on board when they expect them to be. They need to be able to reliably have both gigs of current charts, and gigs of upcoming charts, ready to fly. They might fly to China or elsewhere they won't be able to update, enroute.
- kbob 15y ago
  You have successfully argued that the data is too valuable to purge. You haven't argued that it's not worth backing up. In this case, it sounds like it should be stored in Documents and backed up. (Are you the original article's author? If so, sorry I misunderstood the lifetime of your mapping data.)
  - 15y ago

deleted 15y ago

[deleted]

andrewljohnson 15y ago
Yes, people have complained. And yes, we'll move them to Documents if that's the best option - did you read the whole blog post?
- deleted 15y ago
  [deleted]
- deleted 15y ago
  [deleted]
  - andrewljohnson 15y ago
    So, sorry to be snide, but that's why I thought you hadn't read it though. I noted we plan to do just that - move everything to Documents and then have the user click off syncing. We don't think that's a very elegant solution though, and we can't even provide a button or something to effect the change. We literally have to instruct the user how to drill down into a Setting 5 clicks deep.
    - deleted 15y ago
      [deleted]

andrewljohnson 15y ago

I read all the comments in this thread, and it surprises me that so many people have a hard time imagining a case where the user wants to store several gigabytes of data locally - maybe 10 gigs - but not have that data backed up to a cloud.

Hisoka 15y ago

Probably not a practical solution but could you save some of it in the user preferences? If it's an image, you could probably base64 encode it and save it there. I know it's not meant to save tons of data though so probably not a feasible solution. Or how about encoding it and putting it in a Sqlite database?

flocial 15y ago

The bottom line is, this change is mainly to save Apple the bandwidth and storage costs associated with iCloud backups. It's a straightforward solution to offer developers a programmatic way to opt-out. With maps or news readers, I might want to keep a local copy indefinitely on my iPhone but do I want these constantly backed up? No. I wouldn't mind re-downloading them when I upgrade my phone. But do I want them deleted arbitrarily when storage runs low? No because I have no control over it. Just imagine your browser cache being cleared by the OS at random moments when other applications trigger an arbitrary file limit and causing errors at random moments. If Apple could offer a way for users to allocate a fixed amount of space for certain apps and opt out of syncing certain data per app, it wouldn't be a problem. What's more this change suddenly happened. There was no bright red warning sign going into the release, these developers are finding out the hard way.