8 ms·

Why all my iOS Apps are on hold - or iCloud sucks Part II

46 points by mikecane 14y ago

gfodor 14y ago
it would be helpful if this article included some actual details about what doesn't work for people considering adding iCloud to their apps.
- stevenp 14y ago
  Agreed. This article is completely useless.
- potatolicious 14y ago
  iCloud in general "just works". It was pretty terrible from the get-go, from reliability to really egregious bugs in the OS, but at this point is pretty stable. The real problem here is the combination of iCloud and Core Data (for non-Apple folks, the built-in persistence/object model framework). In a somewhat simplified way, Core Data is a data store most commonly backed by SQLite. It's used everywhere from your mail app to the Facebook app. It's handy and convenient when it isn't trying to kill you - it's one of the more complex frameworks that Apple ships, and the documentation is extremely poor for it. It also doesn't help that there is a lot of quirky behavior that's more or less tribal knowledge to long-time Apple devs and WWDC attendees, and is has enough concurrency gotchas to fill a container ship. So, Core Data is not the simplest piece of code to work with from the get-go, and then you pile iCloud on top. iCloud is conceptually pretty simple - there's a directory on your file system that is replicated to Apple's servers, and back down to each one of your devices. When files change the OS sends notifications to your app informing you of such. If you're using iCloud as a distributed file system (a la Dropbox), it's easy to integrate. The trouble with this model is that Core Data stores can be in fact quite large. If you have a 30MB database of, say, recipes on your disk, you really don't want to resend the whole thing every time your user alters a row. Apple special-cases Core Data stores in iCloud by sending only deltas, but the implementation is utterly broken: - There is no baseline state(s). The database is literally the playback of every delta, with no canonical snapshots. Ever. - iCloud is really bad at sequencing the deltas, especially if (somewhat) concurrent changes are happening. Core Data is also terrifyingly bad at conflict resolution when deltas from different devices don't line up. - The above points means that it's trivially easy to break your entire database, as your deltas no longer play back to a sane database file. When this occurs there is no way to roll back your database, since there is no concept of a snapshot or last known good state. - This is compounded by some really horrible API design on Apple's part. When your iCloud database becomes corrupt it fails silently, with absolutely no API that allows one to determine said broken state. Your app literally stops working with none of the OS making a single peep, and no way for you to proactively check your database's state. iCloud seems to print some debug things into your console indicating this failure, but neither your app nor your users have access to it. This manifests in your app (and in tech support requests) as "iCloud just stopped syncing" - your device stops transmitting deltas and simply works off of the last locally-cached good state, all without telling the dev or the user. - This is further compounded by the fact that the entirety of Core Data+iCloud was rushed out the door. Documentation was non-existent even up to the day the new version of iOS went out to the public. There was literally only one source of information: a thread on the Apple dev forums that Apple engineers would sometimes pop into. - This is further compounded by the fact that even the Apple engineers themselves seemed to have no idea how to implement an app with iCloud + Core Data. The example code (provided deep, deep within aforementioned thread) made 90 degree turns seemingly every few days. The same sample would have dramatically changing implementations from week to week. The Apple engineers in the thread seemed as mystified as the people trying to use the framework. - The poor documentation and poor state of sample code, as well as what is IMO a fundamentally broken architecture means that iCloud + Core Data remains a complete nightmare to this day. I think the poor architecture is really the core of this here - while Apple could have provided better docs and better support, I believe the fundamental architecture choices made here means Core Data + iCloud would never have been workable. - The icing on top of the cake was an extremely egregious bug that existed at launch (unsure now, you couldn't pay me enough money to work with that API ever again). "Okay, so our deltas are completely fucked" you say, "Why don't we just delete the app's entire iCloud bucket and start over?" A jolly good idea until you realize that a bug in iOS/iCloud means that when you delete an app's iCloud storage "bin", it cannot be recreated by that app, ever again.
  - hayksaakian 14y ago
    that's worse than I could have imagined
    - asveikau 14y ago
      I've not used the APIs in question, but I'm not sure how this can be totally unexpected, given the marketing promises and the actual technical goals. I read your comment as: "Transparent distributed storage? What could possibly go wrong!"
      - damncabbage 14y ago
        Wisty said it best in another post: "All sync is horrible. iCould unnecessarily so." (http://news.ycombinator.com/item?id=5185428 http://news.ycombinator.com/item?id=5185428)
      - hayksaakian 14y ago
        My main problem was that it only stored deltas, and thus you could not revert to any specific state if any of the intermediate deltas were lost. Compare this to something like google docs, which has ways of handling reverts and simultaneous editing.
  - gfodor 14y ago
    wow. thanks for writing that up. if it's any consolation, you likely just saved me untold amounts of pain as i am wrangling with what approach to take for saving a small but not trivial amount of structured data in my app. this pretty much rules out core data!
    - barumrho 14y ago
      I wouldn't rule out core data just from this post. Core data may have some quirks, but it's pretty reliable and very useful from my experience. I haven't given iCloud + core data a try, since I wasn't sure if it were reliable yet.
    - 14y ago
cpr 14y ago
I've heard these rumblings for quite a while--haven't done anything with iCloud support myself--, and, yes, we need some details of what's not working. Or maybe the article's implying no one really knows?
adjwilli 14y ago
Yeah with my apps I haven't noticed many iCloud related problems. I generally however just store simple keys in iCloud then pull the real data from my server based on those. It's not as robust as I'd like it but works.
firloop 14y ago
A quick Google search provides this article: http://createlivelove.com/238/why-i-hate-icloud/ http://createlivelove.com/238/why-i-hate-icloud/ that explains his hatred, but it's not terribly detailed either.
- jonknee 14y ago
  It's also linked in the OP...
robterrell 14y ago
Instead of waiting two years, why not implement your own sync service? I know that sounds like snark, but using something like CouchDB and TouchDB can give your users an experience similar to the one promised by iCloud.
- wisty 14y ago
  There's also parse.io, which is apparently not as horrible as iCloud. (All sync is horrible. iCould unnecessarily so).
- DeepDuh 14y ago
  I'm very interested in those two solutions. Does anyone here have some experience about CouchDB/TouchDB on iOS/Android? Specifically: Does it replicate reliably to a CouchDB or Couchbase server? How are conflicts handled? Are there wrappers already available that map to ObjC/Java native data structures, similar to couchdbkit on python?
  - greendestiny 14y ago
    Hey yeah I've used TouchDB on iOS quite a bit. It syncs reliably to Couchdb - I actually haven't tried couchbase. Conflicts are handled in the default couchdb way I think - I haven't really touched on this much. TouchDB syncs to native structures that are derived from CouchModel.
    - DeepDuh 14y ago
      Great, thanks for the helpful info! Oh, one more thing: Did you ever load multiple TouchDBs? I'm thinking the overhead per DB (not per server) should be pretty small, no? Background: I'd like to build an application where I separate entities into multiple couchDBs, partly since it saves me view definition code, partly because it gives me more flexibility for per-document access permissions.
      - greendestiny 14y ago
        Yes I have a framework that splits itself into 3 databases and has quite a lot of attachments. There isn't a big overhead as far as I can see. Does couchDB do what you need with per-document access permissions? It seems to be the biggest downside to couchdb is having to do so many workarounds for per-document access permissions.
stcredzero 14y ago
I was just about to try and convert my app to use iCloud, and incorporate the features in a 2nd one. After going to the 2011 conference, I followed the message and took the extra effort to put the 1st app's information in Core Data, and then only afterwards heard how bad iCloud was. Now I'm thinking of moving my data back to a document-based architecture. Now there's grousing that even that sucks. If iCloud was hitting on all cylinders, then Apple could kill MS Surface by having seamless sync between laptops and tablets. Instead of a chunky tablet/laptop that's mediocre at being both, you can have a fabulous tablet and a great ultra-lightweight laptop and not have to worry about syncing. Need to pass around something to show a coworker? It could just work. This is a sign that Apple is either not hitting on all cylinders, or bit off more than it could chew. (To be fair, distributed data is, and always has been, quite a hard problem.) Apple could be moving in for the kill. Instead, they're recovering from stumbles while their opponent has left themselves wide open.
- taligent 14y ago
  Syncing does generally just work between laptop and tablet. I add a Safari bookmark on one and it appears instantly on the other. Likewise for contacts, calendars, apps, music and any iCloud enabled app e.g. Coda. Then just add Dropbox and Evernote and you're laughing.
  - stcredzero 14y ago
    Then why is there so much grousing in the App Developer community?
    - interpol_p 14y ago
      It's the combination of Core Data + iCloud that is the biggest problem. Using the document model (as long as it's not a Core Data document) with iCloud is fine, as is the Key-Value store. Using Core Data on its own is also fine, it's a great framework and we use it a lot.
      - stcredzero 14y ago
doe88 14y ago
The term iCloud regroups several technologies. As far as I know the simplest one a basic key-value store works very well, I use it without any problems. However, last year I tried to integrate CoreData syncing through iCloud in one of my apps and I couldn't make it work reliabily, I spent a lot of times I tried many things, but usually there always was some irrecoverable corruptions that ended-up preventing any syncing. I never shipped this code, I was too afraid, it was too brittle. I haven't retried since. I heard the situation has improved but I don't know for sure, it was such a nightmare that I simply avoid to spend any more time on this api. Edit: I found this quite recent post with similar issues http://www.jumsoft.com/2013/01/response-to-sync-issues/ http://www.jumsoft.com/2013/01/response-to-sync-issues/
seivan 14y ago
I write my own backends. I feel safer that way. Already got a framework for communicating with a resource based API properly with its local model (Always Core Data unless not trivial like a map).
andrewljohnson 14y ago
Sorry, but it sounds more like user error. I won't argue that CoreData has its pitfalls, but at the end of the day, many of us both write our own syncing solutions, and successfully use iCloud. Neither of these things is actually all that hard, and you certainly don't need to use iCloud to have your apps featured.
- igravious 14y ago
  Uh, see this link up yonder: http://news.ycombinator.com/item?id=5185497 http://news.ycombinator.com/item?id=5185497
  - taligent 14y ago
    The OP said CoreData has pitfalls and they are 100% correct i.e. the link you provided. The OP said you can roll your solution using iCloud and it will work and they are 100% correct. The OP said you don't need iCloud to be featured and they are 100% correct. Which point is wrong exactly ?
bitcartel 14y ago
I posted this exact same link a few hours ago, and it was accepted as a new submission. So what's going on? Can moderators reassign story submissions?
- nkurz 14y ago
  All of your submissions for the last week have been autokilled. Something must have triggered a spam detector. Or maybe the Youtube submission (brilliant video) did it. Your comments are still visible, though. Writing polite email to PG is probably your best recourse.
cloudmike 14y ago
To all those suffering from Core Data + iCloud syncing: we built Simperium (YC S10) to solve this problem. The iOS/OSX and JavaScript libraries are open source, with the backend to follow (or you can use the hosted service). The site is at http://simperium.com http://simperium.com and the repos are at http://github.com/Simperium http://github.com/Simperium We were recently acquired by Automattic, and we're now putting more resources into the project (for example to make it work on Android as well). If cost for the hosted service is a concern for you, you can get in touch and we'll work something out (info@simperium.com).
archagon 14y ago
I've never worked with iCloud, so I'm curious: why do devs say that Dropbox is so much better than iCloud? From what I've seen, both work similarly in that they abstract out the cloud as part of the file system. Wouldn't you still have to resolve conflicts and do all that other dirty stuff if you use Dropbox? Edit: potatolicious has a pretty great explanation. It looks like it's mostly CoreData issues. But for those cases, Dropbox wouldn't do any better, right?
- rdouble 14y ago
  Dropbox isn't a great substitute for iCloud when it comes to CoreData because it only does file syncing.

46 points by mikecane 14y ago

it would be helpful if this article included some actual details about what doesn't work for people considering adding iCloud to their apps.

stevenp 14y ago
Agreed. This article is completely useless.
potatolicious 14y ago
iCloud in general "just works". It was pretty terrible from the get-go, from reliability to really egregious bugs in the OS, but at this point is pretty stable. The real problem here is the combination of iCloud and Core Data (for non-Apple folks, the built-in persistence/object model framework). In a somewhat simplified way, Core Data is a data store most commonly backed by SQLite. It's used everywhere from your mail app to the Facebook app. It's handy and convenient when it isn't trying to kill you - it's one of the more complex frameworks that Apple ships, and the documentation is extremely poor for it. It also doesn't help that there is a lot of quirky behavior that's more or less tribal knowledge to long-time Apple devs and WWDC attendees, and is has enough concurrency gotchas to fill a container ship. So, Core Data is not the simplest piece of code to work with from the get-go, and then you pile iCloud on top. iCloud is conceptually pretty simple - there's a directory on your file system that is replicated to Apple's servers, and back down to each one of your devices. When files change the OS sends notifications to your app informing you of such. If you're using iCloud as a distributed file system (a la Dropbox), it's easy to integrate. The trouble with this model is that Core Data stores can be in fact quite large. If you have a 30MB database of, say, recipes on your disk, you really don't want to resend the whole thing every time your user alters a row. Apple special-cases Core Data stores in iCloud by sending only deltas, but the implementation is utterly broken: - There is no baseline state(s). The database is literally the playback of every delta, with no canonical snapshots. Ever. - iCloud is really bad at sequencing the deltas, especially if (somewhat) concurrent changes are happening. Core Data is also terrifyingly bad at conflict resolution when deltas from different devices don't line up. - The above points means that it's trivially easy to break your entire database, as your deltas no longer play back to a sane database file. When this occurs there is no way to roll back your database, since there is no concept of a snapshot or last known good state. - This is compounded by some really horrible API design on Apple's part. When your iCloud database becomes corrupt it fails silently, with absolutely no API that allows one to determine said broken state. Your app literally stops working with none of the OS making a single peep, and no way for you to proactively check your database's state. iCloud seems to print some debug things into your console indicating this failure, but neither your app nor your users have access to it. This manifests in your app (and in tech support requests) as "iCloud just stopped syncing" - your device stops transmitting deltas and simply works off of the last locally-cached good state, all without telling the dev or the user. - This is further compounded by the fact that the entirety of Core Data+iCloud was rushed out the door. Documentation was non-existent even up to the day the new version of iOS went out to the public. There was literally only one source of information: a thread on the Apple dev forums that Apple engineers would sometimes pop into. - This is further compounded by the fact that even the Apple engineers themselves seemed to have no idea how to implement an app with iCloud + Core Data. The example code (provided deep, deep within aforementioned thread) made 90 degree turns seemingly every few days. The same sample would have dramatically changing implementations from week to week. The Apple engineers in the thread seemed as mystified as the people trying to use the framework. - The poor documentation and poor state of sample code, as well as what is IMO a fundamentally broken architecture means that iCloud + Core Data remains a complete nightmare to this day. I think the poor architecture is really the core of this here - while Apple could have provided better docs and better support, I believe the fundamental architecture choices made here means Core Data + iCloud would never have been workable. - The icing on top of the cake was an extremely egregious bug that existed at launch (unsure now, you couldn't pay me enough money to work with that API ever again). "Okay, so our deltas are completely fucked" you say, "Why don't we just delete the app's entire iCloud bucket and start over?" A jolly good idea until you realize that a bug in iOS/iCloud means that when you delete an app's iCloud storage "bin", it cannot be recreated by that app, ever again.
- hayksaakian 14y ago
  that's worse than I could have imagined
  - asveikau 14y ago
    I've not used the APIs in question, but I'm not sure how this can be totally unexpected, given the marketing promises and the actual technical goals. I read your comment as: "Transparent distributed storage? What could possibly go wrong!"
    - damncabbage 14y ago
      Wisty said it best in another post: "All sync is horrible. iCould unnecessarily so." (http://news.ycombinator.com/item?id=5185428 http://news.ycombinator.com/item?id=5185428)
    - hayksaakian 14y ago
      My main problem was that it only stored deltas, and thus you could not revert to any specific state if any of the intermediate deltas were lost. Compare this to something like google docs, which has ways of handling reverts and simultaneous editing.
- gfodor 14y ago
  wow. thanks for writing that up. if it's any consolation, you likely just saved me untold amounts of pain as i am wrangling with what approach to take for saving a small but not trivial amount of structured data in my app. this pretty much rules out core data!
  - barumrho 14y ago
    I wouldn't rule out core data just from this post. Core data may have some quirks, but it's pretty reliable and very useful from my experience. I haven't given iCloud + core data a try, since I wasn't sure if it were reliable yet.
  - 14y ago

cpr 14y ago

I've heard these rumblings for quite a while--haven't done anything with iCloud support myself--, and, yes, we need some details of what's not working. Or maybe the article's implying no one really knows?

adjwilli 14y ago

Yeah with my apps I haven't noticed many iCloud related problems. I generally however just store simple keys in iCloud then pull the real data from my server based on those. It's not as robust as I'd like it but works.

firloop 14y ago

A quick Google search provides this article: http://createlivelove.com/238/why-i-hate-icloud/ http://createlivelove.com/238/why-i-hate-icloud/ that explains his hatred, but it's not terribly detailed either.

jonknee 14y ago
It's also linked in the OP...

robterrell 14y ago

Instead of waiting two years, why not implement your own sync service? I know that sounds like snark, but using something like CouchDB and TouchDB can give your users an experience similar to the one promised by iCloud.

wisty 14y ago
There's also parse.io, which is apparently not as horrible as iCloud. (All sync is horrible. iCould unnecessarily so).
DeepDuh 14y ago
I'm very interested in those two solutions. Does anyone here have some experience about CouchDB/TouchDB on iOS/Android? Specifically: Does it replicate reliably to a CouchDB or Couchbase server? How are conflicts handled? Are there wrappers already available that map to ObjC/Java native data structures, similar to couchdbkit on python?
- greendestiny 14y ago
  Hey yeah I've used TouchDB on iOS quite a bit. It syncs reliably to Couchdb - I actually haven't tried couchbase. Conflicts are handled in the default couchdb way I think - I haven't really touched on this much. TouchDB syncs to native structures that are derived from CouchModel.
  - DeepDuh 14y ago
    Great, thanks for the helpful info! Oh, one more thing: Did you ever load multiple TouchDBs? I'm thinking the overhead per DB (not per server) should be pretty small, no? Background: I'd like to build an application where I separate entities into multiple couchDBs, partly since it saves me view definition code, partly because it gives me more flexibility for per-document access permissions.
    - greendestiny 14y ago
      Yes I have a framework that splits itself into 3 databases and has quite a lot of attachments. There isn't a big overhead as far as I can see. Does couchDB do what you need with per-document access permissions? It seems to be the biggest downside to couchdb is having to do so many workarounds for per-document access permissions.

stcredzero 14y ago

I was just about to try and convert my app to use iCloud, and incorporate the features in a 2nd one. After going to the 2011 conference, I followed the message and took the extra effort to put the 1st app's information in Core Data, and then only afterwards heard how bad iCloud was. Now I'm thinking of moving my data back to a document-based architecture. Now there's grousing that even that sucks. If iCloud was hitting on all cylinders, then Apple could kill MS Surface by having seamless sync between laptops and tablets. Instead of a chunky tablet/laptop that's mediocre at being both, you can have a fabulous tablet and a great ultra-lightweight laptop and not have to worry about syncing. Need to pass around something to show a coworker? It could just work. This is a sign that Apple is either not hitting on all cylinders, or bit off more than it could chew. (To be fair, distributed data is, and always has been, quite a hard problem.) Apple could be moving in for the kill. Instead, they're recovering from stumbles while their opponent has left themselves wide open.

taligent 14y ago
Syncing does generally just work between laptop and tablet. I add a Safari bookmark on one and it appears instantly on the other. Likewise for contacts, calendars, apps, music and any iCloud enabled app e.g. Coda. Then just add Dropbox and Evernote and you're laughing.
- stcredzero 14y ago
  Then why is there so much grousing in the App Developer community?
  - interpol_p 14y ago
    It's the combination of Core Data + iCloud that is the biggest problem. Using the document model (as long as it's not a Core Data document) with iCloud is fine, as is the Key-Value store. Using Core Data on its own is also fine, it's a great framework and we use it a lot.
    - stcredzero 14y ago

doe88 14y ago

The term iCloud regroups several technologies. As far as I know the simplest one a basic key-value store works very well, I use it without any problems. However, last year I tried to integrate CoreData syncing through iCloud in one of my apps and I couldn't make it work reliabily, I spent a lot of times I tried many things, but usually there always was some irrecoverable corruptions that ended-up preventing any syncing. I never shipped this code, I was too afraid, it was too brittle. I haven't retried since. I heard the situation has improved but I don't know for sure, it was such a nightmare that I simply avoid to spend any more time on this api. Edit: I found this quite recent post with similar issues http://www.jumsoft.com/2013/01/response-to-sync-issues/ http://www.jumsoft.com/2013/01/response-to-sync-issues/

seivan 14y ago

I write my own backends. I feel safer that way. Already got a framework for communicating with a resource based API properly with its local model (Always Core Data unless not trivial like a map).

andrewljohnson 14y ago

Sorry, but it sounds more like user error. I won't argue that CoreData has its pitfalls, but at the end of the day, many of us both write our own syncing solutions, and successfully use iCloud. Neither of these things is actually all that hard, and you certainly don't need to use iCloud to have your apps featured.

igravious 14y ago
Uh, see this link up yonder: http://news.ycombinator.com/item?id=5185497 http://news.ycombinator.com/item?id=5185497
- taligent 14y ago
  The OP said CoreData has pitfalls and they are 100% correct i.e. the link you provided. The OP said you can roll your solution using iCloud and it will work and they are 100% correct. The OP said you don't need iCloud to be featured and they are 100% correct. Which point is wrong exactly ?

bitcartel 14y ago

I posted this exact same link a few hours ago, and it was accepted as a new submission. So what's going on? Can moderators reassign story submissions?

nkurz 14y ago
All of your submissions for the last week have been autokilled. Something must have triggered a spam detector. Or maybe the Youtube submission (brilliant video) did it. Your comments are still visible, though. Writing polite email to PG is probably your best recourse.

cloudmike 14y ago

To all those suffering from Core Data + iCloud syncing: we built Simperium (YC S10) to solve this problem. The iOS/OSX and JavaScript libraries are open source, with the backend to follow (or you can use the hosted service). The site is at http://simperium.com http://simperium.com and the repos are at http://github.com/Simperium http://github.com/Simperium We were recently acquired by Automattic, and we're now putting more resources into the project (for example to make it work on Android as well). If cost for the hosted service is a concern for you, you can get in touch and we'll work something out (info@simperium.com).

archagon 14y ago

I've never worked with iCloud, so I'm curious: why do devs say that Dropbox is so much better than iCloud? From what I've seen, both work similarly in that they abstract out the cloud as part of the file system. Wouldn't you still have to resolve conflicts and do all that other dirty stuff if you use Dropbox? Edit: potatolicious has a pretty great explanation. It looks like it's mostly CoreData issues. But for those cases, Dropbox wouldn't do any better, right?

rdouble 14y ago
Dropbox isn't a great substitute for iCloud when it comes to CoreData because it only does file syncing.