9 ms·

I've worked with iCloud in conjunction with Core Data. It's as broken as the article describes, and in fact has gotten worse over time. No one really fully kno

I've worked with iCloud in conjunction with Core Data. It's as broken as the article describes, and in fact has gotten worse over time. No one really fully knows WTF is going with iCloud - documentation is basically nonexistent, support unavailable, implementation broken in trivial ways, and error messages inscrutable. I was at an Mac/iOS conference recently where the Core Data iCloud talk morphed from a standard talk into a "here's the mic, anybody have any idea what's going on with this thing" affair. Helpful, but indicative of the state of things. IMO, from having studied this for many, many hours over many moons, is that what Apple is trying to do is fundamentally hard, if not entirely unfeasible. They're trying to replace a smart server with one that's dumb as a doorstop. More specifically, they're trying to emulate a CRUD web service with a file sync engine. Conflict resolution is left up to individual clients, since the server doesn't do any "thinking". Likewise, there is no canonical, authoritative state of the store, since the server doesn't "think", only the clients do. Apple was hoping that by shoving a bunch of diffs of your database onto the server, that clients can reliably reconstruct a sane database by playing them back - except that multiple clients are updating the diffs simultaneously and there is no server-side conflict resolution. Oh, and if stuff fails, there are no regular snapshot states to fall back to, because iCloud is a file store, not a database engine. Your whole store is now corrupt. Enjoy. Oh, and there's no way to nuke the database and start over - there are metadata files that are undocumented or that we don't even have sandbox access to, that interfere with completely destroying the database. At this point you might think "Fine! I'll go to my Settings -> iCloud -> Manage Storage menu and delete everything associated with that app and start it off on a clean slate!" And then you find out that, even today, in iOS 6.1.3 land, there is a bug from iOS 5.0 where if you delete an app's iCloud bucket it can never be created by that app, on that account, ever again. I wish I were exaggerating. In perpetuity. Presumably some state gets fucked and now this app is forever more unable to store anything on iCloud. To make even the most trivial experiment on the iCloud API involves creating a brand new app. iCloud (in so far as it relates to Core Data and document storage) isn't just buggy, isn't just poorly implemented - it's non-functional.

aidos 14y ago
Good lord, that sounds like a complete mess. I haven't been anywhere near it as a developer so I can't really speak on how bad it is from that point of view. As a user though I have absolutely no idea what is going in with Apple syncing. My wife has an iphone / mac syncing explosion that I simply can't fix. I just don't have any idea where the data is or why everything is duplicated in her iphoto collection. Or what's going to happen in when you remove something from one place. Most of all I cannot for the life of me explain to my wife why it is that when she lost her phone and restored a previous backup to a new one it DELETED ALL HER RECENT PHOTOS from off her mac without warning. Gone, forever. I watched it happen before my very eyes and it's made me want to never buy another apple product again.
- blub 14y ago
  You mean all iPhoto photos, al photos in the photos folder on the mac or something else?
- crag 14y ago
  "Good lord, that sounds like a complete mess. I haven't been anywhere near it as a developer so I can't really speak on how bad it is from that point of view." Let me tell you it's not exactly happy land for us consumers either. Apple's "cloud" is a mess. And don't even get me going about the "new" and "improved" iTunes. Frankly I pity developers, cause Apple's mess is giving them a bad name. I can't think of 1 icloud enabled app (on Windows or Mac) that I've used that worked well. The apps themselves are fine, it's the cloud integration that fails. I mean just look at Apple's own forums. After 10 years as a Mac user, I'm actually considering moving back to Windows. I'm just over it. It's like every new OS (or app) something goes missing. I'm tired of spending hours searching for replacements for features Apple has removed. And I'm tired of being force-fed services and apps (that Apple's PR states is "gonna change my life") still in beta. Ok ok //end rant
  - killerpopiller 14y ago
    I switched from linux (after 9years) to an iMac and iCloud was one of the first things I couldn't believe how poorly it works. Ubuntu One sets a standards it seems. Right click your desired folder click ubuntu one and it gets synced. with iCloud (and box, dropbox) I have to duplicate the folders, weird. I spare you the finder, sftp-integration and the keyboard-layout bitching.
seivan 14y ago
I just hope people can differentiate between Core Data and iCloud from both your comment and the article.Core Data isn't broken. It might be somewhat magical, but it's not broken.
- pifflesnort 14y ago
  I think there's a good argument for it being broken: 1) Core Data objects are not NSObject objects. You can't treat them remotely like normal objects: https://developer.apple.com/library/mac/documentation/Cocoa/Reference/CoreDataFramework/Classes/NSManagedObject_Class/Reference/NSManagedObject.html#//apple_ref/doc/uid/TP30001171-235678 https://developer.apple.com/library/mac/documentation/Cocoa/... 2) Core Data requires making your model mutable, and spreading dependencies on Core Data's abnormal objects throughout your entire code base. 3) Since the abnormal objects are spread throughout your code, and they're tied to non-thread-safe NSManagedObjectContexts, concurrency becomes rather difficult. You have to deal with multiple managed object contexts becoming out-of-sync across threads. Most people just give up and perform expensive operations like -save on the main thread. 4) Core Data merges the concepts of your on-disk model with your in-memory model, despite the fact that the two problem domains have very different requirements in terms of API design, data longevity, relational design, etc. This leads to code that is neither a clean in-memory model, nor a clean on-disk model. 5) Since Core Data objects are abnormal objects, one must produce a considerable amount of boilerplate just to define new model objects. What time you save upfront using a GUI to design your original model, you spend 10-fold in maintaining the code to support it, and in being handicapped by the contraints Core Data places on your code. 6) NSPredicate is awful. It's a brain-dead query language that often make it impossible to cleanly normalize your data store. Aggregate operations are expensive and often require O(n) cost, the query language is poorly documented and poorly specified. Core Data is a very poorly fit abstraction, but it probably works well enough for simple use-cases. Once you move into concurrency, complex/large models, aggregate operations/projections across your data, or have to deal with it failing, it falls over. After using it myself for a large project and struggling with all of these issues (and more), and watching another team do the same on their own project, I'd never make use of it again. SQL (via sqlite) is a far better abstraction for managing on-disk data, especially when doing so in a highly concurrent application where one can rely on SQLite's transaction support.
raverbashing 14y ago
From what I've heard iCloud is primarily Java (and some C++) Not that I'm blaming the technology specifically, but my impression is that Apple is being naive. Of course you can build with Java, but if you knew better and have the resources Apple has, you would be aiming somewhere else. Google can do this because the infrastructure is there. Not in the case of Apple And as this shows, the concept itself of how the service works is broken. This is smelling of 'walled garden fundamentalism' + 'technical ineptitude'
- dpratt 14y ago
  You realize that a plurality of Google's backend is built in Java, right? I know it's fashionable to hate on Java, for a lot of good reasons, but I think it's naive to dismiss the power of the platform itself. We may all have (completely valid) complaints about the outdated and verbose syntax of the language, but the actual JVM itself is fantastic. There are more scalable, fault-tolerant, high performance Java systems out there than the rest of the implementations on all other platforms, combined.
  - raverbashing 14y ago
    "You realize that a plurality of Google's backend is built in Java, right?" Yes, I know. The problem is not Java directly. Reread what I wrote Google has the infrastructure and resources to make Java work for them at their size. Because they have whole teams dedicated to build their infrastructure (FS, storage, etc) Apple has to optimize their resources. They're not going to rebuild what Google did and I'm not seeing them using, I don't know: Cassandra, Hadoop, etc What I'm saying is that Apple is probably trying to reinvent the wheel. With their resources, they should go with Scala, Clojure or use helper technologies.
    - melling 14y ago
      Apple has 72,000 employees while Google only has 54,000 employees. Apple is over 30% bigger than Google. I would guess that Apple has lots of infrastructure people too.
kybernetyk 14y ago
After seeing the fate of mac.com and me.com I didn't even consider using iCloud for 'cloud storage'/sync for our Mac app. (We went with Dropbox). Apple has a long story of half-finished features they try to force onto developers and users. And if that doesn't work the technology gets killed off. The only downside of not using shiny but questionable new features is that your app won't get featured in the App Store.
- potatolicious 14y ago
  > "The only downside of not using shiny but questionable new features is that your app won't get featured in the App Store." This is a pretty huge downside. The App Store is a huge mess - there is effectively no web frontend for it, it's tied deeply into a desktop application, and it's obtuse. This makes marketing nigh impossible and results in Apple having an enormous, and disproportionate, ability to pick winners and losers. Being featured is a Big Deal(tm).
  - deveac 14y ago
    >The App Store is a huge mess - there is effectively no web frontend for it, it's tied deeply into a desktop application, and it's obtuse. I was a little shocked when the switched to the 'card UI.' On a mobile device where there is very little real estate, they completely nerfed the amount of app information on the discover screens. Added so much physical effort and time it takes for the user to peruse a like number of apps compared to the former UI.
- vor_ 14y ago
  Dropbox only does document-based syncing, which iCloud does correctly. What technology are you referring to that Apple tried to force on developers and then killed off? The only one I can recall is garbage collection, which was never forced.
- lake_rogue 14y ago
  Perhaps rather than killing the technology off this time they tried to get rid of the person in charge of the technology. Of course, if this isn't fixed soon...
pifflesnort 14y ago
Core Data is barely functional locally, requiring an enormous amount of invasive runtime hooks to create the appearance of a "transparent" (it's anything but) object store on top of Objective-C objects. The fact that anyone, anywhere (including Apple), actually expected Core Data to work as a distributed object store is totally beyond my comprehension.
- vor_ 14y ago
  I have no experience with iCloud, but I'm pleased with Core Data for managing a persistent object graph. It automatically handles version migrations, undo, and faulting, and some interfaces can be populating with almost no code via bindings. I'm happy with it. Still, I'll be very interested to watch the Core Data presentations at WWDC.
- pooriaazimi 14y ago
  vor_: unfortunately, you're hellbanned. > I have no experience with iCloud, but I'm pleased with Core Data for managing a persistent object graph. It automatically handles version migrations, undo, and faulting, and some interfaces can be populating with almost no code via bindings. I'm happy with it. Still, I'll be very interested to watch the Core Data presentations at WWDC.
- seivan 14y ago
  Never had any issues. I guess what I build is not complex enough.
  - potatolicious 14y ago
    I work on a fairly complex Core Data implementation in a heavily multi-threaded app. It requires a lot of discipline, and we rely on a lot of learning-through-pain since the documentation beyond the most basic implementation is mostly non-existent. Core Data is pretty simple if you do all of your work on the main thread, but then your'e also blocking your main execution loop for operations that can take a long time. A large save will easily reach into >200ms land on an iOS device, and your app will feel like crap. Thread safety in NSManagedObjectContexts is somewhat documented, but has a lot of quirkiness to it. There is some general knowledge around what is and isn't thread-safe, but almost nothing surrounding how to build a multithreaded app and not be murdered by Core Data. Much of this is trial and error and observing what open source projects are doing. Once you settle into a decent threading scheme that won't kill you (probably with multiple NSManagedObjectContexts) you have to worry about communicating changes across multiple contexts - then you delve into parent-child contexts, where the document gets even more bare. Then you get into propagation delays when crossing context boundaries. Keeping your code clean through all of this is also a challenge. There are also runtime issues - our implementation in Core Data experiences unbounded growth in memory usage on iOS5, but not in iOS6, forcing us to manually mitigate memory use (which Core Data is supposed to take care of), creating even more thread-safety concerns. All in all, it's a pain, but it's also the only realistic object store/graph on the platform.
Aloisius 14y ago
IMO, from having studied this for many, many hours over many moons, is that what Apple is trying to do is fundamentally hard, if not entirely unfeasible. They're trying to replace a smart server with one that's dumb as a doorstop. More specifically, they're trying to emulate a CRUD web service with a file sync engine. Having dug through iCloud's underlying structure, I can tell you what they're doing is very close to what Microsoft's Sync Framework does - right down to the naming of some of the columns in the underlying database. In fact, if I didn't know better, I would think it was a peer implementation of the Sync Framework (which considering the iCloud server is hosted on Microsoft Azure which has native support for Sync Framework built into hosted SQL Server, isn't too far fetched). The difference is that Microsoft's Sync Framework seems to work.
- wagerlabs 14y ago
  How do you know that the iCloud server is hosted on Microsoft Azure?
  - Atanvarno 14y ago
    It's known from the start: http://www.datacenterknowledge.com/archives/2011/06/16/icloud-mystery-is-apple-using-azure-amazon/ http://www.datacenterknowledge.com/archives/2011/06/16/iclou...
gte910h 14y ago
iCloud + CoreData = Pain and Tears and Blood I just change the topic to a different backing store when the client wants "cloud persistance" and says "iCloud" without understanding the pain they're asking for. I agree, completely non-functional
sampsonjs 14y ago
That's funny, I had no clue how it's supposed to work as a user either. Or at least not in the 10 minutes I was willing to give it. The Steve Jobs' Dropbox story is even more hilarious in retrospect. Hilarious at the time too, because Jobs had no intention of providing his product(iCloud) to the philistines who use Windows.
stcredzero 14y ago
> IMO, from having studied this for many, many hours over many moons, is that what Apple is trying to do is fundamentally hard, if not entirely unfeasible. "Distributed Databases are Hard" has been a mantra for many decades, probably getting close to a half century. We have known, basically forever, that this is hard on a fundamental level. (Google "Two Generals" problem.) > Apple was hoping that by shoving a bunch of diffs of your database onto the server, that clients can reliably reconstruct a sane database by playing them back This is called a variety of things. One of the terms is "transactional replication." It can get very very sticky when there is more than one party originating write transactions, though.

aidos 14y ago

Good lord, that sounds like a complete mess. I haven't been anywhere near it as a developer so I can't really speak on how bad it is from that point of view. As a user though I have absolutely no idea what is going in with Apple syncing. My wife has an iphone / mac syncing explosion that I simply can't fix. I just don't have any idea where the data is or why everything is duplicated in her iphoto collection. Or what's going to happen in when you remove something from one place. Most of all I cannot for the life of me explain to my wife why it is that when she lost her phone and restored a previous backup to a new one it DELETED ALL HER RECENT PHOTOS from off her mac without warning. Gone, forever. I watched it happen before my very eyes and it's made me want to never buy another apple product again.

blub 14y ago
You mean all iPhoto photos, al photos in the photos folder on the mac or something else?
crag 14y ago
"Good lord, that sounds like a complete mess. I haven't been anywhere near it as a developer so I can't really speak on how bad it is from that point of view." Let me tell you it's not exactly happy land for us consumers either. Apple's "cloud" is a mess. And don't even get me going about the "new" and "improved" iTunes. Frankly I pity developers, cause Apple's mess is giving them a bad name. I can't think of 1 icloud enabled app (on Windows or Mac) that I've used that worked well. The apps themselves are fine, it's the cloud integration that fails. I mean just look at Apple's own forums. After 10 years as a Mac user, I'm actually considering moving back to Windows. I'm just over it. It's like every new OS (or app) something goes missing. I'm tired of spending hours searching for replacements for features Apple has removed. And I'm tired of being force-fed services and apps (that Apple's PR states is "gonna change my life") still in beta. Ok ok //end rant
- killerpopiller 14y ago
  I switched from linux (after 9years) to an iMac and iCloud was one of the first things I couldn't believe how poorly it works. Ubuntu One sets a standards it seems. Right click your desired folder click ubuntu one and it gets synced. with iCloud (and box, dropbox) I have to duplicate the folders, weird. I spare you the finder, sftp-integration and the keyboard-layout bitching.

seivan 14y ago

I just hope people can differentiate between Core Data and iCloud from both your comment and the article.Core Data isn't broken. It might be somewhat magical, but it's not broken.

pifflesnort 14y ago
I think there's a good argument for it being broken: 1) Core Data objects are not NSObject objects. You can't treat them remotely like normal objects: https://developer.apple.com/library/mac/documentation/Cocoa/Reference/CoreDataFramework/Classes/NSManagedObject_Class/Reference/NSManagedObject.html#//apple_ref/doc/uid/TP30001171-235678 https://developer.apple.com/library/mac/documentation/Cocoa/... 2) Core Data requires making your model mutable, and spreading dependencies on Core Data's abnormal objects throughout your entire code base. 3) Since the abnormal objects are spread throughout your code, and they're tied to non-thread-safe NSManagedObjectContexts, concurrency becomes rather difficult. You have to deal with multiple managed object contexts becoming out-of-sync across threads. Most people just give up and perform expensive operations like -save on the main thread. 4) Core Data merges the concepts of your on-disk model with your in-memory model, despite the fact that the two problem domains have very different requirements in terms of API design, data longevity, relational design, etc. This leads to code that is neither a clean in-memory model, nor a clean on-disk model. 5) Since Core Data objects are abnormal objects, one must produce a considerable amount of boilerplate just to define new model objects. What time you save upfront using a GUI to design your original model, you spend 10-fold in maintaining the code to support it, and in being handicapped by the contraints Core Data places on your code. 6) NSPredicate is awful. It's a brain-dead query language that often make it impossible to cleanly normalize your data store. Aggregate operations are expensive and often require O(n) cost, the query language is poorly documented and poorly specified. Core Data is a very poorly fit abstraction, but it probably works well enough for simple use-cases. Once you move into concurrency, complex/large models, aggregate operations/projections across your data, or have to deal with it failing, it falls over. After using it myself for a large project and struggling with all of these issues (and more), and watching another team do the same on their own project, I'd never make use of it again. SQL (via sqlite) is a far better abstraction for managing on-disk data, especially when doing so in a highly concurrent application where one can rely on SQLite's transaction support.

raverbashing 14y ago

From what I've heard iCloud is primarily Java (and some C++) Not that I'm blaming the technology specifically, but my impression is that Apple is being naive. Of course you can build with Java, but if you knew better and have the resources Apple has, you would be aiming somewhere else. Google can do this because the infrastructure is there. Not in the case of Apple And as this shows, the concept itself of how the service works is broken. This is smelling of 'walled garden fundamentalism' + 'technical ineptitude'

dpratt 14y ago
You realize that a plurality of Google's backend is built in Java, right? I know it's fashionable to hate on Java, for a lot of good reasons, but I think it's naive to dismiss the power of the platform itself. We may all have (completely valid) complaints about the outdated and verbose syntax of the language, but the actual JVM itself is fantastic. There are more scalable, fault-tolerant, high performance Java systems out there than the rest of the implementations on all other platforms, combined.
- raverbashing 14y ago
  "You realize that a plurality of Google's backend is built in Java, right?" Yes, I know. The problem is not Java directly. Reread what I wrote Google has the infrastructure and resources to make Java work for them at their size. Because they have whole teams dedicated to build their infrastructure (FS, storage, etc) Apple has to optimize their resources. They're not going to rebuild what Google did and I'm not seeing them using, I don't know: Cassandra, Hadoop, etc What I'm saying is that Apple is probably trying to reinvent the wheel. With their resources, they should go with Scala, Clojure or use helper technologies.
  - melling 14y ago
    Apple has 72,000 employees while Google only has 54,000 employees. Apple is over 30% bigger than Google. I would guess that Apple has lots of infrastructure people too.

kybernetyk 14y ago

After seeing the fate of mac.com and me.com I didn't even consider using iCloud for 'cloud storage'/sync for our Mac app. (We went with Dropbox). Apple has a long story of half-finished features they try to force onto developers and users. And if that doesn't work the technology gets killed off. The only downside of not using shiny but questionable new features is that your app won't get featured in the App Store.

potatolicious 14y ago
> "The only downside of not using shiny but questionable new features is that your app won't get featured in the App Store." This is a pretty huge downside. The App Store is a huge mess - there is effectively no web frontend for it, it's tied deeply into a desktop application, and it's obtuse. This makes marketing nigh impossible and results in Apple having an enormous, and disproportionate, ability to pick winners and losers. Being featured is a Big Deal(tm).
- deveac 14y ago
  >The App Store is a huge mess - there is effectively no web frontend for it, it's tied deeply into a desktop application, and it's obtuse. I was a little shocked when the switched to the 'card UI.' On a mobile device where there is very little real estate, they completely nerfed the amount of app information on the discover screens. Added so much physical effort and time it takes for the user to peruse a like number of apps compared to the former UI.
vor_ 14y ago
Dropbox only does document-based syncing, which iCloud does correctly. What technology are you referring to that Apple tried to force on developers and then killed off? The only one I can recall is garbage collection, which was never forced.
lake_rogue 14y ago
Perhaps rather than killing the technology off this time they tried to get rid of the person in charge of the technology. Of course, if this isn't fixed soon...

pifflesnort 14y ago

Core Data is barely functional locally, requiring an enormous amount of invasive runtime hooks to create the appearance of a "transparent" (it's anything but) object store on top of Objective-C objects. The fact that anyone, anywhere (including Apple), actually expected Core Data to work as a distributed object store is totally beyond my comprehension.

vor_ 14y ago
I have no experience with iCloud, but I'm pleased with Core Data for managing a persistent object graph. It automatically handles version migrations, undo, and faulting, and some interfaces can be populating with almost no code via bindings. I'm happy with it. Still, I'll be very interested to watch the Core Data presentations at WWDC.
pooriaazimi 14y ago
vor_: unfortunately, you're hellbanned. > I have no experience with iCloud, but I'm pleased with Core Data for managing a persistent object graph. It automatically handles version migrations, undo, and faulting, and some interfaces can be populating with almost no code via bindings. I'm happy with it. Still, I'll be very interested to watch the Core Data presentations at WWDC.
seivan 14y ago
Never had any issues. I guess what I build is not complex enough.
- potatolicious 14y ago
  I work on a fairly complex Core Data implementation in a heavily multi-threaded app. It requires a lot of discipline, and we rely on a lot of learning-through-pain since the documentation beyond the most basic implementation is mostly non-existent. Core Data is pretty simple if you do all of your work on the main thread, but then your'e also blocking your main execution loop for operations that can take a long time. A large save will easily reach into >200ms land on an iOS device, and your app will feel like crap. Thread safety in NSManagedObjectContexts is somewhat documented, but has a lot of quirkiness to it. There is some general knowledge around what is and isn't thread-safe, but almost nothing surrounding how to build a multithreaded app and not be murdered by Core Data. Much of this is trial and error and observing what open source projects are doing. Once you settle into a decent threading scheme that won't kill you (probably with multiple NSManagedObjectContexts) you have to worry about communicating changes across multiple contexts - then you delve into parent-child contexts, where the document gets even more bare. Then you get into propagation delays when crossing context boundaries. Keeping your code clean through all of this is also a challenge. There are also runtime issues - our implementation in Core Data experiences unbounded growth in memory usage on iOS5, but not in iOS6, forcing us to manually mitigate memory use (which Core Data is supposed to take care of), creating even more thread-safety concerns. All in all, it's a pain, but it's also the only realistic object store/graph on the platform.

Aloisius 14y ago

IMO, from having studied this for many, many hours over many moons, is that what Apple is trying to do is fundamentally hard, if not entirely unfeasible. They're trying to replace a smart server with one that's dumb as a doorstop. More specifically, they're trying to emulate a CRUD web service with a file sync engine. Having dug through iCloud's underlying structure, I can tell you what they're doing is very close to what Microsoft's Sync Framework does - right down to the naming of some of the columns in the underlying database. In fact, if I didn't know better, I would think it was a peer implementation of the Sync Framework (which considering the iCloud server is hosted on Microsoft Azure which has native support for Sync Framework built into hosted SQL Server, isn't too far fetched). The difference is that Microsoft's Sync Framework seems to work.

wagerlabs 14y ago
How do you know that the iCloud server is hosted on Microsoft Azure?
- Atanvarno 14y ago
  It's known from the start: http://www.datacenterknowledge.com/archives/2011/06/16/icloud-mystery-is-apple-using-azure-amazon/ http://www.datacenterknowledge.com/archives/2011/06/16/iclou...

gte910h 14y ago

iCloud + CoreData = Pain and Tears and Blood I just change the topic to a different backing store when the client wants "cloud persistance" and says "iCloud" without understanding the pain they're asking for. I agree, completely non-functional

sampsonjs 14y ago

That's funny, I had no clue how it's supposed to work as a user either. Or at least not in the 10 minutes I was willing to give it. The Steve Jobs' Dropbox story is even more hilarious in retrospect. Hilarious at the time too, because Jobs had no intention of providing his product(iCloud) to the philistines who use Windows.

stcredzero 14y ago

> IMO, from having studied this for many, many hours over many moons, is that what Apple is trying to do is fundamentally hard, if not entirely unfeasible. "Distributed Databases are Hard" has been a mantra for many decades, probably getting close to a half century. We have known, basically forever, that this is hard on a fundamental level. (Google "Two Generals" problem.) > Apple was hoping that by shoving a bunch of diffs of your database onto the server, that clients can reliably reconstruct a sane database by playing them back This is called a variety of things. One of the terms is "transactional replication." It can get very very sticky when there is more than one party originating write transactions, though.