4 ms·

Genuinely interested in why you guys decided to roll your own cloud sync service instead of using something like dropbox or the like. I recently started using

by malvim 12y ago

Genuinely interested in why you guys decided to roll your own cloud sync service instead of using something like dropbox or the like. I recently started using YNAB (youneedabudget.com) for OSX and iPhone, and one of the things I liked the most is how they used dropbox to sync flawlessly between devices. It's probably hard enough doing sync right even with the help of dropbox API, but it just seems to me that rolling your own makes it two problems instead of one. Care to share your reasoning? Thanks!

jafitc 12y ago
You are as weak as the weakest link in your system. The more you rely on 3rd party services the more likely you're going to be burned by them. If you have sync under your control, you can guarantee a certain level of user experience. Is it more work? Definitely. Is it worth it? It depends on what your priorities are.
- gress 12y ago
  You have this the wrong way around. If you roll your own sync service, then you become the weak link in the user's system. Given how much effort it takes to get this stuff right and to maintain the service, it is irresponsible to do it yourself unless there are features you cannot otherwise deliver. It's far better from the user's perspective for developers to use Dropbox or even iCloud, because the chances of them abandoning or losing user data, or simply failing as a business, are much slimmer than those of an app developer selected at random.
  - jafitc 12y ago
    If you've never heard of the horror stories, I invite you to read up on that. It's not about the company going out of business, it's about having issues with a "black box" that you can't possibly debug or fix. The things you don't have control over will end up breaking your promise to the user. http://www.theverge.com/2013/3/26/4148628/why-doesnt-icloud-just-work http://www.theverge.com/2013/3/26/4148628/why-doesnt-icloud-... http://arstechnica.com/apple/2013/03/frustrated-with-icloud-apples-developer-community-speaks-up-en-masse/ http://arstechnica.com/apple/2013/03/frustrated-with-icloud-...
    - gress 12y ago
      This isn't about trying to defend iCloud's teething problems, so these links are irrelevant. Yes, there are risks associated with using a 3rd party service. However if you don't use one, you are claiming that you can do better in architecture, implementation, and DevOps. How many apps do you have? Let's say you have 30. What percentage of the developers of these apps do you think will do a better job of sync than Dropbox or Apple? Unless you think that number is in the high 90's, then your advice is a severe disservice to end users.
      - jafitc 12y ago
        You cannot shrug real problems off as "irrelevant". Let me do a comparison to highlight how relevant it is: 1. iCloud: A user experiences an issue. Contacts your support. You finally figure out that the problem is caused by a system you have no control over. You tell the user a) there's nothing you can do about it b) they should prepare for it happening in the future. The user is unhappy. This repeats over and over again. 2. Custom sync: A user experiences an issue. You go to your server, find the root cause and fix it. The user is happy, and you also fixed problems for all the other users with the same issue. So this particular situation is never repeated again. I've never said everything in your system had to be custom made. I emphasized, "it depends on what your priorities are". If you priority is customer experience, then you can take the huge field of syncing, slice off a little manageable portion of it -- that makes sense for your app -- and attack it with elegance. See you don't have to do a better iCloud, you just have to do a better slice of the syncing pie. Big difference. Clearly not all developers are suited for this. But in a lot of cases, it is not only possible, but even advisable at the moment. Vesper is a good example. (scenario number (1.) is not just a hypothetical situation you just have to look at the mountains of evidence: lot of apps had to remove iCloud syncing, because of the tons of complaints that flooded their support channels)
        gress 12y ago
        Do you know anything about current icloud stability for document syncing? I agree that scenario 1 is not hypothetical, and that iCloud was too unusable unstable for the first two years. I personally avoided including it in my apps for that reason, to my great disappointment. However, it is routinely incorporated in new apps now, and as and end user I have had no serious problems since iOS7. As a user I'd far rather trust iCloud or Dropbox than yet another sync service. I've read Brent Simmons articles on how he did syncing for vesper, and it seems like he's done it well. Having said that, there is no indication of how well it will scale, and more importantly Brent has the experience to do this job well, whereas most App Developers do not.
adolph 12y ago
You might be interested in one of the team's writings on the subject: http://inessential.com/vespersyncdiary http://inessential.com/vespersyncdiary There is a deep dive on the granularity of syncing that hints at why they aren't doing Dropbox file-level syncing: http://inessential.com/2013/11/05/vesper_sync_diary_3_immutability_del http://inessential.com/2013/11/05/vesper_sync_diary_3_immuta...
jgruber 12y ago
As the other posters have remarked, Brent's sync diary provides some of the concrete technical reasons why we rolled our own. Big picture, though, we consider sync so essential to Vesper's long-term success that we wanted to control it. Dropbox is fantastic. We all rely on it personally. But we don't want Vesper to rely on it. I'm sure everyone here on Hacker News has a Dropbox account, and I'll bet most Daring Fireball readers do too. But we want Vesper to appeal to everyone, including people who don't know what Dropbox is or don't want it. (Plus, Dropbox doesn't work exactly the way we want it to work; we have code running on our servers, not just data stored on them. Vesper Sync works exactly the way we want sync to work.) iCloud is very tempting too, because it puts all the onus of identity, privacy, and security on Apple's shoulders. But with iCloud, you have to choose between document syncing and Core Data -- neither of which is a perfect fit for the way Vesper is designed. Lastly, we want to keep the door open for a web app version of Vesper, and a non-Mac App Store version for OS X. iCloud, by design, only works for apps distributed through the App Store. We wanted to offer one and only one good way to sync. That meant rolling our own.
- Holbein 12y ago
  In other words, you agree that iCloud's attempt at vendor lock-in is bad for developers and consumers alike.
  - jgruber 12y ago
    Not at all. I would only argue that iCloud is not a good fit for all apps, and Vesper is one of them.
  - Holbein 12y ago
    I stand by my statement. Your feeble attempts and downvoting me into light grey can't silence me! Stupendous man will save the day again! :-P
- malvim 12y ago
  That makes a lot of sense. What I was thinking about was just how many times I gave up on an app because it made me "sign up" for something just to create simple documents, play a game (sometimes even ones that should work offline!), or take notes. But, obviously, in your case it seems having more control clearly outweights these small problem-user cases. ;) Congrats on shipping, good luck! EDIT: Also, making syncing optional, as I just read you saying you would, makes this problem go away.

by malvim 12y ago

jafitc 12y ago

You are as weak as the weakest link in your system. The more you rely on 3rd party services the more likely you're going to be burned by them. If you have sync under your control, you can guarantee a certain level of user experience. Is it more work? Definitely. Is it worth it? It depends on what your priorities are.

gress 12y ago
You have this the wrong way around. If you roll your own sync service, then you become the weak link in the user's system. Given how much effort it takes to get this stuff right and to maintain the service, it is irresponsible to do it yourself unless there are features you cannot otherwise deliver. It's far better from the user's perspective for developers to use Dropbox or even iCloud, because the chances of them abandoning or losing user data, or simply failing as a business, are much slimmer than those of an app developer selected at random.
- jafitc 12y ago
  If you've never heard of the horror stories, I invite you to read up on that. It's not about the company going out of business, it's about having issues with a "black box" that you can't possibly debug or fix. The things you don't have control over will end up breaking your promise to the user. http://www.theverge.com/2013/3/26/4148628/why-doesnt-icloud-just-work http://www.theverge.com/2013/3/26/4148628/why-doesnt-icloud-... http://arstechnica.com/apple/2013/03/frustrated-with-icloud-apples-developer-community-speaks-up-en-masse/ http://arstechnica.com/apple/2013/03/frustrated-with-icloud-...
  - gress 12y ago
    This isn't about trying to defend iCloud's teething problems, so these links are irrelevant. Yes, there are risks associated with using a 3rd party service. However if you don't use one, you are claiming that you can do better in architecture, implementation, and DevOps. How many apps do you have? Let's say you have 30. What percentage of the developers of these apps do you think will do a better job of sync than Dropbox or Apple? Unless you think that number is in the high 90's, then your advice is a severe disservice to end users.
    - jafitc 12y ago
      You cannot shrug real problems off as "irrelevant". Let me do a comparison to highlight how relevant it is: 1. iCloud: A user experiences an issue. Contacts your support. You finally figure out that the problem is caused by a system you have no control over. You tell the user a) there's nothing you can do about it b) they should prepare for it happening in the future. The user is unhappy. This repeats over and over again. 2. Custom sync: A user experiences an issue. You go to your server, find the root cause and fix it. The user is happy, and you also fixed problems for all the other users with the same issue. So this particular situation is never repeated again. I've never said everything in your system had to be custom made. I emphasized, "it depends on what your priorities are". If you priority is customer experience, then you can take the huge field of syncing, slice off a little manageable portion of it -- that makes sense for your app -- and attack it with elegance. See you don't have to do a better iCloud, you just have to do a better slice of the syncing pie. Big difference. Clearly not all developers are suited for this. But in a lot of cases, it is not only possible, but even advisable at the moment. Vesper is a good example. (scenario number (1.) is not just a hypothetical situation you just have to look at the mountains of evidence: lot of apps had to remove iCloud syncing, because of the tons of complaints that flooded their support channels)
      - gress 12y ago
        Do you know anything about current icloud stability for document syncing? I agree that scenario 1 is not hypothetical, and that iCloud was too unusable unstable for the first two years. I personally avoided including it in my apps for that reason, to my great disappointment. However, it is routinely incorporated in new apps now, and as and end user I have had no serious problems since iOS7. As a user I'd far rather trust iCloud or Dropbox than yet another sync service. I've read Brent Simmons articles on how he did syncing for vesper, and it seems like he's done it well. Having said that, there is no indication of how well it will scale, and more importantly Brent has the experience to do this job well, whereas most App Developers do not.

adolph 12y ago

You might be interested in one of the team's writings on the subject: http://inessential.com/vespersyncdiary http://inessential.com/vespersyncdiary There is a deep dive on the granularity of syncing that hints at why they aren't doing Dropbox file-level syncing: http://inessential.com/2013/11/05/vesper_sync_diary_3_immutability_del http://inessential.com/2013/11/05/vesper_sync_diary_3_immuta...

jgruber 12y ago

As the other posters have remarked, Brent's sync diary provides some of the concrete technical reasons why we rolled our own. Big picture, though, we consider sync so essential to Vesper's long-term success that we wanted to control it. Dropbox is fantastic. We all rely on it personally. But we don't want Vesper to rely on it. I'm sure everyone here on Hacker News has a Dropbox account, and I'll bet most Daring Fireball readers do too. But we want Vesper to appeal to everyone, including people who don't know what Dropbox is or don't want it. (Plus, Dropbox doesn't work exactly the way we want it to work; we have code running on our servers, not just data stored on them. Vesper Sync works exactly the way we want sync to work.) iCloud is very tempting too, because it puts all the onus of identity, privacy, and security on Apple's shoulders. But with iCloud, you have to choose between document syncing and Core Data -- neither of which is a perfect fit for the way Vesper is designed. Lastly, we want to keep the door open for a web app version of Vesper, and a non-Mac App Store version for OS X. iCloud, by design, only works for apps distributed through the App Store. We wanted to offer one and only one good way to sync. That meant rolling our own.

Holbein 12y ago
In other words, you agree that iCloud's attempt at vendor lock-in is bad for developers and consumers alike.
- jgruber 12y ago
  Not at all. I would only argue that iCloud is not a good fit for all apps, and Vesper is one of them.
- Holbein 12y ago
  I stand by my statement. Your feeble attempts and downvoting me into light grey can't silence me! Stupendous man will save the day again! :-P
malvim 12y ago
That makes a lot of sense. What I was thinking about was just how many times I gave up on an app because it made me "sign up" for something just to create simple documents, play a game (sometimes even ones that should work offline!), or take notes. But, obviously, in your case it seems having more control clearly outweights these small problem-user cases. ;) Congrats on shipping, good luck! EDIT: Also, making syncing optional, as I just read you saying you would, makes this problem go away.