3 ms·

Hey Mark, I'm interested in several different features. First off, you make a great point about "Hosting your own database is a pain" and/or expensive. I'm sur

Hey Mark, I'm interested in several different features. First off, you make a great point about "Hosting your own database is a pain" and/or expensive. I'm sure you know about the raging debate that has been happening between MapReduce and Parallel DBs, and a big part of the reason that MapReduce (Hadoop) style has had so much support is because it's easy and cheap. In lots of instances it may run slower, but why pay for the hassle when you can just run Hadoop for free (not including server costs)? You're making the same point here. Whats the threshold on money savings? How much does it cost to use a DaaS (SQL or NoSQL) compared to what you want to do with Gun/Redis? (Also, is this going to be open source?) Next, the conflict resolution should be interesting. What kind of eventual consistency guarantees are you hoping to have? I see you're planning on addressing this. I'm also interested in the querying. That is, are you using Javascript? My problem with what you're saying about current query languages is that it's not really that hard to get the bare usage out of them. Simple queries in SQL are extremely straightforward. In addition, declarative languages really make things easier for the programmer, not harder. Who wants to have to deal with all the nuances of a join? Certainly not me.

marknadal 13y ago
Realistically I won't have cost-comparisons for a while, as this project is just in the infancy and obviously needs to stabilize/mature first. Although as a quick summary, your costs should approximately be server(s) that are expensive enough to fit your smallest set of per-user active-data, plus your S3 storage amount, plus S3 API calls (I've made this part a simple options parameter, so you control how often these calls are being made - the more frequent, the more integrity from worst-case disruptions but also more expensive, less frequent and your costs go down). What is worst-case disruption? Everything goes offline simultaneously. User's don't have localstorage fallbacks in their phone/browser so retries rely on the server cache. The server cache is running the default method, not Redis - and then your server crashes or the machine goes under. And finally, of course, there is an S3 outage and you only persisted to 1 region, not multiple regions. I'll get into hairy situations like that in some of my following posts, but too much for here. Yes, you have another good point about query languages. I agree with you very much - somebody else commented about this, check out my reply to muaddirac. Please keep in touch with more questions/comments, or even email me!

marknadal 13y ago

Realistically I won't have cost-comparisons for a while, as this project is just in the infancy and obviously needs to stabilize/mature first. Although as a quick summary, your costs should approximately be server(s) that are expensive enough to fit your smallest set of per-user active-data, plus your S3 storage amount, plus S3 API calls (I've made this part a simple options parameter, so you control how often these calls are being made - the more frequent, the more integrity from worst-case disruptions but also more expensive, less frequent and your costs go down). What is worst-case disruption? Everything goes offline simultaneously. User's don't have localstorage fallbacks in their phone/browser so retries rely on the server cache. The server cache is running the default method, not Redis - and then your server crashes or the machine goes under. And finally, of course, there is an S3 outage and you only persisted to 1 region, not multiple regions. I'll get into hairy situations like that in some of my following posts, but too much for here. Yes, you have another good point about query languages. I agree with you very much - somebody else commented about this, check out my reply to muaddirac. Please keep in touch with more questions/comments, or even email me!