5 ms·

This is not new. But how good is it anyways?

by dmishe 13y ago

deno 13y ago
It’s highly‐available NoSQL database (Megastore) with ACID transactions based on entity‐groups[1]. No other cloud provider has anything even remotely comparable. And it’s now available outside of the App Engine framework. I was hoping they’d make App Engine to actually run on GCE, but this is fine for now. I’m a little bit worried about this though: > You should be aware that Cloud Datastore has a serving component that runs on Google App Engine, so there will be instance hour costs. What I think is going on is they are either worried about about DDoSing their internal APIs or they’re not willing to opensource the part of the client that runs on App Engine instances behind SDK. I think there’s a tech talk coming up on Cloud Datastore that may shed some light. [1] http://www.cidrdb.org/cidr2011/Papers/CIDR11_Paper32.pdf http://www.cidrdb.org/cidr2011/Papers/CIDR11_Paper32.pdf
- dmishe 13y ago
  I mean I remember reading about this a couple of days ago. I guess "outside the cloud" is what I missed.
- to_jon 13y ago
  Yes, high availability with ACID and 'eventual consistency' on reads. I never understood why an eventually consistent datastore could qualify as ACID because in the real world if you need ACID, then eventual consistency is never good enough. It's good to see Google roll this service out separately from GAE but it's not as good as DynamoDB or data services from Parse and others.
  - deno 13y ago
    You only have eventual consistency for queries outside of entity‐groups. For ancestor queries you get strong consistency. https://developers.google.com/appengine/docs/python/datastore/structuring_for_strong_consistency https://developers.google.com/appengine/docs/python/datastor... Imagine a comment system. You’d fire one global query with eventual consistency guarantees for all comments in a thread and another one in parallel for only comments made by the signed in user. Merging the two gives you the best of both – great scalability and consistency where it matters. > it's not as good as DynamoDB DynamoDB’s transaction support is limited to incrementing counters and optimistic concurrency on a single item. Compared to Datastore, it looks like a toy.
latchkey 13y ago
My entire business is built on top of the datastore running on GAE. I've heard stories from googleers about loosing an entire datacenter and nobody on the outside noticed. Even with the limitations of developing on top of the datastore, there is some really amazing technology around it that sure makes it worth using. Exposing the datastore to GCE is a step in the right direction.

This is not new. But how good is it anyways?

by dmishe 13y ago

This is not new. But how good is it anyways?

deno 13y ago

It’s highly‐available NoSQL database (Megastore) with ACID transactions based on entity‐groups[1]. No other cloud provider has anything even remotely comparable. And it’s now available outside of the App Engine framework. I was hoping they’d make App Engine to actually run on GCE, but this is fine for now. I’m a little bit worried about this though: > You should be aware that Cloud Datastore has a serving component that runs on Google App Engine, so there will be instance hour costs. What I think is going on is they are either worried about about DDoSing their internal APIs or they’re not willing to opensource the part of the client that runs on App Engine instances behind SDK. I think there’s a tech talk coming up on Cloud Datastore that may shed some light. [1] http://www.cidrdb.org/cidr2011/Papers/CIDR11_Paper32.pdf http://www.cidrdb.org/cidr2011/Papers/CIDR11_Paper32.pdf

dmishe 13y ago
I mean I remember reading about this a couple of days ago. I guess "outside the cloud" is what I missed.
to_jon 13y ago
Yes, high availability with ACID and 'eventual consistency' on reads. I never understood why an eventually consistent datastore could qualify as ACID because in the real world if you need ACID, then eventual consistency is never good enough. It's good to see Google roll this service out separately from GAE but it's not as good as DynamoDB or data services from Parse and others.
- deno 13y ago
  You only have eventual consistency for queries outside of entity‐groups. For ancestor queries you get strong consistency. https://developers.google.com/appengine/docs/python/datastore/structuring_for_strong_consistency https://developers.google.com/appengine/docs/python/datastor... Imagine a comment system. You’d fire one global query with eventual consistency guarantees for all comments in a thread and another one in parallel for only comments made by the signed in user. Merging the two gives you the best of both – great scalability and consistency where it matters. > it's not as good as DynamoDB DynamoDB’s transaction support is limited to incrementing counters and optimistic concurrency on a single item. Compared to Datastore, it looks like a toy.

latchkey 13y ago

My entire business is built on top of the datastore running on GAE. I've heard stories from googleers about loosing an entire datacenter and nobody on the outside noticed. Even with the limitations of developing on top of the datastore, there is some really amazing technology around it that sure makes it worth using. Exposing the datastore to GCE is a step in the right direction.