5 ms·

What is the AWS equivalent to Cloud Spanner?

by claar 9y ago

deleted 9y ago
[deleted]
jamesblonde 9y ago
There is none. DynamoDB would be the closest - it's a HA DB for OLTP workloads (but without the transactions). AWS do not have the TrueTime API that Spanner builds on to provide globally consistent transactions. AWS's storage offering S3 is pretty limited in terms of its consistency model: https://medium.com/@jim_dowling/reflections-on-s3s-architectural-flaws-71f14c05a5fa https://medium.com/@jim_dowling/reflections-on-s3s-architect... In contrast, Google's filesystem (yes, not an object store, but a filesystem) is hardly known - Collosus. It provides a HA consistent metadata layer to a distributed filesystem. The closest open-source architecture is HopsFS (distributed metadata for HDFS) - disclosure I work on Hops.
- elvinyung 9y ago
  I would add to this by saying that AWS Aurora "looks" the most similar to Spanner by being a transparent scale-out relational database with strongly-consistent transactionality, but it doesn't seem to be designed for geo-replication.
  - nishantvyas 9y ago
    Agree. AWS aurora is closest to spanner and would work for 95% of use-cases... master-master, cross-colo or geo-replication is great problem for less than 2% of companies...
    - cosmie 9y ago
      Multi-master Aurora is actually in Preview right now[1]. Only for the MySQL variant of Aurora though, not the Postgres variant. [1] https://aws.amazon.com/about-aws/whats-new/2017/11/sign-up-for-the-preview-of-amazon-aurora-multi-master/ https://aws.amazon.com/about-aws/whats-new/2017/11/sign-up-f...
  - rpedela 9y ago
    AWS Aurora supports cross region read replicas. Is geo-replication something different?
  - deleted 9y ago
    [deleted]
- cube2222 9y ago
  Isn't DynamoDB more in the direction of Google Cloud Bigtable/Datastore though? If I'm right with this, then this is basically stating: "yeah, aws also has a high scale database offering" but it's not really comparable anyhow.
- mrep 9y ago
  Actually, AWS recently released a time sync service that anyone can use so you could possible build it: https://aws.amazon.com/blogs/aws/keeping-time-with-amazon-time-sync-service/ https://aws.amazon.com/blogs/aws/keeping-time-with-amazon-ti...
amq 9y ago
CockroachDB + AWS Time Sync.
- elvinyung 9y ago
  gcloud also has an atomic-clock-backed NTP service: https://developers.google.com/time/ https://developers.google.com/time/ Neither provide the guarantees of TrueTime.
  - lobo_tuerto 9y ago
    This might be interesting: https://www.cockroachlabs.com/blog/living-without-atomic-clocks/ https://www.cockroachlabs.com/blog/living-without-atomic-clo... Excerpt: "...The designers of Spanner call this ‘TrueTime’, and it provides a tight bound on clock offset between any two nodes in the system. TrueTime enables high levels of external consistency. As an open source database based on Spanner, our challenge was in providing similar guarantees of external consistency without atomic clocks."
wahnfrieden 9y ago
AWS tries to cover this strategically with Aurora (as well as Serverless Aurora) and DynamoDB. Serverless Aurora can autoscale up and down based on usage, and Aurora has powerful replication functionality, but this is a far cry from Spanner's multi-master horizontal scaling. DynamoDB can autoscale (crudely), and they just added multi-region multi-master. It's non-relational though and requires more capacity planning than Spanner.
- cosmie 9y ago
  I mentioned it in another comment, but along with Aurora Serverless they also announced Aurora Multi-master in Preview[1]. No clue how far it is from General Availability or if it allows even remotely close to the horizontal scaling of Spanner. But it’s at least being worked on. [1] https://aws.amazon.com/about-aws/whats-new/2017/11/sign-up-for-the-preview-of-amazon-aurora-multi-master/ https://aws.amazon.com/about-aws/whats-new/2017/11/sign-up-f...
  - ddorian43 9y ago
    No mention of partitioning/sharding.

What is the AWS equivalent to Cloud Spanner?

by claar 9y ago

What is the AWS equivalent to Cloud Spanner?

deleted 9y ago

[deleted]

jamesblonde 9y ago

There is none. DynamoDB would be the closest - it's a HA DB for OLTP workloads (but without the transactions). AWS do not have the TrueTime API that Spanner builds on to provide globally consistent transactions. AWS's storage offering S3 is pretty limited in terms of its consistency model: https://medium.com/@jim_dowling/reflections-on-s3s-architectural-flaws-71f14c05a5fa https://medium.com/@jim_dowling/reflections-on-s3s-architect... In contrast, Google's filesystem (yes, not an object store, but a filesystem) is hardly known - Collosus. It provides a HA consistent metadata layer to a distributed filesystem. The closest open-source architecture is HopsFS (distributed metadata for HDFS) - disclosure I work on Hops.

elvinyung 9y ago
I would add to this by saying that AWS Aurora "looks" the most similar to Spanner by being a transparent scale-out relational database with strongly-consistent transactionality, but it doesn't seem to be designed for geo-replication.
- nishantvyas 9y ago
  Agree. AWS aurora is closest to spanner and would work for 95% of use-cases... master-master, cross-colo or geo-replication is great problem for less than 2% of companies...
  - cosmie 9y ago
    Multi-master Aurora is actually in Preview right now[1]. Only for the MySQL variant of Aurora though, not the Postgres variant. [1] https://aws.amazon.com/about-aws/whats-new/2017/11/sign-up-for-the-preview-of-amazon-aurora-multi-master/ https://aws.amazon.com/about-aws/whats-new/2017/11/sign-up-f...
- rpedela 9y ago
  AWS Aurora supports cross region read replicas. Is geo-replication something different?
- deleted 9y ago
  [deleted]
cube2222 9y ago
Isn't DynamoDB more in the direction of Google Cloud Bigtable/Datastore though? If I'm right with this, then this is basically stating: "yeah, aws also has a high scale database offering" but it's not really comparable anyhow.
mrep 9y ago
Actually, AWS recently released a time sync service that anyone can use so you could possible build it: https://aws.amazon.com/blogs/aws/keeping-time-with-amazon-time-sync-service/ https://aws.amazon.com/blogs/aws/keeping-time-with-amazon-ti...

amq 9y ago

CockroachDB + AWS Time Sync.

elvinyung 9y ago
gcloud also has an atomic-clock-backed NTP service: https://developers.google.com/time/ https://developers.google.com/time/ Neither provide the guarantees of TrueTime.
- lobo_tuerto 9y ago
  This might be interesting: https://www.cockroachlabs.com/blog/living-without-atomic-clocks/ https://www.cockroachlabs.com/blog/living-without-atomic-clo... Excerpt: "...The designers of Spanner call this ‘TrueTime’, and it provides a tight bound on clock offset between any two nodes in the system. TrueTime enables high levels of external consistency. As an open source database based on Spanner, our challenge was in providing similar guarantees of external consistency without atomic clocks."

wahnfrieden 9y ago

AWS tries to cover this strategically with Aurora (as well as Serverless Aurora) and DynamoDB. Serverless Aurora can autoscale up and down based on usage, and Aurora has powerful replication functionality, but this is a far cry from Spanner's multi-master horizontal scaling. DynamoDB can autoscale (crudely), and they just added multi-region multi-master. It's non-relational though and requires more capacity planning than Spanner.

cosmie 9y ago
I mentioned it in another comment, but along with Aurora Serverless they also announced Aurora Multi-master in Preview[1]. No clue how far it is from General Availability or if it allows even remotely close to the horizontal scaling of Spanner. But it’s at least being worked on. [1] https://aws.amazon.com/about-aws/whats-new/2017/11/sign-up-for-the-preview-of-amazon-aurora-multi-master/ https://aws.amazon.com/about-aws/whats-new/2017/11/sign-up-f...
- ddorian43 9y ago
  No mention of partitioning/sharding.