3 ms·

Feel free to maintain your skepticism until public benchmarks are available, but Amazon has specifically advertised multi-master will horizontally scale out wri

Feel free to maintain your skepticism until public benchmarks are available, but Amazon has specifically advertised multi-master will horizontally scale out writes for increased write performance [1]: > Workloads that require even higher write throughput will benefit from horizontally scaling out their writes with additional master instances. [1]: https://aws.amazon.com/about-aws/whats-new/2017/11/sign-up-for-the-preview-of-amazon-aurora-multi-master/ https://aws.amazon.com/about-aws/whats-new/2017/11/sign-up-f...

ddorian43 9y ago
But no sharding. All writes will go to all servers.
- wgjordan 9y ago
  You're probably thinking of 'group replication' in standard MySQL. Aurora's horizontal scaling for reads, writes and storage capacity is quite different since compute and storage is decoupled, and the concepts of 'sharding' or sending writes to 'servers' don't really translate directly. In Aurora [1], data is partitioned into 10GB chunks ('protection groups'), with copies stored in 6 'storage nodes' spread across 3 Availability Zones. When the 'server' (database node) handles a write, the data is sent in parallel to all 6 storage nodes, and the write is accepted as soon as a quorum of storage nodes (4) writes the data to its redo log and responds. The storage nodes also continuously back up the data to S3 for extra durability. These 6 parallel storage-node operations are performed for every write, even in a single-master Aurora setup. Multi-master simply scales out the number of 'database nodes' (compute) for handling queries, while keeping the number of storage nodes (and the total write-amplification) constant. In the rare case when conflicting write-transactions occur that can't be resolved locally at either the database or storage layer (e.g., when data changed at both multiple database nodes AND multiple storage nodes), a hierarchical conflict-resolution mechanism is invoked to decide which transaction succeeds and which gets rolled back [2]. This reportedly provides 'near-linear performance scaling when there is no or low levels of conflicts', even when dealing with multiple masters spread across multiple regions [3]. [1]: https://aws.amazon.com/blogs/database/introducing-the-aurora-storage-engine/ https://aws.amazon.com/blogs/database/introducing-the-aurora... [2]: https://www.slideshare.net/AmazonWebServices/deep-dive-on-the-amazon-aurora-mysqlcompatible-edition-dat301-reinvent-2017/35 https://www.slideshare.net/AmazonWebServices/deep-dive-on-th... [3]: https://www.slideshare.net/AmazonWebServices/deep-dive-on-the-amazon-aurora-mysqlcompatible-edition-dat301-reinvent-2017/38 https://www.slideshare.net/AmazonWebServices/deep-dive-on-th...
  - ddorian43 9y ago
    When (and ONLY WHEN) they make the chunks only be available on SOME servers, then it will be. See tikv/tidb. Tikv is storage but data is PARTITIONED while tidb is computation (no persistence). Currently, difference is that, functions only happen on "compute" (so data transfer) while on tikv, functions happen on "storage" layer, and "compute" layer is for aggregation. When they do that it will be called horizontal scaling.

ddorian43 9y ago

But no sharding. All writes will go to all servers.

wgjordan 9y ago
You're probably thinking of 'group replication' in standard MySQL. Aurora's horizontal scaling for reads, writes and storage capacity is quite different since compute and storage is decoupled, and the concepts of 'sharding' or sending writes to 'servers' don't really translate directly. In Aurora [1], data is partitioned into 10GB chunks ('protection groups'), with copies stored in 6 'storage nodes' spread across 3 Availability Zones. When the 'server' (database node) handles a write, the data is sent in parallel to all 6 storage nodes, and the write is accepted as soon as a quorum of storage nodes (4) writes the data to its redo log and responds. The storage nodes also continuously back up the data to S3 for extra durability. These 6 parallel storage-node operations are performed for every write, even in a single-master Aurora setup. Multi-master simply scales out the number of 'database nodes' (compute) for handling queries, while keeping the number of storage nodes (and the total write-amplification) constant. In the rare case when conflicting write-transactions occur that can't be resolved locally at either the database or storage layer (e.g., when data changed at both multiple database nodes AND multiple storage nodes), a hierarchical conflict-resolution mechanism is invoked to decide which transaction succeeds and which gets rolled back [2]. This reportedly provides 'near-linear performance scaling when there is no or low levels of conflicts', even when dealing with multiple masters spread across multiple regions [3]. [1]: https://aws.amazon.com/blogs/database/introducing-the-aurora-storage-engine/ https://aws.amazon.com/blogs/database/introducing-the-aurora... [2]: https://www.slideshare.net/AmazonWebServices/deep-dive-on-the-amazon-aurora-mysqlcompatible-edition-dat301-reinvent-2017/35 https://www.slideshare.net/AmazonWebServices/deep-dive-on-th... [3]: https://www.slideshare.net/AmazonWebServices/deep-dive-on-the-amazon-aurora-mysqlcompatible-edition-dat301-reinvent-2017/38 https://www.slideshare.net/AmazonWebServices/deep-dive-on-th...
- ddorian43 9y ago
  When (and ONLY WHEN) they make the chunks only be available on SOME servers, then it will be. See tikv/tidb. Tikv is storage but data is PARTITIONED while tidb is computation (no persistence). Currently, difference is that, functions only happen on "compute" (so data transfer) while on tikv, functions happen on "storage" layer, and "compute" layer is for aggregation. When they do that it will be called horizontal scaling.