4 ms·

Before Aurora popped up and saved the day, we were looking at sharding our MySQL database. The levels of added complexity were keeping me up at night. I am sure

by donatj 5y ago

Before Aurora popped up and saved the day, we were looking at sharding our MySQL database. The levels of added complexity were keeping me up at night. I am sure we’ll reach a point where Aurora doesn’t solve it anymore but until then.

manigandham 5y ago
Using Aurora is analogous to vertical scaling (as in scaling up with bigger and bigger hardware). Distributed relational databases have made tremendous progress in recent years and you have many compelling options now if you need it. For MySQL there's SingleStore (memsql), Vitesse/Planetscale, and TiDB.
- matwood 5y ago
  Aurora also horizontally scales with readers.
  - zinodaur 5y ago
    Replication lag can make that pretty tricky - you never know if the client doing the reading has seen a state further in the future than the replica
    - daneel_w 5y ago
      That's not a concern unless your application is broken. An application should stick to one DB instance for one and the same isolated task, not open multiple parallel connections to interact with.
      - evanelias 5y ago
        You're trivializing read-after-write consistency. It's actually quite a challenging problem for most applications, especially for anything web-based. Let's say you add a comment to a piece of content and then refresh the page. With a naively-implemented CRUD app, your comment disappears if your request's read query was routed to a lagging replica. This gets even trickier when multiple datacenters in different geographic regions are involved.
  - manigandham 5y ago
    That's not sharding, it's replication. It's not splitting up your data but making a copy, and still requires an instance big enough (scaled up) to store it all. Just about every other major database also supports replicas.
- ksec 5y ago
  I am still waiting for Planetscale's big announcement.
  - manigandham 5y ago
    What do you mean?
    - ksec 5y ago
      https://twitter.com/planetscaledata/status/1504117361418420238 https://twitter.com/planetscaledata/status/15041173614184202...
- daneel_w 5y ago
  No, Aurora scales horizontally for reads. It does for writes as well, though not nearly as much due to replication round trip.
  - manigandham 5y ago
    No to what exactly? That's replication, not partitioning, and involves scaling up every instance to be big enough to hold the entire dataset. You must scale up first before you can scale out.

by donatj 5y ago

manigandham 5y ago

Using Aurora is analogous to vertical scaling (as in scaling up with bigger and bigger hardware). Distributed relational databases have made tremendous progress in recent years and you have many compelling options now if you need it. For MySQL there's SingleStore (memsql), Vitesse/Planetscale, and TiDB.

matwood 5y ago
Aurora also horizontally scales with readers.
- zinodaur 5y ago
  Replication lag can make that pretty tricky - you never know if the client doing the reading has seen a state further in the future than the replica
  - daneel_w 5y ago
    That's not a concern unless your application is broken. An application should stick to one DB instance for one and the same isolated task, not open multiple parallel connections to interact with.
    - evanelias 5y ago
      You're trivializing read-after-write consistency. It's actually quite a challenging problem for most applications, especially for anything web-based. Let's say you add a comment to a piece of content and then refresh the page. With a naively-implemented CRUD app, your comment disappears if your request's read query was routed to a lagging replica. This gets even trickier when multiple datacenters in different geographic regions are involved.
- manigandham 5y ago
  That's not sharding, it's replication. It's not splitting up your data but making a copy, and still requires an instance big enough (scaled up) to store it all. Just about every other major database also supports replicas.
ksec 5y ago
I am still waiting for Planetscale's big announcement.
- manigandham 5y ago
  What do you mean?
  - ksec 5y ago
    https://twitter.com/planetscaledata/status/1504117361418420238 https://twitter.com/planetscaledata/status/15041173614184202...
daneel_w 5y ago
No, Aurora scales horizontally for reads. It does for writes as well, though not nearly as much due to replication round trip.
- manigandham 5y ago
  No to what exactly? That's replication, not partitioning, and involves scaling up every instance to be big enough to hold the entire dataset. You must scale up first before you can scale out.