3 ms·

There are hybrid designs for separation of storage and compute that are aimed at mixed workloads[1]. They avoid writes to remote storage on transaction commit

There are hybrid designs for separation of storage and compute that are aimed at mixed workloads[1]. They avoid writes to remote storage on transaction commit (i.e., act like a shared nothing databases for commits, but still push data asynchronously to a shared remote disk that can be used for scale up/point in time restores/branching). [1] https://www.singlestore.com/blog/separating-storage-and-compute-for-transaction-and-analytics/ https://www.singlestore.com/blog/separating-storage-and-comp... (disclosure: SingleStoreDB CTO)

nikita 4y ago
Adam, you still synchronously replicate each log record to 2 places, right? Technically this should be roughly equivalent to writing into a consensus. And use write through cache for reads.
- AdamProut 4y ago
  Yep, for writes network bandwidth usage is independent of separation of storage of compute in some sense. Any database that provides high availability is writing over the network somewhere before it acks a transaction committed. It matters who is on the other end of that network write though. Take the typical Cloud DW (i.e., Snowflake) design of forcing writes to the blob storage (shared remote disks) before commit. That is a much higher latency write then what a high performance transaction log replication protocol will do to replicate a write to another host.
zX41ZdbW 4y ago
SingleStore is a kind of uninteresting example because it is not open-source. It makes potential customers feel non-confident. Why move large data volumes to SingleStore if the company can cease to exist in a year?
- AdamProut 4y ago
  Yep, SingleStoreDB (formerly MemSQL) is not open source (probably never will be), but it does have many paying customers who have had workloads in production for over a decade at this point. Also, isn't taking a bet on a very recently launched database as a service based around an open source database also pretty risky? Say for example the very recently launched Clickhouse Inc. service (which your a co-founder of?).
  - zX41ZdbW 4y ago
    Touché.
- password4321 4y ago
  I'm interested to see when SingleStore will support ARM. https://www.singlestore.com/forum/t/singlestore-for-arm64-m1-macs/2877/31 https://www.singlestore.com/forum/t/singlestore-for-arm64-m1...

nikita 4y ago

Adam, you still synchronously replicate each log record to 2 places, right? Technically this should be roughly equivalent to writing into a consensus. And use write through cache for reads.

AdamProut 4y ago
Yep, for writes network bandwidth usage is independent of separation of storage of compute in some sense. Any database that provides high availability is writing over the network somewhere before it acks a transaction committed. It matters who is on the other end of that network write though. Take the typical Cloud DW (i.e., Snowflake) design of forcing writes to the blob storage (shared remote disks) before commit. That is a much higher latency write then what a high performance transaction log replication protocol will do to replicate a write to another host.

zX41ZdbW 4y ago

SingleStore is a kind of uninteresting example because it is not open-source. It makes potential customers feel non-confident. Why move large data volumes to SingleStore if the company can cease to exist in a year?

AdamProut 4y ago
Yep, SingleStoreDB (formerly MemSQL) is not open source (probably never will be), but it does have many paying customers who have had workloads in production for over a decade at this point. Also, isn't taking a bet on a very recently launched database as a service based around an open source database also pretty risky? Say for example the very recently launched Clickhouse Inc. service (which your a co-founder of?).
- zX41ZdbW 4y ago
  Touché.
password4321 4y ago
I'm interested to see when SingleStore will support ARM. https://www.singlestore.com/forum/t/singlestore-for-arm64-m1-macs/2877/31 https://www.singlestore.com/forum/t/singlestore-for-arm64-m1...