4 ms·

>in-memory architectures for transactions Can someone explain this to me ? In terms of acid principles wouldn't you want your transactions to be durable. >Du

>in-memory architectures for transactions Can someone explain this to me ? In terms of acid principles wouldn't you want your transactions to be durable. >Durability means that once a transaction has been committed, it will remain so, even in the event of power loss, crashes, or errors. In a relational database, for instance, once a group of SQL statements execute, the results need to be stored permanently (even if the database crashes immediately thereafter). To defend against power loss, transactions (or their effects) must be recorded in a non-volatile memory.

ericfrenkiel 13y ago
I'm a cofounder at MemSQL. The explanation around durability and in-memory databases is more nuanced than what is assumed with disk-based systems. First, it's worth mentioning that you use in-memory databases when you have performance requirements that far exceed physical capabilities of hard disks or even SSDs. The workload will have a high degree of contention, high concurrency, etc. Second, you're probably working with non-human-generated data. If you're not a bank that needs to guarantee a debit and credit went through (and that's slow human-generated data anyway), then you're looking at in-memory technology because you can guarantee every read will hit memory. For writes, you basically have to use more machines to guarantee durability. Any in-memory database worth its salt will write a transaction log to disk, so in the event of a power failure, it will read back from disk into memory. You can increase the probability of 0 data loss, even in high contention and concurrent workloads, if you run the dataset across a set of machines, which both multiplies the number of disks capable of writing a sequential log as well as storing an extra copy in memory for high availability.
mtdewcmu 13y ago
Transaction is being used in different senses there. The first sense refers to an OLTP workload. The second sense refers to ACID transactions. The in-memory architecture would relate to details of how the database manages memory internally. There's no question that updates need to be written out to disk as soon as possible for them to be durable.

ericfrenkiel 13y ago

I'm a cofounder at MemSQL. The explanation around durability and in-memory databases is more nuanced than what is assumed with disk-based systems. First, it's worth mentioning that you use in-memory databases when you have performance requirements that far exceed physical capabilities of hard disks or even SSDs. The workload will have a high degree of contention, high concurrency, etc. Second, you're probably working with non-human-generated data. If you're not a bank that needs to guarantee a debit and credit went through (and that's slow human-generated data anyway), then you're looking at in-memory technology because you can guarantee every read will hit memory. For writes, you basically have to use more machines to guarantee durability. Any in-memory database worth its salt will write a transaction log to disk, so in the event of a power failure, it will read back from disk into memory. You can increase the probability of 0 data loss, even in high contention and concurrent workloads, if you run the dataset across a set of machines, which both multiplies the number of disks capable of writing a sequential log as well as storing an extra copy in memory for high availability.

mtdewcmu 13y ago

Transaction is being used in different senses there. The first sense refers to an OLTP workload. The second sense refers to ACID transactions. The in-memory architecture would relate to details of how the database manages memory internally. There's no question that updates need to be written out to disk as soon as possible for them to be durable.