4 ms·

Replying to the deeper nested comment. 1) I totally concede that you could use Dynamo-style consistency to be the system of record for financial exchanges. I j

by jhugg 16y ago

Replying to the deeper nested comment. 1) I totally concede that you could use Dynamo-style consistency to be the system of record for financial exchanges. I just don't think it's a good idea. I'm not sure anyone building these systems does either. Though if you're doing e-commerce with a payment processing gateway like PayPal, you might be in a different situation. You're no longer anything like a bank at that point (PayPal is). 2) I don't know what Amazon does now, but last time I talked with them (2009) they were huge users of legacy RDBMSs for actual order processing. I know they're not thrilled with this arrangement, but I doubt they think that they think Dynamo is the answer. Perhaps cores of ACID-ness with Dynamo-esque replication between them... 3) You're right that banks use logs for a tremendous amount of things. And a VISA charge looks nothing like a simple debit and credit transaction. However, if you think that banks don't have debit and credit transactions in their applications, you haven't worked on their applications.

rbranson 16y ago
I am starting to believe that we both essentially believe mostly the same points, but we're using different nomenclature. You stated previously that most of these real-world, high-scalability, high-availability systems use ACID databases as a backing store, something I echo'd in the context of Dynamo's use of BerkeleyDB (and MySQL/InnoDB) as an underlying data store. I agree with this. You state that a Dynamo-style key-value store wouldn't be that great for a financial institution's accounting system. I agree with this. Dynamo is terrible for "log" data. This doesn't change the idea that these system as a whole aren't ACID, and do not require ACID semantics. Financial systems are often based on networks (in the conceptual sense) with varying degrees of trust, and are batch-reconciled logs of transactions. I'm talking about stock and commodity markets, electronic transfer systems, and traditional banking. They may be built from ACID building blocks at the low level, but that does not change the fact that systems that manage my checking account, credit cards, and stock market transactions are not functionally ACID or that they even need to be ACID. Dynamo itself is built on ACID databases, but it is not ACID itself. The balance of an account can be calculated by adding all credits and debits, which is what makes a financial account radically different than say, a Facebook profile. Each transaction is essentially immutable. Even if the transaction must be reversed, it's reversed as an additional credit (or a unique reversal transaction), not a deletion of the original transaction. While transactions against the account are almost always immediately available, they aren't ALWAYS, and this is alright, because the bank's customers understand this. Banks still reconcile accounts on a nightly basis, using batch processes. This is when accounts are officially settled and consistency is applied, which is why it's eventual. However, financial accounts are somewhat unique because their date-sorted, log-structured nature makes them quite suited to eventual consistency.

by jhugg 16y ago

rbranson 16y ago

I am starting to believe that we both essentially believe mostly the same points, but we're using different nomenclature. You stated previously that most of these real-world, high-scalability, high-availability systems use ACID databases as a backing store, something I echo'd in the context of Dynamo's use of BerkeleyDB (and MySQL/InnoDB) as an underlying data store. I agree with this. You state that a Dynamo-style key-value store wouldn't be that great for a financial institution's accounting system. I agree with this. Dynamo is terrible for "log" data. This doesn't change the idea that these system as a whole aren't ACID, and do not require ACID semantics. Financial systems are often based on networks (in the conceptual sense) with varying degrees of trust, and are batch-reconciled logs of transactions. I'm talking about stock and commodity markets, electronic transfer systems, and traditional banking. They may be built from ACID building blocks at the low level, but that does not change the fact that systems that manage my checking account, credit cards, and stock market transactions are not functionally ACID or that they even need to be ACID. Dynamo itself is built on ACID databases, but it is not ACID itself. The balance of an account can be calculated by adding all credits and debits, which is what makes a financial account radically different than say, a Facebook profile. Each transaction is essentially immutable. Even if the transaction must be reversed, it's reversed as an additional credit (or a unique reversal transaction), not a deletion of the original transaction. While transactions against the account are almost always immediately available, they aren't ALWAYS, and this is alright, because the bank's customers understand this. Banks still reconcile accounts on a nightly basis, using batch processes. This is when accounts are officially settled and consistency is applied, which is why it's eventual. However, financial accounts are somewhat unique because their date-sorted, log-structured nature makes them quite suited to eventual consistency.