3 ms·
It uses DynamoDB and transactions. Each double-entry appears to be 4 items, 1 for each balance update and 1 for each transaction. Each “journal” will use 2x4 W
by kondro 6y ago
It uses DynamoDB and transactions. Each double-entry appears to be 4 items, 1 for each balance update and 1 for each transaction.
Each “journal” will use 2x4 WCU for the write to the table and 4 WCU for the global index updates (a total of 12 WCU for each journal).
I do have a few issues with the structure though. They seem to use LSIs to add a timestamp (instead of just using KSUID or similar for the ID), which ends up limiting each partition to a maximum of 10GB (a limit that doesn’t exist if you don’t use LSIs).
10GB might seem like a lot per account (which seems like the PK the partition is based off), but even at a conservative 200 bytes per transaction, that limits a single account to 50 million transactions forever.
Coupled with the fact this only seems to support literal double-entry accounting (not 2+ balanced transactions per journal item), you could quickly hit that limit if you are processing multi-item invoices for instance where items end up being allocated to different sales accounts.
Although if you were processing multi-item invoices you’d also have to work out how to make this transactionally-safe yourself, as this only guarantees ACID-compliance of a single DR/CR transaction.
Unfortunately, this wouldn’t necessarily be as easy as just allowing extra entries to a single DynamoDB transaction as those are limited to a max of 25 (and as covered, each entry requires 4 item updates).
DynamoDB isn’t actually a bad idea if you need an ACID-compliant database with read-after-write consistency. But I think the model needs to be rethought a bit to deal with some of the other design considerations DynamoDB forces on you.
- andriosr 6y agoWow, that was a great review, thanks! I agree with most points. There are definitely improvements to be made. This is the first version, I wanted to get it out and start getting feedback sooner. This is my take on each point: For some reason Dynamodb was not sorting the partition with KSUIDs string, so I added the LSI. This is probably something I did wrong and is just matter of fixing the query or the insert and move back to using the PK. Adding more than one balanced transaction per request is on the roadmap, but this will be limited to the DynamoDB transactions API limit of 25 items as you mentioned (25 / 2*2 WCU per tx = 6 transactions per request). Each entry only uses 4 items in the worst case (seeding a new currency in an account). In most cases (existing balances/currencies), there will be only 2 new items. I think even more improvements can be made. And the goal of open-sourcing is to get more heads thinking about how we can do it :)