3 ms·

Completely agree! I've seen SQL implementations of ledgers fail quite a few times. I'm using DynamoDB, but any no-SQL database will do the work (currently work

Completely agree! I've seen SQL implementations of ledgers fail quite a few times. I'm using DynamoDB, but any no-SQL database will do the work (currently working on Cassandra and MongoDB connectors to enable self-hosting for enterprise customers) On the implementation side, it's a mix of both: I'm using the conditional insert/optimistic locking of no-SQL and modeled the ledger to be a hash-chain at the application layer. The design is actually simple, and it scales indefinitely. Besides that, I'm using DynamoDB streams to build roll-ups of the transactions and make sure that all transactions are always balancing to zero in real-time.

jonahbenton 6y ago
Cool, got it. Are you actually seeing demand for on-prem hosting? Serious demand such that you would lose the sale from a customer you cannot live without? Completely different support model and neither Cassandra and especially Mongo are anywhere near Dynamo. I also have seen plenty of cases where balance to zero only at end of day is feasible, but real time is a good way of picking the right customers. My completely unsolicited advice- do the same with the DB, stay away from customers who say they need prem. But I'm just a schmo on the internet. Very best of luck to you!
- andriosr 6y ago
  A couple of companies asked for it, but I couldn't agree more with you. We are in the design phase of these implementations, as there is a lot more than code to consider for a self-hosted version. Thanks for the advice, it's great to get your view on it.
  - ethn 6y ago
    DynamoDB is the wrong technology for this. You can't prevent phantom reads. Transactional databases (precisely invented for this!) are the way to go. To quote DDIA: 'A service that allows users to spend money or points needs to check that a user doesn’t spend more than they have. You might implement this by inserting a ten‐ tative spending item into a user’s account, listing all the items in the account, and checking that the sum is positive [44]. With write skew, it could happen that two spending items are inserted concurrently that together cause the balance to go negative, but that neither transaction notices the other.'
    - andriosr 6y ago
      Thanks for the comment! It’s a common misunderstanding that no-sql dbs can’t be used for this, but any would work. Dynamo dB have consistent reads and conditional inserts, which is sufficient to build an optimistic locking system that ensures no double spent money.
      - ethn 6y ago
        No problem. You should look into this more. Conditional expressions in DDB do not provide aggregate expressions and can only determinate on single attributes belonging to the same table. Consistent reads aren’t enough. You would need to prevent phantom reads. DDB doesn’t support range locking since it’s a LSMT. You can contact the AWS solutions engineers or perform a quick online search: ‘However, neither DynamoDB nor the library support locking for range queries, thus phantom reads can potentially result in phantom reads.’ https://dbdb.io/db/dynamodb https://dbdb.io/db/dynamodb

jonahbenton 6y ago

Cool, got it. Are you actually seeing demand for on-prem hosting? Serious demand such that you would lose the sale from a customer you cannot live without? Completely different support model and neither Cassandra and especially Mongo are anywhere near Dynamo. I also have seen plenty of cases where balance to zero only at end of day is feasible, but real time is a good way of picking the right customers. My completely unsolicited advice- do the same with the DB, stay away from customers who say they need prem. But I'm just a schmo on the internet. Very best of luck to you!

andriosr 6y ago
A couple of companies asked for it, but I couldn't agree more with you. We are in the design phase of these implementations, as there is a lot more than code to consider for a self-hosted version. Thanks for the advice, it's great to get your view on it.
- ethn 6y ago
  DynamoDB is the wrong technology for this. You can't prevent phantom reads. Transactional databases (precisely invented for this!) are the way to go. To quote DDIA: 'A service that allows users to spend money or points needs to check that a user doesn’t spend more than they have. You might implement this by inserting a ten‐ tative spending item into a user’s account, listing all the items in the account, and checking that the sum is positive [44]. With write skew, it could happen that two spending items are inserted concurrently that together cause the balance to go negative, but that neither transaction notices the other.'
  - andriosr 6y ago
    Thanks for the comment! It’s a common misunderstanding that no-sql dbs can’t be used for this, but any would work. Dynamo dB have consistent reads and conditional inserts, which is sufficient to build an optimistic locking system that ensures no double spent money.
    - ethn 6y ago
      No problem. You should look into this more. Conditional expressions in DDB do not provide aggregate expressions and can only determinate on single attributes belonging to the same table. Consistent reads aren’t enough. You would need to prevent phantom reads. DDB doesn’t support range locking since it’s a LSMT. You can contact the AWS solutions engineers or perform a quick online search: ‘However, neither DynamoDB nor the library support locking for range queries, thus phantom reads can potentially result in phantom reads.’ https://dbdb.io/db/dynamodb https://dbdb.io/db/dynamodb