4 ms·
Just like I commented on the original incident report post, I think systems like Redis are not suitable to work as a db for payment processing and transaction s
by Vitaly 13y ago
Just like I commented on the original incident report post, I think systems like Redis are not suitable to work as a db for payment processing and transaction storage. Reading through the report I can't imagine something like this happening with a payment system built around Postgres. Not unless you are doing something incredibly stupid. And stupid those guys are not.
They are obviously bright guys meaning well, and yet they've designed and implemented payment system with such a bad failure mode.
I do understand that they have a LOT of billing events, and have to update customer billable amounts for each of them. But instead of holding the customer balances in Redis and doing payment processing on top of that, my paranoia would most probably lead me to only store 'amount to charge' in Redis and update it as frequently as needed, and store customer balances and transactions in an RDBMS. And only change during actual charge event. This way, if Redis data were to be lost, I'd under-charge my customers and not over-double-tripple charge them. The failure mode becomes less disastrous.
- ra 13y agoIf you're running with AOF then redis is perfectly fine for storing eg: call logs where the transaction is naturally atomical to a single command. In my experience problems have occur because AOF isn't the default persistence setting (Snapshotting is the default in Ubuntu apt at least). So if Redis get's upgraded in an apt-get upgrade then the sysadmin needs to take care not to override the AOF configuration. This is not unlike postgres upgrading from 9.1 to 9.2, for example. Not catastrophic, but boy it'll make your heart pump! I think the best solution at the moment it to not use apt to manage Redis updates so that you have full control over the configuration.