3 ms·
The transaction data was well understood, but did change over time. It was well normalised in a relational backend that was (at the time) a few hundred GB, but
by leejo 8y ago
The transaction data was well understood, but did change over time. It was well normalised in a relational backend that was (at the time) a few hundred GB, but it did not have any partitioning.
The problem we were trying to solve was that allowing, essentially, arbitrary queries against the main database had the potential to cause problems because we would have to ensure that those queries were optimal and all the relations were correct before we allowed those rules to be put into place - i.e. more dev and DBA work and potentially schema changes (addition of more indexes, etc).
It was far easier to have an ephemeral duplicate of the transaction data with a flat structure, a single key value document, as that mitigated the risks and removed the aforementioned problem.
The velocity system was essentially a firewall in front of the main database server - this was one of the main requirements, the velocity system must not have any affect on the load of the main transaction system. We could have added another db server, but that would have put us on the route to more servers, more complexity, etc, as we were self hosting at the time.
- jacques_chester 8y agoWhat I'm hearing is that you used MongoDB as a read cache, rather than as a database per se.