5 ms·
Multiple datomic transactors can't talk to the same database, even if they share the same storage.
by Scarbutt 7y ago
Multiple datomic transactors can't talk to the same database, even if they share the same storage.
- jonahbenton 7y agoSure they can. Just give them different database names.
- Scarbutt 7y agoA different name is literally a different database.
- jonahbenton 7y agoA single postgres daemon- a unit of scaling- can support many datomic transactors, each talking to a named "database" hosted by that one daemon. You don't have to scale your postgres daemons with your microservices- each of which have their own transactor- which would be painful and out of the ordinary for an ops team. Scaling datomic on top of postgres is no different from scaling any other microservice. That all said, the architectural point that did sound painful in the presentation- and which is a common pain point in microservice architectures, not unique to what NuBank is doing with Datomic- was having to maintain an ETL for analytics purposes, to pull together all of the distinct microservice-specific Datomic-hosted data sets into a single uniformly SQL-queryable data set. The details of that implementation, and whether it made use of the new SQL interface supported by Datomic, were not discussed. But it smelled brittle and fragile.
- Scarbutt 7y agoA single postgres daemon- a unit of scaling- can support many datomic transactors yes ,each talking to a named "database" hosted by that one daemon. No. That would imply distributed writes and break the single write serializability property of datomic. Think about it, transactors don't sync with each other(only one for HA but that's orthogonal). To put it simply, multiple transactors can share the same storage but only one can write to a single database at a time.
- dustingetz 7y agodatomic peers can interleave N databases at query time, which is what you want right?
- Scarbutt 7y agoTo a single transactor only.
- dustingetz 7y agoI don't think that's right
- Scarbutt 7y agoAre you implying that a single peer can be configured to use multiple transactors at the same time?
- dustingetz 7y agoyes i believe they simply compete for object cache
- Scarbutt 7y agoOk, I never tried but I don't see any reason why it wouldn't work but also don't know how the peer library handles multiple connection objects. Anyway this is a different issue from having multiple transactors configured to write to the same logical database.
- dustingetz 7y agoit is interesting to consider because it enables a constellation architecture, where writes and reads are sharded independently. you could imagine a distributed computation like a social network where each person owns their own transactor but you can still query across the parts of the social graph you care about.