3 ms·
Author here. Didn’t expect this to be on the front page of HN. I have a flight to catch but I can answer questions.
by kccqzy 9y ago
Author here. Didn’t expect this to be on the front page of HN. I have a flight to catch but I can answer questions.
- ubercow 9y agoHey! I'm mostly curious about the details behind the flat-file persistence. Is the storage on the same physical machine as the API and Business Logic? If not how is communication done? How do you handle horizontal scalability for either perf or HA reasons?
- kccqzy 9y agoHi! That’s actually a good question, an aspect that I basically avoided in the article. Truth be told, we aren’t happy with the way things are, but indeed the storage is on the same physical machine (in fact a shared docker volume) as everything else. We have a read-only replica that we use sometimes to execute slow queries. However, the vision is that eventually we would use Raft or other kinds of consensus on top of the current layer of storage. Event sourcing supposedly should make this kind of replication easy because data is always append-only. Unfortunately the reality of a startup is that we often have to do things quickly and this aspect has been neglected for pretty long.
- vemv 9y agoDoesn't Haskell STM limit you to a certain non-distributed architecture? At least in the Clojure world, STM is not used that much (despite praise) since it's not a good fit for horizontally scalable web servers.
- kccqzy 9y agoYes. In general it is a nightmare to attempt to coordinate STM with any sort of external action because STM inherently needs to have the ability to do computations multiple times and have “wasted” work when sometimes a transaction cannot commit immediately. Coupling STM with IO can only be done very carefully, making sure for example that the IO is abortable and idempotent. In our architecture (which is of course by no means perfect!) we largely use a single machine for the majority of our interactions so right now we aren’t facing this issue per se.