3 ms·
> You POST to get a transaction ID, the thread starts a DB transaction [3]. I don't disagree with the logic here, but this is going to get old fast in a modern
by atombender 8y ago
> You POST to get a transaction ID, the thread starts a DB transaction [3].
I don't disagree with the logic here, but this is going to get old fast in a modern load-balanced, multi-region/zone topology, since it means pegging the client's state interaction to a particular backend process.
Sticky sessions are the classical answer to this, but they are very much something you want to avoid these days, because they require that every proxy along the path to your app is aware of the stickiness. If you end up deploying your app to, say, Google Kubernetes Engine, you don't even get a load balancer that supports stickiness.
The only way I would implement this would be with a level of indirection. When a request comes in, start the thread, open a listener socket (with, say, gRPC), then store the socket address in some reliable shared data store (Etcd, ZooKeeper, Postgres, etc.). Any requests from the same client can then be routed to that socket internally. If you're writing this in Erlang, you can replace "socket" with "process" and be done in half the time.
That might work okay for latency-sensitive OLTP stuff, but a good pattern I've used before is to model transactions as explicit state. The client uses CRUD actions to build up a "batch" of data (e.g. in a CMS-type app, create a folder, then create a document in the folder), which is stored separate from live data and completely invisible. Once the client is ready to commit, it sends a commit request, and the backend takes the stored state and tries to apply it to the main database.
There are some benefits to this asynchronous, batch-oriented view of transactions: For example, you get audit logging and debuggability "for free", and the backend can be smart about conflict merging and retrying without involving the client. It also allows the client to post incomplete data; it can create a document before the folder exists, even though the document at this point is not yet valid according to business rules, and later in the same transaction it can "patch" the document with the right parent folder. This maps well to the kinds of ETL that apps tends to want to do, where it's nice to write a thin dumb import process that doesn't have to keep lots of local state.
- jerf 8y ago"Scaling code is great, but don't pay the price for 10^8 when you're never going to pass 10^3." I am responsible for applications both where holding open a DB transaction as I describe would be suicidal levels of stupid as I expect hundreds of events per second as the applications hits maturity, and perfectly acceptable because the load of the application is measured in single-digit write sessions per week. (They're high-value sessions on a per-session basis, having massive effects, but there aren't a lot of them.) It's obviously a fail to engineer the 10^8 system to 10^-2 standards; it's less obviously a fail, but still a fail, to make the opposite error.
- atombender 8y agoI was talking about the operational aspect, not scalability. Read my first paragraph again. :-) I don't see any huge issues with the design in terms of scalability.