5 ms·
i wish it had an optional server for more concurrent and networked transactions in the cloud
by Kalanos 4y ago
i wish it had an optional server for more concurrent and networked transactions in the cloud
- jjtheblunt 4y agoyou could make one pretty easily, no?
- axelthegerman 4y agoI'd like to see that. I also think the single write situation is not great for web applications, but I don't see an easy way around it without sacrificing things like consistency
- dinosaurdynasty 4y agoIf you can treat sqlite transactions like redis transactions (send the entire transaction up front) it can work.
- bob1029 4y agoSee: LMAX Disruptor and friends. The magic spell that serializes many threads of events into one without relying on hard contention. You can even skip the hot busy waiting if you aren't trying to arbitrage the stock market. The way I would do it is a MPSC setup wherein the single consumer holds an exclusive connection to the SQLite database and writes out transactions in terms of the batch size. Basically: BEGIN -> iterate & process event batch -> END. This is very very fast due to how the CPU works at hardware level. It's also a good place to insert stuff like [a]synchronous replication logic. Completion is handled with busy/yield waiting on a status flag attached to the original event instance. You'd typically do a thing where all flags are acknowledged at the batch grain (i.e. after you committed the txn). This has some overhead, but the throughput & latency figures are really hard to argue with. It also has very compelling characteristics at the extremes in terms of system load. The harder you push, the faster it goes.
- bityard 4y agoYou may be interested in rqlite: https://github.com/rqlite/rqlite https://github.com/rqlite/rqlite