3 ms·
I think the point is that treating your database as an arms-length, RPC component that's independent from your "application" isn't necessarily the only pattern
by NovemberWhiskey 3y ago
I think the point is that treating your database as an arms-length, RPC component that's independent from your "application" isn't necessarily the only pattern.
- foobiekr 3y agoStrong agree. there are vast, massive cost savings and performance advantages to be had if the model is that a shard of the dataset is in memory and the data persistence problem is the part that's made external. The only reason we are where we are today is that doing that well is hard.
- robertlagrant 3y agoIs this not the case already? Database drivers (or just your application code) are allowed to cache results if they like. The problem is cache invalidation.
- foobiekr 3y agoCaches aren't the same. In the shard-in-memory case, the shard is the thing serving the queries, meaning it's not a cache it _is_ the live data.
- robertlagrant 3y agoUnderstood. For a single-node or read-only system it sounds fine, but then there are a variety of ways to solve that (e.g. a preloaded in-memory sqlite).
- bombolo 3y agoIf it's in memory you must live with the fact that it might be gone at any moment though.
- foobiekr 3y agoHence "the data persistence part."
- robertlagrant 3y agoThis sounds like a write-back / write-through cache with extra terminology. What's the difference?
- foobiekr 3y agoThe difference is the 10,000x slower performance if you move the actual DB transaction leader for the shard to over-the-wire access. Anything can be anywhere if we ignore latency and throughput.
- robertlagrant 3y agoI'm not saying that. I'm asking if this is just new terminology for a previous technology idea, or whether it's a new concept.