freels

10 ms·

31.
by freels 8y ago
Unless you have specifics, I’m not sure this comment stands up to reality. Modern networks are pretty reliable in my experience. The state of the art in consensus reduces learning to one round trip. Calvin further eliminates all but one glo
32.
by freels 8y ago
The latest released version, 2.6.0, has most of the fixes as outlined in the post & report. Minor/ergonomic fixes are forthcoming. If you’re running on cloud, we do the upgrade work for you.
33.
by freels 8y ago
The constraint that all observers see T1 before T2 because T1 is acknowledged before T2 begins is the “in real time” component of a strict serializability guarantee. Mere serializability does not guarantee this, as unintuitive as that sound
34.
by freels 8y ago
It is 3 transactions. Alice deposits money into her savings acount, then subsequently withdraws money from her checking account. In the meantime, some concurrent process transactionally reads the balances of both accounts. Since there is no
35.
by freels 8y ago
Usually 2PC is used to run transactions across multiple shard which each have a subset of the total dataset. Though it doesn't provide you with any extra reliability. There's a tradeoff in sync vs async replication... a node which
36.
by freels 8y ago
Yes, it potentially is, but it entirely depends on the implementation details of the system. It's certainly allowed behavior by the definition of serializability.
37.
by freels 8y ago
Yes, I believe so. Since in Postgres, snapshot isolation (on top of which its serializable implementation is built) provides a stable view of the system at a point in time, and also doesn't allow stale reads, transactions _can be_ stri
38.
by freels 8y ago
There is some overhead, but it is minimal. For example in Fauna only the modified timestamps of read data are re-checked in transaction processing (which can be stored separately from the data itself), rather than entire records.
39.
by freels 8y ago
The big difference is that it's a much easier problem to solve in the context of a log (see Paxos, Raft, ZAB, et al), where the consensus membership is more or less fixed, as opposed to having to manage it among an adhoc membership set
40.
by freels 8y ago
FaunaDB already does this. It provides distributed transactions based on Calvin, which does not rely on classic 2PC. See https://fauna.com/blog/acid-transactions-in-a-globally-distr... and http://cs.yale.edu
41.
by freels 8y ago
Yes, I meant regions as well. Speed-of-light bounded latency certainly means that serializable ACID transactions are not workable in some situations, but I disagree with OP that this proves them not viable period, at least at planet-scale a
42.
by freels 8y ago
TiDB uses the Percolator protocol
43.
by freels 8y ago
I don't think it's correct to pit ACID against AP systems this way. Consistency is orthogonal. ACID is model for reasoning about transaction guarantees in terms of allowed anomalies at each isolation level. Serializability in ACID
44.
by freels 8y ago
You're welcome! I've enjoyed following TiKV/TiDB's development as well. It is an interesting time for databases. :-)
45.
by freels 8y ago
Support for dependent reads is a subject that the Calvin paper only touches on briefly, describing one strategy of using "reconnaissance reads" to determine the key set. In FaunaDB, this is formalized as an optimistic concurrency
46.
by freels 8y ago
Another point that often gets lost when talking about ACID guarantees is the maximum scope of a transaction that a system supports. There are many systems which support ACID guarantees for a single record or shard, but fewer that can suppor
47.
by freels 8y ago
That's a great resource. We've tried to be consistent in using these terms. I think a lot of the confusion and lack of standardization over terminology came from the fact that database and distributed systems research had less ove
48.
by freels 8y ago
I believe what we spanner calls 'external consistency' we call strict serializability. Read-write transactions in FaunaDB are strictly serializable, whereas reads default to serializable. This allows us to serve reads independentl
49.
by freels 8y ago
We run FaunaDB as a service, or you can license it for on-prem use. Drivers and tooling are all open however, and we plan to release the source for correctness testing as well. Our property tests are written in Scala as a Monad for controll
50.
by freels 8y ago
You make a valid point. We fixed the link.
51.
by freels 8y ago
Here is a direct link to the report: https://www2.fauna.com/l/517431/2018-06-25/6c9gwb/517431/114...
52.
by freels 8y ago
That’s right. Along with the load and fault generators, it includes some other useful tools such as a linearizability checker, as well as quite a few existing tests for other systems. From this we built out tests for Fauna’s stronger ACID g
53.
by freels 9y ago
Admittedly, the specific mechanics of how the log interacts with data partitions was glossed over a bit in that post. However, to answer your question, the input log is partitioned: Transactions for each epoch are striped across the log par
54.
by freels 9y ago
In FaunaDB all transaction resolution is deterministic. Once global log order is determined, partitions will wait for the transaction inputs at the correct version and will each independently come up with an identical result.
55.
by freels 9y ago
It is true that Spanner and FaunaDB partition a cluster's dataset across multiple nodes but it's handled transparently by the database. Whenever I've heard the term "sharding" it's usually in reference to the a
56.
by freels 10y ago
Snapshot reads require just as many network communications as Cassandra does at consistency ONE, and provide a better correctness guarantee because effect order has already been determined. Data replicas can the single valid state at the re
57.
by freels 10y ago
IME, while Google's network is really good (based on my experience w/ GCP at least), AWS cross-region traffic, for example, is still pretty reliable. Reliable enough, at least, that trading off perfect resiliency in the face of pa
58.
by freels 10y ago
Calvin is still a CP system, so nodes outside of the quorum cannot proceed. The point however is that partitions are rare enough that a CP system can still provide a high level of availability, despite the theoretical limitation. Eric Brewe
59.
by freels 10y ago
Yes. For log data, it's simply a matter of reading from a replica peer of the down node. For transaction resolution it's a bit easier if you are able to assume more about the storage layer's semantics. For example, if you sto
60.
by freels 10y ago
A better way to think about how Calvin fits into our stack is as a building block similar to two-phase-commit, on which we've built the higher level transaction semantics. And while we do not have any plans on supporting session trans

More ›