4 ms·

This is very impressive, however... See this tweet by @aphyr: https://twitter.com/aphyr/status/542755074380791809 https://twitter.com/aphyr/status/542755074380

by hendzen 12y ago

This is very impressive, however... See this tweet by @aphyr: https://twitter.com/aphyr/status/542755074380791809 https://twitter.com/aphyr/status/542755074380791809 (All credit for the idea in this comment is due to @aphyr) Basically because the transactions modified keys selected from a uniform distribution, the probability of contention was extremely low. AKA this workload is basically a data-parallel problem, somewhat lessening the impressiveness of the high throughput. Would be interesting to see it with a Zipfian distribution (or even better, a Biebermark [0]) [0] - http://smalldatum.blogspot.co.il/2014/04/biebermarks.html http://smalldatum.blogspot.co.il/2014/04/biebermarks.html

vosper 12y ago
Yeah, this would have been more interesting if they could tell us what the throughput is given various levels of contention.
jrallison 12y ago
Absolutely. Choosing the proper data model based on your access patterns is one of the best ways of keeping FDB performance high by reducing the likelihood of contention.
deleted 12y ago
[deleted]
Dave_Rosenthal 12y ago
Actually, @aphyr's analysis is wrong. In fact it's actually worse that he says. There is an exactly 0% chance of conflicts between two transactions that each only write 20 random keys (as they do in this test). This is perhaps counterintuitive, but, because there are no reads, any serialization order is possible and therefore there is no chance of conflict. Equally wrong is his assumption that this has any bearing on the performance of FoundationDB. In fact FoundationDB will perform the same whether the conflict rate is high or low. This isn't to say that FoundationDB somehow cheats the laws of transaction conflicts, just that it has to do all the work in either case. There is no trick or cheat on this test--this same performance will hold on a variety of workloads with varying conflict rates as well as those including reads.
- dmitrig01 12y ago
  I'm not an expert in this, but: because there are no reads, any serialization order is possible and therefore there is no chance of conflict I don't understand how this is the case. If clients A and B try to write to the same key, it can be serialized as {A,B} or {B,A}, but in either case, there is some kind of conflict... no?
  - Dave_Rosenthal 12y ago
    Nope. Not unless someone actually read that same key in their transaction. Then, with optimistic concurrency at least, you might get a conflict at commit time that basically says "Hey, someone changed that key you read in the meantime so your write might not be correct anymore".
    - dmitrig01 12y ago
      Ah – I see what you mean now! If they are blind writes, not some kind of CAS operation (reading from and modifying the same key), then there is no conflict. I thought you meant that clients A and B do their writes, and only if someone at a later time were to observe the value at that key, a conflict would be caused (Heisenberg-style). Thanks for clarifying!
      - AYBABTME 12y ago
        DBs that are consistent until you observe them, a funny thought. In some parallel universe, the DB is still consistent; one could argue.
        dmitrig01 12y ago
        Heh, this is exactly why I could not understand what was being said.
      - itp 12y ago
        Just to further clarify, the transactions (and therefore transaction conflict detection) are more flexible than simple CAS operations. If transaction A reads key K1, and then writes to key K2, it will be rejected as conflicting if transaction B writes to key K1 and is serialized as happening before transaction K1 is committed (even though no other transaction wrote to key K2). Every transaction in FoundationDB is submitted as a collection of writes/mutations, but also contains records of all keys read (and the consistent version at which they were read). In this test, the transactions all have empty conflict ranges, and thus cannot conflict with other transactions (but we still have to check!).
wmf 12y ago
Unfortunately this problem isn't specific to FoundationDB; the old "industry-standard" TPC-C benchmark has a similar low-contention design which has led to years of unrepresentative performance tuning and benchmarketing.
- imanaccount247 12y ago
  How is it unrepresentative? TPC-C involves several queries simulating new orders being entered, being updated, filled, paid, etc. It is very representative of the workload it is simulating. If you want to simulate a different workload, pick one of the other benchmarks instead of C.
deleted 12y ago
[deleted]
- felixgallo 12y ago
  all your wishes are granted: https://github.com/aphyr https://github.com/aphyr To be sure, he revels in the discomfort of others too much. But he's a young kid just finding his voice, and his contributions to practical distributed database discussions are pretty much without parallel. Just ignore the bluster and you'll learn something new from kyle every day, even if it's about his buttcheeks.

by hendzen 12y ago

vosper 12y ago

Yeah, this would have been more interesting if they could tell us what the throughput is given various levels of contention.

jrallison 12y ago

Absolutely. Choosing the proper data model based on your access patterns is one of the best ways of keeping FDB performance high by reducing the likelihood of contention.

deleted 12y ago

[deleted]

Dave_Rosenthal 12y ago

Actually, @aphyr's analysis is wrong. In fact it's actually worse that he says. There is an exactly 0% chance of conflicts between two transactions that each only write 20 random keys (as they do in this test). This is perhaps counterintuitive, but, because there are no reads, any serialization order is possible and therefore there is no chance of conflict. Equally wrong is his assumption that this has any bearing on the performance of FoundationDB. In fact FoundationDB will perform the same whether the conflict rate is high or low. This isn't to say that FoundationDB somehow cheats the laws of transaction conflicts, just that it has to do all the work in either case. There is no trick or cheat on this test--this same performance will hold on a variety of workloads with varying conflict rates as well as those including reads.

dmitrig01 12y ago
I'm not an expert in this, but: because there are no reads, any serialization order is possible and therefore there is no chance of conflict I don't understand how this is the case. If clients A and B try to write to the same key, it can be serialized as {A,B} or {B,A}, but in either case, there is some kind of conflict... no?
- Dave_Rosenthal 12y ago
  Nope. Not unless someone actually read that same key in their transaction. Then, with optimistic concurrency at least, you might get a conflict at commit time that basically says "Hey, someone changed that key you read in the meantime so your write might not be correct anymore".
  - dmitrig01 12y ago
    Ah – I see what you mean now! If they are blind writes, not some kind of CAS operation (reading from and modifying the same key), then there is no conflict. I thought you meant that clients A and B do their writes, and only if someone at a later time were to observe the value at that key, a conflict would be caused (Heisenberg-style). Thanks for clarifying!
    - AYBABTME 12y ago
      DBs that are consistent until you observe them, a funny thought. In some parallel universe, the DB is still consistent; one could argue.
      - dmitrig01 12y ago
        Heh, this is exactly why I could not understand what was being said.
    - itp 12y ago
      Just to further clarify, the transactions (and therefore transaction conflict detection) are more flexible than simple CAS operations. If transaction A reads key K1, and then writes to key K2, it will be rejected as conflicting if transaction B writes to key K1 and is serialized as happening before transaction K1 is committed (even though no other transaction wrote to key K2). Every transaction in FoundationDB is submitted as a collection of writes/mutations, but also contains records of all keys read (and the consistent version at which they were read). In this test, the transactions all have empty conflict ranges, and thus cannot conflict with other transactions (but we still have to check!).

wmf 12y ago

Unfortunately this problem isn't specific to FoundationDB; the old "industry-standard" TPC-C benchmark has a similar low-contention design which has led to years of unrepresentative performance tuning and benchmarketing.

imanaccount247 12y ago
How is it unrepresentative? TPC-C involves several queries simulating new orders being entered, being updated, filled, paid, etc. It is very representative of the workload it is simulating. If you want to simulate a different workload, pick one of the other benchmarks instead of C.

deleted 12y ago

[deleted]

felixgallo 12y ago
all your wishes are granted: https://github.com/aphyr https://github.com/aphyr To be sure, he revels in the discomfort of others too much. But he's a young kid just finding his voice, and his contributions to practical distributed database discussions are pretty much without parallel. Just ignore the bluster and you'll learn something new from kyle every day, even if it's about his buttcheeks.