4 ms·

> Can't secondary indexes just be implemented in your state machine? Secondary indexes are implemented as an additional smaller table in most databases (Index

by geenat 3y ago

> Can't secondary indexes just be implemented in your state machine? Secondary indexes are implemented as an additional smaller table in most databases (Index -> Primary Key), which requires a write, too. Don't take my word for it- run some benchmarks yourself to see the performance cliffs appear in these "distributed strongly consistent" databases when you have a handful of indexes. It's why many require 10x the hardware to scale past the performance of a single node postgres / mysql. > Calling FoundationDB "just a layer on top of TiKV" is… well… :-) FoundationDB recommends running TiDB ontop to get an SQL layer...

richieartoul 3y ago
I don’t think there is any officially recommended way to run SQL ontop of FoundationDB. I suspect the TiDB comment you saw was someone speculating about an idea maybe (I vaguely recall some GitHub issues about it). There are some well supported document interfaces like the official record layer or Tigris though.
- geenat 3y ago
  You're right, my mistake.
nemothekid 3y ago
>FoundationDB recommends running TiDB ontop to get an SQL layer... You might be mixing up projects here - TiDB/TiKV are completely separate projects from FoundationDB. I don't think FoundationDB has a supported SQL layer.
eatonphil 3y ago
> Secondary indexes are implemented as an additional smaller table in most databases (Index -> Primary Key), which requires a write, too. > Don't take my word for it- run some benchmarks yourself to see the performance cliffs appear in these "distributed strongly consistent" databases when you have a handful of indexes. I'm not saying secondary indexes aren't a performance cliff. I'm saying I can't think of a reason they must require additional quorum on top of the initial message that would require a secondary index update. Since you said: > Secondary indexes in "distributed strongly consistent" systems is what ruins performance: because each index is +1 write to another "index table" (... +1 raft quorum check!).
- _benedict 3y ago
  They will be partitioned by a different key. You may in fact have to touch two extra quorums - one to delete the old entry and one to insert the new entry. This depends on the specifics of how your database works, of course. I’m not sure this would involve multiple consensus quorums in FDB for instance, given that I believe it instead relies on a centralised timestamp service.

by geenat 3y ago

richieartoul 3y ago

I don’t think there is any officially recommended way to run SQL ontop of FoundationDB. I suspect the TiDB comment you saw was someone speculating about an idea maybe (I vaguely recall some GitHub issues about it). There are some well supported document interfaces like the official record layer or Tigris though.

geenat 3y ago
You're right, my mistake.

nemothekid 3y ago

>FoundationDB recommends running TiDB ontop to get an SQL layer... You might be mixing up projects here - TiDB/TiKV are completely separate projects from FoundationDB. I don't think FoundationDB has a supported SQL layer.

eatonphil 3y ago

> Secondary indexes are implemented as an additional smaller table in most databases (Index -> Primary Key), which requires a write, too. > Don't take my word for it- run some benchmarks yourself to see the performance cliffs appear in these "distributed strongly consistent" databases when you have a handful of indexes. I'm not saying secondary indexes aren't a performance cliff. I'm saying I can't think of a reason they must require additional quorum on top of the initial message that would require a secondary index update. Since you said: > Secondary indexes in "distributed strongly consistent" systems is what ruins performance: because each index is +1 write to another "index table" (... +1 raft quorum check!).

_benedict 3y ago
They will be partitioned by a different key. You may in fact have to touch two extra quorums - one to delete the old entry and one to insert the new entry. This depends on the specifics of how your database works, of course. I’m not sure this would involve multiple consensus quorums in FDB for instance, given that I believe it instead relies on a centralised timestamp service.