5 ms·

The client is complex and needs very sophisticated testing, so there is only one implementation. All the language bindings use the C library.

eklavya 8y ago
Ok, thanks :)
ddorian43 8y ago
Curious why the client is complicated compared to other dbs in same space ?
- voidmain 8y ago
  In some distributed databases the client just connects to some machine in the cluster and tells it what it wants to do. You pay the extra latency as it redirects these requests where they should go. In FDB's envisioned architecture, the "client" is usually a (stateless, higher layer) database node itself! So the client encompasses the first layer of distributed database technology, connects directly to services throughout the cluster, does reads directly (1xRTT happy path) from storage replicas, etc. It simulates read-your-writes ordering within a transaction, using a pretty complex data structure. It shares a lot of code with the rest of the database. If you wanted, you could write a "FDB API service" over the client and connect to it with a thin client, reproducing the more conventional design (but you had better have a good async RPC system!)
  - ddorian43 8y ago
    Wouldn't layers be hard to be built on the server (since you have to also change the client) and slow to be built as a layer (since it will be another separate service) ?
    - qaq 8y ago
      I am guessign you'd pretty much embed the client into your higher layer
    - voidmain 8y ago
      I'm not sure what you are asking, but depending on their individual performance and security needs layers are usually either (a) libraries embedded into their clients, (b) services colocated with their clients, (c) services running in a separate tier, or (d) services co-located with fdbservers. In any of these cases they use the FoundationDB client to communicate with FoundationDB.
      - mogui 8y ago
        In case (c) or (d) how can a layer leverage the distributed facilities that FDB gives? I mean if I have clients that connect to a "layer service" that is the one who talks to FDB, I have to manage "layer service" scalabily, fault tolerance etc... by myself.
        voidmain 8y ago
        Yes, and that's the main advantage of choosing (a) or (b). But it's not quite as hard as it sounds; since all your state is safely in fdb you "just" have to worry about load balancing a stateless service.
        mogui 8y ago
        got it, what will you suggest to do something like that? a simple RPC with a good async framework I've read, like what? an RPC service on top of Twisted for python, similar things in other languages? thanks :)
  - zbentley 8y ago
    > but you had better have a good async RPC system! The microservices crew with their "our database is behind a REST/Thrift/gRPC/FizzBuzzWhatnot microservice" pattern is still catching up to the significance of this statement.
    - foodbaby 8y ago
      This might be a dumb question (from someone used to using blocking JBDC) but why is async RPC important in this case? Just trying to understand. And can gRPC not provide good async RPC?
      - ddorian43 8y ago
        Imagine single core single threaded design. You send 2 requests for 1 row each. First request the row needs to be read from disk HDD. It takes 2ms. Second request, the row is already in ram, it takes microseconds but still has to wait for the first request to finish. Threads have overhead when having a lot of concurrency (thousands/millions requests/second). For extreme async, see seastar-framework and scylladb design. TLDR: high concurrency, low overhead etc.
      - zbentley 8y ago
        I was referring to the trend of splitting up applications into highly distributed collections of services without addressing the fact that every point where they communicate over the network is a potential point of pathological failure (from blocking to duplicate-delivery etc). This tendency replaces highly reliable network protocols (i.e. the one you use to talk to your RDBMS) with ad hoc and frequently technically shoddy communication patterns, with minimal consideration for how it might fail in complex, distributed ways. While not always done wrong, a lot of microservice-ification efforts are quite hubristic in this area, and suffer for it over the long term.

eklavya 8y ago

Ok, thanks :)

ddorian43 8y ago

Curious why the client is complicated compared to other dbs in same space ?

voidmain 8y ago
In some distributed databases the client just connects to some machine in the cluster and tells it what it wants to do. You pay the extra latency as it redirects these requests where they should go. In FDB's envisioned architecture, the "client" is usually a (stateless, higher layer) database node itself! So the client encompasses the first layer of distributed database technology, connects directly to services throughout the cluster, does reads directly (1xRTT happy path) from storage replicas, etc. It simulates read-your-writes ordering within a transaction, using a pretty complex data structure. It shares a lot of code with the rest of the database. If you wanted, you could write a "FDB API service" over the client and connect to it with a thin client, reproducing the more conventional design (but you had better have a good async RPC system!)
- ddorian43 8y ago
  Wouldn't layers be hard to be built on the server (since you have to also change the client) and slow to be built as a layer (since it will be another separate service) ?
  - qaq 8y ago
    I am guessign you'd pretty much embed the client into your higher layer
  - voidmain 8y ago
    I'm not sure what you are asking, but depending on their individual performance and security needs layers are usually either (a) libraries embedded into their clients, (b) services colocated with their clients, (c) services running in a separate tier, or (d) services co-located with fdbservers. In any of these cases they use the FoundationDB client to communicate with FoundationDB.
    - mogui 8y ago
      In case (c) or (d) how can a layer leverage the distributed facilities that FDB gives? I mean if I have clients that connect to a "layer service" that is the one who talks to FDB, I have to manage "layer service" scalabily, fault tolerance etc... by myself.
      - voidmain 8y ago
        Yes, and that's the main advantage of choosing (a) or (b). But it's not quite as hard as it sounds; since all your state is safely in fdb you "just" have to worry about load balancing a stateless service.
        mogui 8y ago
        got it, what will you suggest to do something like that? a simple RPC with a good async framework I've read, like what? an RPC service on top of Twisted for python, similar things in other languages? thanks :)
- zbentley 8y ago
  > but you had better have a good async RPC system! The microservices crew with their "our database is behind a REST/Thrift/gRPC/FizzBuzzWhatnot microservice" pattern is still catching up to the significance of this statement.
  - foodbaby 8y ago
    This might be a dumb question (from someone used to using blocking JBDC) but why is async RPC important in this case? Just trying to understand. And can gRPC not provide good async RPC?
    - ddorian43 8y ago
      Imagine single core single threaded design. You send 2 requests for 1 row each. First request the row needs to be read from disk HDD. It takes 2ms. Second request, the row is already in ram, it takes microseconds but still has to wait for the first request to finish. Threads have overhead when having a lot of concurrency (thousands/millions requests/second). For extreme async, see seastar-framework and scylladb design. TLDR: high concurrency, low overhead etc.
    - zbentley 8y ago
      I was referring to the trend of splitting up applications into highly distributed collections of services without addressing the fact that every point where they communicate over the network is a potential point of pathological failure (from blocking to duplicate-delivery etc). This tendency replaces highly reliable network protocols (i.e. the one you use to talk to your RDBMS) with ad hoc and frequently technically shoddy communication patterns, with minimal consideration for how it might fail in complex, distributed ways. While not always done wrong, a lot of microservice-ification efforts are quite hubristic in this area, and suffer for it over the long term.