4 ms·

This is very cool! FoundationDB excites a lot of people because it's an extremely scalable and extremely reliable distributed database that supports ACID trans

This is very cool! FoundationDB excites a lot of people because it's an extremely scalable and extremely reliable distributed database that supports ACID transactions, and which is both open-source and has Apple standing behind it. And yeah, all of that is pretty nice. But arguably the real power comes from the fact that it exposes a relatively low-level data model that can then be wrapped in one or more stateless "layers". All of these layers write to the same storage substrate, so you can have your document database, your SQL database, your time-series database, your consensus/coordination store, your distributed task queue, etc., etc., but you're only actually operating one stateful system. Your SREs will thank you. Writing these layers to be scalable and high-performance can be challenging, but it looks like Apple is actively doing it and willing to release the results to the rest of us. This also suggests that their previous open-sourcing of the MongoDB-compatible document layer wasn't a one-off fluke. All of this is very good news for everybody who needs to run databases in the real world. Full disclosure: I worked on FoundationDB a long, long time ago.

wwilson 8y ago
Wow, from the post: "Together, the Record Layer and FoundationDB form the backbone of Apple's CloudKit. We wrote a paper describing how we built the Record Layer to run at massive scale and how CloudKit uses it." I think this is the first time that little detail has been publicly disclosed.
- Scriptor 8y ago
  Isn't the code on GitHub?
rubyn00bie 8y ago
Just wanted to say thanks for your testing distributed systems talk! That has been a huge help to me as I develop distributed systems and consider the problems inherent to them.
- wikibob 8y ago
  Is this the video you mention? https://youtu.be/4fFDFbi3toc https://youtu.be/4fFDFbi3toc
  - rubyn00bie 8y ago
    Yes it is! Sorry for not linking it and thank you for doing so!
nathan_long 8y ago
This definitely sounds cool. > your SQL database They mention "a declarative query API", but as far as I can tell that's not actually SQL, right? So migrating from another relational db would require learning a new query language?
- rdsubhas 8y ago
  One could technically write an SQL translation layer on top of it, as a client-side library? Or does it need support on the server-side / record layer?
  - ryanworl 8y ago
    It would be tough to implement every SQL construct on top of what is there today. I could explain why (I have tried and presumably ran into the same issues they did described in various areas in the docs and the paper), but the docs give an authoritative answer to save me typing :) "In the future it is possible that the Record Layer may develop a formal query language, but it is unlikely that such a language would closely resemble the SQL standard." [0] [0] https://foundationdb.github.io/fdb-record-layer/FAQ.html https://foundationdb.github.io/fdb-record-layer/FAQ.html
    - dman 8y ago
      Could you briefly describe the challenges you ran into?
      - ryanworl 8y ago
        * Schema evolution and schema management. Keeping schema state updated transactionally for a single, high-traffic table is complicated. An FDB core feature I requested could make this easier. Basically managing the lifecycle of adding, building, using, and dropping indexes is the hard part. * Full table scans on large tables in a R/W transaction would fail due to the 5s transaction duration restriction. This is obviously a bad idea regardless of database, but if you wanted to support SQL you would have to allow it * Sorting and joins require (theoretically) the same amount of memory as your data size, or the ability to spill to disk. This isn't FDB specific, but offering this kind of feature in a scale-out, high-concurrency model would be tough for QoS. There are plenty more, but those are the big ones I ran across during design. The actual SQL interface part is not hard, but you would have to disallow many useful constructs people normally expect to work.
shereadsthenews 8y ago
I think foundation excites a lot of people who have never read its code or tried to operate it and therefore have only these statements of hype to go on.
- ryanworl 8y ago
  FoundationDB is about as honest and up-front about its limitations and flaws as any system I've ever seen. Have you had a negative experience with it you can share?
Nullabillity 8y ago
> and has Apple standing behind it. So far behind it that they already shut it down once.
SloopJon 8y ago
Do I understand correctly that the record layer is only usable by Java clients? That strikes me as a drawback of the layer approach: unless you supply a server protocol, like the document layer does, every language essentially reimplements the layer from scratch. That seems to be the case for the tuple layer, for example.
- gregwebs 8y ago
  It certainly doesn't have to be the case. The MongoDB layer can be used via the MongoDB protocol. TiDB is an entire MySQL implementation that any MySQL client can connect to which is built on top of TiKV (equivalent to FDB). If the goal is lower overhead of data transfer, another approach is writing layers in C (or Rust exposed as C) and then generating bindings for different languages.

wwilson 8y ago

Wow, from the post: "Together, the Record Layer and FoundationDB form the backbone of Apple's CloudKit. We wrote a paper describing how we built the Record Layer to run at massive scale and how CloudKit uses it." I think this is the first time that little detail has been publicly disclosed.

Scriptor 8y ago
Isn't the code on GitHub?

rubyn00bie 8y ago

Just wanted to say thanks for your testing distributed systems talk! That has been a huge help to me as I develop distributed systems and consider the problems inherent to them.

wikibob 8y ago
Is this the video you mention? https://youtu.be/4fFDFbi3toc https://youtu.be/4fFDFbi3toc
- rubyn00bie 8y ago
  Yes it is! Sorry for not linking it and thank you for doing so!

nathan_long 8y ago

This definitely sounds cool. > your SQL database They mention "a declarative query API", but as far as I can tell that's not actually SQL, right? So migrating from another relational db would require learning a new query language?

rdsubhas 8y ago
One could technically write an SQL translation layer on top of it, as a client-side library? Or does it need support on the server-side / record layer?
- ryanworl 8y ago
  It would be tough to implement every SQL construct on top of what is there today. I could explain why (I have tried and presumably ran into the same issues they did described in various areas in the docs and the paper), but the docs give an authoritative answer to save me typing :) "In the future it is possible that the Record Layer may develop a formal query language, but it is unlikely that such a language would closely resemble the SQL standard." [0] [0] https://foundationdb.github.io/fdb-record-layer/FAQ.html https://foundationdb.github.io/fdb-record-layer/FAQ.html
  - dman 8y ago
    Could you briefly describe the challenges you ran into?
    - ryanworl 8y ago
      * Schema evolution and schema management. Keeping schema state updated transactionally for a single, high-traffic table is complicated. An FDB core feature I requested could make this easier. Basically managing the lifecycle of adding, building, using, and dropping indexes is the hard part. * Full table scans on large tables in a R/W transaction would fail due to the 5s transaction duration restriction. This is obviously a bad idea regardless of database, but if you wanted to support SQL you would have to allow it * Sorting and joins require (theoretically) the same amount of memory as your data size, or the ability to spill to disk. This isn't FDB specific, but offering this kind of feature in a scale-out, high-concurrency model would be tough for QoS. There are plenty more, but those are the big ones I ran across during design. The actual SQL interface part is not hard, but you would have to disallow many useful constructs people normally expect to work.

shereadsthenews 8y ago

I think foundation excites a lot of people who have never read its code or tried to operate it and therefore have only these statements of hype to go on.

ryanworl 8y ago
FoundationDB is about as honest and up-front about its limitations and flaws as any system I've ever seen. Have you had a negative experience with it you can share?

Nullabillity 8y ago

> and has Apple standing behind it. So far behind it that they already shut it down once.

SloopJon 8y ago

Do I understand correctly that the record layer is only usable by Java clients? That strikes me as a drawback of the layer approach: unless you supply a server protocol, like the document layer does, every language essentially reimplements the layer from scratch. That seems to be the case for the tuple layer, for example.

gregwebs 8y ago
It certainly doesn't have to be the case. The MongoDB layer can be used via the MongoDB protocol. TiDB is an entire MySQL implementation that any MySQL client can connect to which is built on top of TiKV (equivalent to FDB). If the goal is lower overhead of data transfer, another approach is writing layers in C (or Rust exposed as C) and then generating bindings for different languages.