3 ms·

Hmm, a previous competitor slamming them only after they're certain FoundationDB's gag order will prevent a rebuttal. Classy.

by orand 12y ago

jhugg 12y ago
Meh. Yes, it's critical about the "layers" model and about the SQL implementation in particular. Most of what I write isn't this critical, but I thought there was actually an interesting point here so I wrote it down. Take it for whatever it's worth.
lmb 12y ago
Correct me if I'm wrong, but what happened to FoundationDB smells like an acqui-hire par excellence. This article provides an interesting alternative to the "Apple bought this space age technology to keep it to themselves" narrative. Seems like the guy knows what he's talking about as well, b/c surprise, he's working on a DB.
- exelius 12y ago
  There's a third option: Apple had a need for a massive, distributed K/V store, and at their scale, it's cheaper to buy a company than a license.
  - fweespeech 12y ago
    http://cassandra.apache.org/ http://cassandra.apache.org/ I'm not sure why Apple would prefer FoundationDB to Cassandra for this usecase.
    - robertszkutak 12y ago
      Perhaps because FoundationDB is ACID compliant?
      - fweespeech 12y ago
        Quorum Writes, Quorum Reads is as ACID compliant as FoundationDB iirc. The only thing I can think of is Apple has an internal team that is building a Cassandra replacement and they need more qualified engineers with experience.
        jhugg 12y ago
        Not even close. FDB did arbitrary multi-key transactions. Cassandra can only do Compare-And-Set on single keys IIRC.
      - jhugg 12y ago
        So there is a point here. Apple is trying to compete with Google. Google has some amazing distributed systems, including Spanner, F1, MillWheel, etc... Apple has Cassandra and other OSS/COTS software. Not to ding Cassandra, but this is a problem for Apple. We've seen repeatedly that at huge scale, it often makes sense to own (or control) the software. See Facebook and LinkedIn as well. Now I don't think FDB (the product) is the answer, at least not in the short term. There are more problems scaling it to Apple's use case than there are working around Cassandra's lack of ACID. So I'm convinced the value of FDB is the experience in the engineers' brains. Apple need brains to run Cassandra, but also to figure out if Cassandra is the right long term path. Build, buy, adapt? It takes veterans to make the right call.
        threeseed 12y ago
        Apple runs Cassandra at scale today. It underpins most of iTunes Match and last time I checked all your iCloud data was sharded and stored in Cassandra. They run one of the latest clusters in the world (at least the last time I spoke with Datastax). Cassandra is a pretty easy database to run and scale and Datastax is just down the road from them to help. My guess is that FoundationDB is replacing their Teradata installation. Better to buy the company and invest heavily in it then let it not met it's full potential as a small startup.
      - threeseed 12y ago
        Why do people think ACID is some panacea for every database problem ? It doesn't save you from data loss. It doesn't help you scale. It doesn't make it easier to manage. It doesn't actually help with 99% of the problems you typically experience with a database. A more measured, intelligent consideration of pros/cons is needed.
    - electrum 12y ago
      According to the Cassandra home page: "One of the largest production deployments is Apple's, with over 75,000 nodes storing over 10 PB of data."
  - threeseed 12y ago
    Apple is a heavy user of Teradata which according to this article they were the quickest to reach a petabyte. It is now 2 years later and they could be in the tens of petabytes given the success of iPhone 6 and integration with Beats. https://gigaom.com/2013/03/27/why-apple-ebay-and-walmart-have-some-of-the-biggest-data-warehouses-youve-ever-seen/ https://gigaom.com/2013/03/27/why-apple-ebay-and-walmart-hav... Teradata is VERY expensive and my guess is reaching also scalability limits.
    - jhugg 12y ago
      FoundationDB solves completely different problems than Teradata. FDB isn't an analytical store.
arielweisberg 12y ago
Former VoltDB developer here. John makes some good technical arguments about why implementing transactional SQL on top of a distributed KV store (even a transactional one) is hard. The points about metadata performance and consistency were actually new ideas to me. I already had beef with moving the data around to SQL nodes as that is an obvious waste of capacity. But the importance of metadata in processing SQL queries in a distributed database never occurred to me even though I've lost a decent chunk of my life implementing that consistency. It's one of those things you take for granted if it's how you have always done it.
peterevans 12y ago
It seems like John had some technical points to make. It's hard for me to justify his post as "slamming" FoundationDB if his assertions are based on logic. Now, if you have a criticism of his points, you should feel free to rebut them. You certainly have not do so here.
- orand 12y ago
  The skillful use of logic in no way negates the slamminess. My primary issue is that the people best able to rebut John's well-argued points are no longer able to do so. My secondary issue is that John could have easily made the same logical arguments _before_ FoundationDB was acquired, but chose not to do so for whatever reason. This would have led to a much more enlightening debate than the one we're able to have now.
  - jhugg 12y ago
    I could have easily written the same post months ago, yes. None of these thoughts are new. It's actually precisely because FDB isn't a perceived competitor anymore that I can write this. As a vendor, I actually have much less of an agenda now. It's not like I'm worried about FDB stealing a customer. If I had posted this months ago, it would have been less credible and it would have felt tacky. The other point is that I think there's actually something to say here. I'm not just dumping on a dead product, I'm trying to show there's a lesson to learn here about trying to bolt SQL onto things (a trend). To contrast, if NuoDB disappeared tomorrow, I could write a solid post on why they could never technically achieve what their marketing said they could, but there's no lesson there. "Don't make bad engineering choices" is too generic.

Hmm, a previous competitor slamming them only after they're certain FoundationDB's gag order will prevent a rebuttal. Classy.

by orand 12y ago

Hmm, a previous competitor slamming them only after they're certain FoundationDB's gag order will prevent a rebuttal. Classy.

jhugg 12y ago

Meh. Yes, it's critical about the "layers" model and about the SQL implementation in particular. Most of what I write isn't this critical, but I thought there was actually an interesting point here so I wrote it down. Take it for whatever it's worth.

lmb 12y ago

Correct me if I'm wrong, but what happened to FoundationDB smells like an acqui-hire par excellence. This article provides an interesting alternative to the "Apple bought this space age technology to keep it to themselves" narrative. Seems like the guy knows what he's talking about as well, b/c surprise, he's working on a DB.

exelius 12y ago
There's a third option: Apple had a need for a massive, distributed K/V store, and at their scale, it's cheaper to buy a company than a license.
- fweespeech 12y ago
  http://cassandra.apache.org/ http://cassandra.apache.org/ I'm not sure why Apple would prefer FoundationDB to Cassandra for this usecase.
  - robertszkutak 12y ago
    Perhaps because FoundationDB is ACID compliant?
    - fweespeech 12y ago
      Quorum Writes, Quorum Reads is as ACID compliant as FoundationDB iirc. The only thing I can think of is Apple has an internal team that is building a Cassandra replacement and they need more qualified engineers with experience.
      - jhugg 12y ago
        Not even close. FDB did arbitrary multi-key transactions. Cassandra can only do Compare-And-Set on single keys IIRC.
    - jhugg 12y ago
      So there is a point here. Apple is trying to compete with Google. Google has some amazing distributed systems, including Spanner, F1, MillWheel, etc... Apple has Cassandra and other OSS/COTS software. Not to ding Cassandra, but this is a problem for Apple. We've seen repeatedly that at huge scale, it often makes sense to own (or control) the software. See Facebook and LinkedIn as well. Now I don't think FDB (the product) is the answer, at least not in the short term. There are more problems scaling it to Apple's use case than there are working around Cassandra's lack of ACID. So I'm convinced the value of FDB is the experience in the engineers' brains. Apple need brains to run Cassandra, but also to figure out if Cassandra is the right long term path. Build, buy, adapt? It takes veterans to make the right call.
      - threeseed 12y ago
        Apple runs Cassandra at scale today. It underpins most of iTunes Match and last time I checked all your iCloud data was sharded and stored in Cassandra. They run one of the latest clusters in the world (at least the last time I spoke with Datastax). Cassandra is a pretty easy database to run and scale and Datastax is just down the road from them to help. My guess is that FoundationDB is replacing their Teradata installation. Better to buy the company and invest heavily in it then let it not met it's full potential as a small startup.
    - threeseed 12y ago
      Why do people think ACID is some panacea for every database problem ? It doesn't save you from data loss. It doesn't help you scale. It doesn't make it easier to manage. It doesn't actually help with 99% of the problems you typically experience with a database. A more measured, intelligent consideration of pros/cons is needed.
  - electrum 12y ago
    According to the Cassandra home page: "One of the largest production deployments is Apple's, with over 75,000 nodes storing over 10 PB of data."
- threeseed 12y ago
  Apple is a heavy user of Teradata which according to this article they were the quickest to reach a petabyte. It is now 2 years later and they could be in the tens of petabytes given the success of iPhone 6 and integration with Beats. https://gigaom.com/2013/03/27/why-apple-ebay-and-walmart-have-some-of-the-biggest-data-warehouses-youve-ever-seen/ https://gigaom.com/2013/03/27/why-apple-ebay-and-walmart-hav... Teradata is VERY expensive and my guess is reaching also scalability limits.
  - jhugg 12y ago
    FoundationDB solves completely different problems than Teradata. FDB isn't an analytical store.

arielweisberg 12y ago

Former VoltDB developer here. John makes some good technical arguments about why implementing transactional SQL on top of a distributed KV store (even a transactional one) is hard. The points about metadata performance and consistency were actually new ideas to me. I already had beef with moving the data around to SQL nodes as that is an obvious waste of capacity. But the importance of metadata in processing SQL queries in a distributed database never occurred to me even though I've lost a decent chunk of my life implementing that consistency. It's one of those things you take for granted if it's how you have always done it.

peterevans 12y ago

It seems like John had some technical points to make. It's hard for me to justify his post as "slamming" FoundationDB if his assertions are based on logic. Now, if you have a criticism of his points, you should feel free to rebut them. You certainly have not do so here.

orand 12y ago
The skillful use of logic in no way negates the slamminess. My primary issue is that the people best able to rebut John's well-argued points are no longer able to do so. My secondary issue is that John could have easily made the same logical arguments _before_ FoundationDB was acquired, but chose not to do so for whatever reason. This would have led to a much more enlightening debate than the one we're able to have now.
- jhugg 12y ago
  I could have easily written the same post months ago, yes. None of these thoughts are new. It's actually precisely because FDB isn't a perceived competitor anymore that I can write this. As a vendor, I actually have much less of an agenda now. It's not like I'm worried about FDB stealing a customer. If I had posted this months ago, it would have been less credible and it would have felt tacky. The other point is that I think there's actually something to say here. I'm not just dumping on a dead product, I'm trying to show there's a lesson to learn here about trying to bolt SQL onto things (a trend). To contrast, if NuoDB disappeared tomorrow, I could write a solid post on why they could never technically achieve what their marketing said they could, but there's no lesson there. "Don't make bad engineering choices" is too generic.