3 ms·

I see a lot of snarky comments (maybe that has to do with the journalism of the article), but my recollection of FoundationDB is that it is interesting. It's t

by fdr 14y ago

I see a lot of snarky comments (maybe that has to do with the journalism of the article), but my recollection of FoundationDB is that it is interesting. It's trying to tackle the problem of enabling ACID, while also not being SQL-dialect oriented, putting it in a somewhat unique part of the overall landscape. So, if you are put off by TechCrunch's journalism, I don't know of a great reason to let that spill over into one's gut assessment FoundationDB. Here is a link to previous discussion, where many people try to poke holes into the tradeoffs being made for FoundationDB: http://news.ycombinator.com/item?id=4294719 http://news.ycombinator.com/item?id=4294719 Perhaps the company needs to work to reign in some of the language, it seems like the last discussion revolved around some aspects of the system that tripped a lot of skepticism-detectors (or at least "glaring caveat-omission" detectors) last time also.

Dave_Rosenthal 14y ago
FoundationDB founder here. Thanks for this comment. It's pretty hard to talk about this detailed technical stuff in the chaos of a conference. The discussion you link to and our web page both have a bunch of information that I think is high signal/noise for people interested in the tech details. If anyone is at Disrupt, please come by and see our live demo of a portable cluster of the database dealing with power and network failures. Of course, we'll try our best to answer questions here too.
- fdr 14y ago
  If I had any feedback, it might be to produce a frank document of trade-offs in your system: workloads that work very well, pathological workloads that perform poorly or lead to some awkward code, given the mechanisms at play. I'm sure you have chosen your trade-offs carefully, or else you wouldn't have bothered doing the hard work to commercialize them. Here you may want to assess the advantages of other general approaches (I think specific implementations would be too much detail and invite a quagmire of nit-picking) versus your own, but the problems they also create that you do not have. It's something I'm going to try to do anyway, but it would make me feel as though I have a more constructive relationship with the company rather than trying to guess what mix of design decisions you fall in, probably all of which have at least been implemented at least once independently before in some system or another -- however, your mix is likely novel. However, I don't see this done very often in proprietary offerings (that's still the case for FoundationDB, right?), and I'm not sure if that's because such frankness is a stupid idea or if because everyone is just doing what everyone around them does. It is something that more collaborative open source projects do more freely.
  - nlavezzo 14y ago
    Different FoundationDB founder here :) We'd be happy to give you information that would be helpful for your document - your offer to put something like that together is generous, helpful, and we'll definitely take you up on it. Why don't you shoot me an email at nick dot lavezzo at foundationdb. If you're at Disrupt or in the SF area we might even be able to get together before we head back to DC on Thursday morning. Thanks again for the helpful comments and interest!
    - fdr 14y ago
      I am speaking more generally -- I'm not evaluating a new database for my toolchain at the moment, and to be honest, I am very shy of anything that is proprietary-only. However, I thought your ideas were interesting regardless, so I felt compelled to defend their interesting-ness. I'm just tired of trying pierce through proprietary database literature language to sort a new database system that has Apparently Magical Properties into the multi-dimensional bin of tradeoffs. Consider the FAQ for Redis: http://redis.io/topics/faq http://redis.io/topics/faq Or, the very short rundown in the second half of the home page of Postgres-XC http://postgres-xc.sourceforge.net/ http://postgres-xc.sourceforge.net/, although it probably could use some fine-tuning. Many of the developers, including the first one, are not native english speakers. A good example in proprietary-land is Datomic: http://www.datomic.com/faq.html http://www.datomic.com/faq.html. Here's a good paragraph: How does Datomic handle consistency, availability and scale? Because Datomic separates the handling of transaction processing and query/read, it can make independent choices for each facility. Transactions favor consistency over availability, and the loss of a transactor will require traditional failover approaches. Queries and reads, however, are serviced by the highly available, replicated and scalable infrastructure of DynamoDB. And, since the data segments stored by Datomic in Dynamo are immutable, they are always consistent. Thus, Datomic is well suited for applications that require write consistency and read scalability. On the other hand, I really appreciate your benchmarks to give me a ballpark idea, and think it's great that you offer them up so easily. They could probably take a less pronounced position in your literature if you find you are in need of space for other material, though.
nirvana 14y ago
It appears that they don't actually understand what the problem is, and are solving it by introducing a situation where the database will just stop working in the face of a partition. Maybe that's not right, but they need to stop hand waving. A much better solution is being worked on by a Mr. Hellerstein who will be speaking at the upcoming Ricon: http://basho.com/community/ricon2012/ http://basho.com/community/ricon2012/
- nlavezzo 14y ago
  Hi - FoundationDB founder here. I'll point you to a comment thread from a previous HN discussion where my co-founder Dave Scherer addressed how we deal with partitions: http://news.ycombinator.com/item?id=4295770 http://news.ycombinator.com/item?id=4295770 Please let us know if you have any questions this doesn't answer and we'd be happy to answer. Also, if you'd like to actually SEE us deal with a data partition, we can show you that live at Disrupt with the demo cluster we've built :)

by fdr 14y ago

Dave_Rosenthal 14y ago

FoundationDB founder here. Thanks for this comment. It's pretty hard to talk about this detailed technical stuff in the chaos of a conference. The discussion you link to and our web page both have a bunch of information that I think is high signal/noise for people interested in the tech details. If anyone is at Disrupt, please come by and see our live demo of a portable cluster of the database dealing with power and network failures. Of course, we'll try our best to answer questions here too.

fdr 14y ago
If I had any feedback, it might be to produce a frank document of trade-offs in your system: workloads that work very well, pathological workloads that perform poorly or lead to some awkward code, given the mechanisms at play. I'm sure you have chosen your trade-offs carefully, or else you wouldn't have bothered doing the hard work to commercialize them. Here you may want to assess the advantages of other general approaches (I think specific implementations would be too much detail and invite a quagmire of nit-picking) versus your own, but the problems they also create that you do not have. It's something I'm going to try to do anyway, but it would make me feel as though I have a more constructive relationship with the company rather than trying to guess what mix of design decisions you fall in, probably all of which have at least been implemented at least once independently before in some system or another -- however, your mix is likely novel. However, I don't see this done very often in proprietary offerings (that's still the case for FoundationDB, right?), and I'm not sure if that's because such frankness is a stupid idea or if because everyone is just doing what everyone around them does. It is something that more collaborative open source projects do more freely.
- nlavezzo 14y ago
  Different FoundationDB founder here :) We'd be happy to give you information that would be helpful for your document - your offer to put something like that together is generous, helpful, and we'll definitely take you up on it. Why don't you shoot me an email at nick dot lavezzo at foundationdb. If you're at Disrupt or in the SF area we might even be able to get together before we head back to DC on Thursday morning. Thanks again for the helpful comments and interest!
  - fdr 14y ago
    I am speaking more generally -- I'm not evaluating a new database for my toolchain at the moment, and to be honest, I am very shy of anything that is proprietary-only. However, I thought your ideas were interesting regardless, so I felt compelled to defend their interesting-ness. I'm just tired of trying pierce through proprietary database literature language to sort a new database system that has Apparently Magical Properties into the multi-dimensional bin of tradeoffs. Consider the FAQ for Redis: http://redis.io/topics/faq http://redis.io/topics/faq Or, the very short rundown in the second half of the home page of Postgres-XC http://postgres-xc.sourceforge.net/ http://postgres-xc.sourceforge.net/, although it probably could use some fine-tuning. Many of the developers, including the first one, are not native english speakers. A good example in proprietary-land is Datomic: http://www.datomic.com/faq.html http://www.datomic.com/faq.html. Here's a good paragraph: How does Datomic handle consistency, availability and scale? Because Datomic separates the handling of transaction processing and query/read, it can make independent choices for each facility. Transactions favor consistency over availability, and the loss of a transactor will require traditional failover approaches. Queries and reads, however, are serviced by the highly available, replicated and scalable infrastructure of DynamoDB. And, since the data segments stored by Datomic in Dynamo are immutable, they are always consistent. Thus, Datomic is well suited for applications that require write consistency and read scalability. On the other hand, I really appreciate your benchmarks to give me a ballpark idea, and think it's great that you offer them up so easily. They could probably take a less pronounced position in your literature if you find you are in need of space for other material, though.

nirvana 14y ago

It appears that they don't actually understand what the problem is, and are solving it by introducing a situation where the database will just stop working in the face of a partition. Maybe that's not right, but they need to stop hand waving. A much better solution is being worked on by a Mr. Hellerstein who will be speaking at the upcoming Ricon: http://basho.com/community/ricon2012/ http://basho.com/community/ricon2012/

nlavezzo 14y ago
Hi - FoundationDB founder here. I'll point you to a comment thread from a previous HN discussion where my co-founder Dave Scherer addressed how we deal with partitions: http://news.ycombinator.com/item?id=4295770 http://news.ycombinator.com/item?id=4295770 Please let us know if you have any questions this doesn't answer and we'd be happy to answer. Also, if you'd like to actually SEE us deal with a data partition, we can show you that live at Disrupt with the demo cluster we've built :)