7 ms·
The end of NoSQL? FathomDB launches scalable relational DB
- justinsb 17y agoNot sure whether it's the end of NoSQL, but we're certainly working to deliver the promises of NoSQL while retaining the power of SQL. To borrow from a much better man than I... This is not the end. It is not even the beginning of the end. But it is, perhaps, the end of the beginning.
- oomkiller 17y agoHere we go again. The AltDB movement is not just about scaling. While one of the goals is to make things scalable, it's not THE goal. BTW AltDB is my term for the NoSQL movement. It implies much less.
- justinsb 17y agoCare to define the goals? Apparently it's not about scaling. It's not about SQL, because there's no reason you can't support a SQL-style query language. So I do seriously want to know what's left... We're always on the lookout for the next feature to add ;-)
- _pius 17y agoTo me, it's about reducing the impedance mismatch between your data model and the implementation of that model in the database schema. Some data work better in a relational model and others work better in a document model. The idea of one ending the other seems preposterous to me. Congrats on the launch though!
- justinsb 17y agoCool answer - thanks. I guess some of this is about better ORMs, some of this is the schemaless idea, and probably the real answer is a combination of both. I worry that object databases were the last attempt to solve the impedance mismatch...
- neilc 17y agoOf course, this is a very old idea (predating the invention of relational databases, even) -- it just gets reinvented every decade or so, with a fresh helping of media hype.
- rbanffy 17y agoIt really gets old, doesn't it? I've seen so many market changes in my career and, yet, I still write code in a text editor with a terminal window on its side. All of them under a nice windowing environment.
- jrockway 17y agoOf course, this decade, we actually have OOP, so OO databases make a lot more sense.
- easp 17y agoWe had OOP last decade, and the one before that too.
- jemfinch 17y ago> To me, it's about reducing the impedance mismatch between your data model and the implementation of that model in the database schema. The whole point of that "impedance" in the relational model is to for people to think about their data and describe it in a program-independent way: years of experience in the field is what led to its development and what contributes to its continued relevance. Saying that's a problem with the relational model is like saying that design patterns are flawed because they describe commonly used OO designs. > To me, it's about reducing the impedance mismatch between your data model and the implementation of that model in the database schema. I'm not sure what sort of impedance mismatch could exist between a program and a relational model but not between a program and a document model. The latter is trivially implementable in terms of the former.
- _pius 17y agoTake a look at my comment again, I think you think I'm saying something I didn't. The whole point of that "impedance" in the relational model is to for people to think about their data and describe it in a program-independent way .... The relational model is hardly the only way to reason about data in a program-independent way. ... years of experience in the field is what led to its development and what contributes to its continued relevance. I agree, never said otherwise. Saying that's a problem with the relational model is like saying that design patterns are flawed because they describe commonly used OO designs. I didn't say there's a problem with the relational model. I'm not sure what sort of impedance mismatch could exist between a program and a relational model ... I never said the word "program," you did. I said "data model." You're conflating the data model with the program code (or maybe you think I am, can't really tell). They are not the same thing. http://en.wikipedia.org/wiki/Abstract_data_type http://en.wikipedia.org/wiki/Abstract_data_type All I said is that there are some domains for which relational models make sense and there are others for which document-oriented (or other "NoSQL") models make sense.
- oomkiller 17y agoIn my opinion, one of the goals of the AltDB movement is building software that fits in different parts of the CAP Theorem triangle. While relational databases are REALLY good (usually) at providing consistent data, things start to fall apart when you setup geographical replication. In the past, consistency seemed to be the most important thing for all applications. While it still is desirable and important, some users are beginning to prefer availability and partition tolerance over consistency.
- justinsb 17y agoI am reminded of the Judean People's Front from Life of Brian. Henceforth we shall call you AltDB, and we shall add geographical replication to your list of demands :-) More seriously, we'll see what we can do meet that list with our tech!
- cmelbye 17y agoI don't know if this is a goal of the NoSQL movement in general, but I use Redis rather than something like MySQL because I need speed. We're using Redis to route web requests, and we can't add 50+ ms onto every request on top of the >300 ms request time we have already from the application itself.
- justinsb 17y agoIf any DB is taking 50ms to reply (SQL, NoSQL, even MS Access!), something is seriously wrong. If you're using Rails, I'd fire up NewRelic or something like that and check out what on earth is going on. Of course, 300ms is also way too long. Do you mean under stress-testing, with request queuing? 300ms is a mind-boggling quantity of CPU cycles...
- cmelbye 17y agoIf we were to use MySQL, we'd be using multiple complex queries to get the job done. From what I've seen in New Relic on Heroku, queries have generally taken about 40-50 total for one request on PostgreSQL. We're building a general purpose method of running Ruby applications, so 300 ms is certainly a possible request time (including database, processing things, and rendering a view) for some applications that we may be running.
- justinsb 17y agoI think your performance here stems more from configuration than from MySQL vs Redis. If you're using Redis in-memory, if you dedicate the same amount of memory to your database, you should get roughly comparable performance between MySQL/PostreSQL and Redis. The under-the-hood differences if you're in memory just aren't that great, particularly because modern machines have such ridiculously powerful CPUs. Yes, it sounds like you'll have to do table join(s); yes, the joined tables need indexes; yes, you have the SQL parsing overhead; yes, it's a B-tree rather than hash-based indexing. But I'd be very surprised if you ended up with the order-of-magnitude differences you're suggesting. One of the things we're working on with FathomDB is ways to identify slow queries and help figure out what to do about them. If you're game, you can contact me with the queries and the table structure you were using in SQL and I'll try to figure out what's going on!
- adamt 17y agoI am sorry - but the subject here misses the point. Just having someone else who can host a cloud MySQL database for you in what appears to be a single location doesn't solve the problem that people have running high-availabilty global internet services. The challenges with MySQL are that scaling it effectively to a multiple-master, multi-datacenter solution is almost impossible. Especially if that's over a wide area network.
- justinsb 17y agoThe new technology isn't based on MySQL. It's a fully distributed, no single-point-of-failure, scalable relational database. You can choose fully managed hosted traditional MySQL, or we'll be hand-picking early customers for the scalable technology. Contact me if you're interested. We're starting with single-location, but we can certainly support cross-datacenter configurations. CAP seems to dictate that this will either require acknowledgement from the remote datacenter (choose consistency) or a window of loss (choose availability / partition tolerance). Until we see customers regularly hosting their webservers in multiple datacenters, we're going to stay focused on optimizing for a single location. But maybe my viewpoint is out-of-date... How many datacenters are you currently distributing your webservers across?
- btilly 17y agoBut maybe my viewpoint is out-of-date... How many datacenters are you currently distributing your webservers across? Heh, heh, heh. I work for Google. Need I say more? That said, we like to insist on N+2 globally or N+1 per region. Meaning that if it takes N data centers to host a specific service world wide, we need at least N+2 data centers. And if it is a global application we insist that every region have at least one more data center than needed. (If 2 go down then we can spill across regions, but we'd like to avoid that for latency reasons.) Failover from one data center to another usually is fairly automated. Which means that we are keeping data mirrored on a constant basis. Obviously we're not in your target market. But my suspicion is that as cloud technology becomes more broadly used you should expect variations on our approach to become more common. And therefore you should at least keep the needs of multiple data centers in your head.
- FooBarWidget 17y agoHow does it scale? After some reading it seems to be just a hosted database service with several vertical levels of scaling. What happens if Tera instance can't handle your load? Can FathomDB scale horizontally without sharding?
- justinsb 17y agoSorry - it's difficult to be totally clear in 6 minutes! We have two offerings ... a fully-managed MySQL database-as-a-service which lets you grow up to the biggest server on the cloud, and a new database which we've built ourselves, which we've announced but isn't publicly available yet. The scalable technology does horizontal scaling, and no - you don't have to shard. Of course, in some sense your data is sharded, because it is distributed across multiple machines, but we're not doing anything that you'd consider sharding in anything other than the most pedantic sense!
- btilly 17y agoConsistent, Available, and Partitionable on the fly. Choose any two. Each choice is justifiable for different applications. But you have to choose. Anyone who thinks they can make the choice for me and then tries to tell me that they have met all my possible data needs is selling snake oil.
- justinsb 17y agoI think you mean Partition Tolerance. I would ask you whether you've actually read the proof, but that question seems redundant.
- btilly 17y agoYou're right that I did mean partition tolerance. And yes, I've read the proof, but that was about a year and a half ago.
- jbooth 17y agoWhatever, it was obvious what you meant -- if someone has the choice of either figuring what you mean and addressing your question, or changing the subject with defensive pedantry, and chooses the latter, it may say something about their argument.
- justinsb 17y agoActually, in the distributed database field, 'partitionable on the fly' makes a lot of sense. I used this to infer the poster wasn't really familiar with the CAP theorem (which is only formally expressed in the proof), but it seems this wasn't fair in this case. The big problem with the CAP theorem is trying to decide what on earth C, A and P _really_ mean. Sounds like I need to do a blog post...
- moe 17y agoSo, ad hominem attacks aside, which tradeoff did you make for your product, or did you find a way to sidestep the problem altogether? I mean, you're announcing this thing like the 8th world wonder ("The end of NoSQL") and seem to have no problem comparing yourself to Oracle, no less. Do you have anything to show for it? Or At least a date when you will show us something?
- Semiapies 17y agoTacking "The end of NoSQL?" onto the title is karma-whoring via exploiting the silly database holy-warring we've seen on HN lately.
- jemfinch 17y agoAccusations of karma-whoring are karma-whoring ;)
- VBprogrammer 17y agoIt feels a little bit like the whole Erlang joke all over again.
- pmichaud 17y agoRecursion, I love it.
- GrandMasterBirt 17y agoHN has fixed that, can only vote up once. Damnit, genius!
- Semiapies 17y agoNo, it's a demand for people to stop douching up the site with trolling titles.
- stephenjudkins 17y agoThis sounds like a great, very valuable service. However, as far as I can tell, this service is not fundamentally different than running a MySQL server with replication. Nothing indicates it's "scalable" in the way that say, Cassandra, is. Replication is a perfectly adequate system for many people but it has its limits, especially for a write-heavy workload. If FathomDB is more sophisticated than a highly managed MySQL-as-a-service I would be very curious to know. That said, many of the complaints about scaling out MySQL stem from the painful management overhead of running a replicated setup, managing backups, etc. If FathomDB can vastly reduce the cost and difficulty of doing these things it may move MySQL a long ways towards addressing many of the reasons people are moving away from it. "The end of NoSQL" is just another piece of claptrap that I've seen recently on this and related topics. I would agree that "AltDB" might be a better term simply because it's less inflammatory and better expresses the goals of many of the diverse projects now lumped under "NoSQL".
- justinsb 17y agoWe need to work on our messaging! We're offering two different technologies: a fully managed MySQL as-a-service, and the new scalable database technology. But they're both relational databases in the service model. With the MySQL-as-a-service 'traditional' tech, we take care of the backups, monitoring etc. We'll offer fully managed replication in future. As you say, these basic steps will make running MySQL much more attractive. We've learned though, that there are still problems even here. You still have to think about how big your server should be. You still have to think about what happens when you outgrow the biggest server your cloud offers. The new technology does scaling across machines in the same way that Cassandra promises, or that Oracle's Exadata does for relational DBs. It lets you start on a shared server and grow seamlessly to the point where you're running across multiple servers. But it's still early days for that tech (after all, lots of people still think it's impossible, despite the fact that Oracle is happily selling it!), and so we're not opening it up publicly yet, whereas the standard hosted MySQL is publicly available.
- deleted 17y ago[deleted]
- davidmathers 17y agoIs this drizzle?
- justinsb 17y agoNo - the scalable database is our own technology. We looked at writing a MySQL/Drizzle storage engine, but there's a lot of pieces that are different that stretch up through the stack, and I was a bit concerned over the MySQL licensing now that it's part of Oracle. In future we could build a MySQL/Drizzle storage engine, and we'll probably run Drizzle-as-a-service once we start to see customer demand.
- rbanffy 17y agoYou know... Most of these nosql databases feel oddly nostalgic for me. It's the kind of stuff mainframe folks were using in the 70's with some clustering thrown in.
- rbranson 17y agoWhat? IMS?
- davidmathers 17y agoWhat? IMS? That would be XML. I'm guessing: http://en.wikipedia.org/wiki/CODASYL http://en.wikipedia.org/wiki/CODASYL
- deleted 17y ago[deleted]
- clintavo 17y agoA tad off topic but...Does anyone actually know how to get a FathomDB account? I thought, as a long time Rackspace customer and the promo code "RACK" they promote on their site, it would be relatively simple, I've tried to register three times over the past few months, but I never hear from a peep from FathomDB........