16 ms·

Well structured relational databases are an awesome tool. I have built and used hundreds. But they are a bit of cargo cult architecture. Deciding to use one

Well structured relational databases are an awesome tool. I have built and used hundreds. But they are a bit of cargo cult architecture. Deciding to use one means adding lots of code. It adds a lot of complexity around testing. All of it multiplies. It's another big Artifact to maintain and you may have to decide how you will scale it, replicate it, make it highly-available, etc. NOSQL was the first crack in the seemingly unassailable assumption that you need to use a DB. Now I think a few of us are questioning whether to use a datastore at all. After all, once you've dropped SQL for some functions, why stuff them into yet another opaque datastore? You might end up having to use a relational database anyway at some point and then you're stuck with another piece of junk and all its conceptual overhead and extra code. Start with files and introduce a relational database only when necessary. Formats are straight forward these days. Just write out JSON, XML, HTML5 w/ microformats or even Yaml. The art is in the "schema" or how you store the files. Choose wisely with some knowledge of your requirements and it seems like it can work well. Files on disk mean you can use all your great UNIX tooling to do all sorts of complex operations that would take you many many hours of skilled development to do with a database. Then there are all the possible things you could do with Git or (imagine!) ZFS! Versioning everything for free? Keep an activity log within a hierarchy. What kind of cool stuff can you do with that? I aim to find out.

ucee054 13y ago
Formats are straight forward these days. Just write out JSON, XML, HTML5 w/ microformats or even Yaml. And when my data grows to multiple terabytes in size?
jacques_chester 13y ago
Your argument makes perfect sense. If all you need is storage and retrieval. Take storing project management data as JSON files. Suddenly the boss bursts in: he needs to know, right now, how many hours Jeremy has spent working on these three projects. Given that your JSON hierarchy runs project->person and not person->project, you will now have to traverse your entire database to give an answer. The deep magic of relational algebra or relational calculus is that they privilege no one view of the data over the other. This is, incidentally, one of the causes of OO/relational mismatch. A class hierarchy locks in a particular view of the problem domain. Relational has many potential views of the problem domain. The cost and inconvenience of traversing trees for ad hoc queries is one of the big reasons that RDBMSes swept away network and hierarchical databases.

ucee054 13y ago

Formats are straight forward these days. Just write out JSON, XML, HTML5 w/ microformats or even Yaml. And when my data grows to multiple terabytes in size?

jacques_chester 13y ago

Your argument makes perfect sense. If all you need is storage and retrieval. Take storing project management data as JSON files. Suddenly the boss bursts in: he needs to know, right now, how many hours Jeremy has spent working on these three projects. Given that your JSON hierarchy runs project->person and not person->project, you will now have to traverse your entire database to give an answer. The deep magic of relational algebra or relational calculus is that they privilege no one view of the data over the other. This is, incidentally, one of the causes of OO/relational mismatch. A class hierarchy locks in a particular view of the problem domain. Relational has many potential views of the problem domain. The cost and inconvenience of traversing trees for ad hoc queries is one of the big reasons that RDBMSes swept away network and hierarchical databases.