2 ms·
Thanks for taking the time to respond in such detail! > I've read tons of documents, articles, and how-tos on schema-free non-RDBMS DBs like SimpleDB, CouchDB,
by janl 18y ago
Thanks for taking the time to respond in such detail!
> I've read tons of documents, articles, and how-tos on schema-free non-RDBMS DBs like SimpleDB, CouchDB, BigTable, DBSlayer etc. by now and I agree they seem very interesting and powerful. However, I just don't see how having each document (e.g. invoice) have it's own field structure would be a good thing. Every database I've ever worked on is structured and every new document has almost always the same data as every other document. That is not because of the limitations of the database but rather the nature of the business. Every invoice should have the same exact fields as every other invoice because employees are trained to fill and understand the implications of the data in each field. This has a lot more to do with business process efficiency and nothing to do with computers. RDBMS just facilitate that business rigidity more naturally. Of course, nobody's saying I can't do this in CouchDB but then why use CouchDB if I don't need the fluid document structure?
It is not useful if every document has its own structure :) But being able to model subtle differences like e.g. fax numbers in address records (basically that is a NULL-column in a relational table).
> Also, the biggest question I have is, how powerful are the dynamic views? I have tables with 10m rows and often serve 100+ select queries per second with joins/where/having/order/group-by/full-text-search functions. The business goal is not to make group-by queries but to summarize data in real-time as the analysts want e.g. sales by product per day or shipments by employee per week. How fast would a view that performs an equivalent function work on CouchDB? Can I optimize it somehow because I don't see much mention of index colums anywhere?
There are two kinds of views in CouchDB, temporary and permanent views. Temporary views should only be used in development on small amounts of data. Permanent views can do the exact same things temporary views can do, except their results are cached. For dynamic lookups you use the view key range that you can query in numerous ways as well as reduce that lets you do all sorts of computations. Something like average/total/max spending over a dynamic period of time (last week, last month, last year) can be generated from a single view in O(log n) time (where n is the number of documents). Concurrent reads happen non-blocking. I have seen a couple of hundred req/s on a view running on a dated MacBook.
> What bugs me most is that when dealing with databases like CouchDB and SimpleDB, the "best practice" advice I hear is to just write summary data and search indexes on my own. That is every time a new invoice is generated, add +1 to the summary.count.invoice record, add +300 to the balance.customer.acme-corp record and -5 to inventory.widget record. And don't forget to add every unique word in the document to the search index table. While that would be great if I knew the exact business needs ahead of the time but in reality, first you capture all the data you can capture and then generate reports based on the needs of the user.
I can't speak for SimpleDB, but in CouchDB the housekeeping is done for you automatically if you created a view for all your stats & things. You can even notify external services (fulltext search) to index new or changed data).