3 ms·
Have your read Kimball's Data Warehouse Toolkit? I'm partial to the first edition (1996). It's mostly conceptual stuff: design rationale, OLTP vs OLAP, fact vs
by pradocchia 17y ago
Have your read Kimball's Data Warehouse Toolkit? I'm partial to the first edition (1996). It's mostly conceptual stuff: design rationale, OLTP vs OLAP, fact vs dimension, type II SCDs (very important), surrogate keys, aggregate navigation etc.
Most code in a data warehouse is ETL code, and much of that is obscured behind the interface of big-name tools like Informatica. I don't think you are interested in that. People who roll their own have typically rolled with Perl. Try a search on Perl + ETL and you might find something.
Other items of interest:
* Pentaho, an open-source BI suite. Includes ETL, reporting and multidimensional cubes.
* Column-oriented data stores like Vertica, Sybase IQ, MonetDB, and Infobright. MonetDB and Infobright are both open source. Infobright integrates with MySQL, so that might be up your alley.
* SSDs may or may not turn data warehousing on its head. Yet to be determined.
* schema-driven ETL is very powerful: define an input view/function, a target table, a few naming conventions, and generate your ETL code directly off the system tables--ETL with opinions, as it were.
EDIT:
For online resources, I used read intelligententerprise.com. Can't vouch for its current quality. Back then it was hit or miss, and promotional items frequently overshadowed substantive material.
- 3pt14159 17y agoThanks a ton. I'll make sure I pick up Kimball's Data Warehouse Toolkit and check out intelligententerprise.com. The four that pop out most from your post are Pentaho, MonetDB, Infobright, and schema-driven ETL, so I'll start my search there. Thanks once again for the help!
- pradocchia 17y agoHow has it gone? Have you found any helpful sources? I have since learned that LucidDB may be more appropriate for DW-style workloads than MonetDB. There was a LucidDB thread on HN recently that had more details.