4 ms·

In graduate school, I was taught RDBMS & ACID by a Math Professor with a Math PhD, who later got a CS PhD, & personally interacted with Date & Codd. My key take

by dxbydt 15y ago

In graduate school, I was taught RDBMS & ACID by a Math Professor with a Math PhD, who later got a CS PhD, & personally interacted with Date & Codd. My key takeaways from that experience were: 1. There's a very stringent one-to-one mapping between RDBMS & relational logic. To put it another way, BCNF is not a matter of taste. Its a matter of adhering to the rules. If two dozen data architects are given the same many to many relation, and asked to take it through 1NF to 2NF to 3NF to BCNF and then stop, every single one of them will have the exact same answer. And this is because normalization is an extremely formal process => formal in the mathematical sense of the word. Two sets that have a given many-many relation will decompose to the exact same subsets because the mathematics of BCNF is unambiguous. This is incredibly powerful => Because there's a wrong answer! So if you hire a data architect & the guy gives you a BCNF & you crosscheck that BCNF with a more reliable architect, you are checking if the answers agree objectively. Its just straight math, not a matter of subjective taste as to which should be the primary key & which should be the foreign key & how the transtive dependence works out. 2. With objects in the mix, the relationship is already on creaky foundation. ORMs have an impedance mismatch because the relationships captured by notions of inheritance, polymorphisms, mixin traits - these don't have an exact mapping onto the set-theoretic notions captured by BCNF. Morover, there is a matter of subjective taste. You might design your UML differently from me & you might have valid reasons to do so. So while your BCNF exactly matches mine , your object model won't. 3. With kv stores, your relations are even more mathematically fuzzy. You are storing entire multi-valued objects and your rows have variable number of columns & at times you are dealing with completely denormalized structure. The math simply doesn't apply at these levels, and people tasked with populating a NOSQL structure will come up with wildly differing solutions that are completely unsound from a mathematical standpoint, but will nevertheless perform ( upto a certain threshold ) simply because machines have gotten faster & network latency issues aren't holding them back. I see more & more firms pick #3 because of trends( latest & greatest), lack of expertise to do #1 properly, or because of some false sense that scalability will bite them in the ass if they do #1 instead of #3, but primarily because #3 is EASY and #1 is hard. Its nice to not worry about normal forms & just have a multi-column row which grows & shrinks...but is it sound ? Am not sure.

oconnor0 15y ago
Are you arguing that RDBMS can handle all of the data, performance, and scalability requirements as well as a kv store while also in BCNF?
- dxbydt 15y ago
  There's no real argument cause these are orthogonal concerns. In general, you'll have a bunch of concerns. 1. BCNF 2. ACID 3. Performance 4. Scalability 5. Ease of implementation Before noSQL, you'd prioritize in that exact order. Post noSQL, the order has precisely reversed. There used to be pricey Oracle consultants ( and Sybase consultants & DB2 consultants & so on....data architects of all stripes). They'd come in, study your data in some depth & figure out which attributes were single-valued, which ones were multivalued, which bunch went into which tables, who the primary keys & foreign keys were, what the typical views would look like, what joins would that involve & could you do away with some of those joins....hell, even the typical interview questions for a fresh grad having absolutely nothing to do with databases would still involve Outer Join, Inner Join, difference between the two, here take the Employee spreadsheet and the Company spreadsheet and decompose into tables upto 3.5NF, list all the keys & tell me how many joins you'd need for a view that lists all employees in Accounts making over 70k between the age of 35 and 45 in Illinois. All of the above is pure relational logic. Its completely tech-agnostic. You don't even need an actual database to answer any of that. Nowadays, the questions are more along the lines of - How do you deploy MongoDB ? Great, you're hired, cause Mongo will handle the scalability & performance & if it doesn't we'll just add another node....imo, this is a vacuous line of thought.

by dxbydt 15y ago

oconnor0 15y ago

Are you arguing that RDBMS can handle all of the data, performance, and scalability requirements as well as a kv store while also in BCNF?

dxbydt 15y ago
There's no real argument cause these are orthogonal concerns. In general, you'll have a bunch of concerns. 1. BCNF 2. ACID 3. Performance 4. Scalability 5. Ease of implementation Before noSQL, you'd prioritize in that exact order. Post noSQL, the order has precisely reversed. There used to be pricey Oracle consultants ( and Sybase consultants & DB2 consultants & so on....data architects of all stripes). They'd come in, study your data in some depth & figure out which attributes were single-valued, which ones were multivalued, which bunch went into which tables, who the primary keys & foreign keys were, what the typical views would look like, what joins would that involve & could you do away with some of those joins....hell, even the typical interview questions for a fresh grad having absolutely nothing to do with databases would still involve Outer Join, Inner Join, difference between the two, here take the Employee spreadsheet and the Company spreadsheet and decompose into tables upto 3.5NF, list all the keys & tell me how many joins you'd need for a view that lists all employees in Accounts making over 70k between the age of 35 and 45 in Illinois. All of the above is pure relational logic. Its completely tech-agnostic. You don't even need an actual database to answer any of that. Nowadays, the questions are more along the lines of - How do you deploy MongoDB ? Great, you're hired, cause Mongo will handle the scalability & performance & if it doesn't we'll just add another node....imo, this is a vacuous line of thought.