6 ms·

A Relational Model of Data for Large Shared Data Banks (1970) [pdf]

EngineerBetter 8y ago
The key, the whole key, and nothing but the key.
- mtVessel 8y ago
  So help me, Codd.
DanielBMarkham 8y ago
This changed the way people coded, stored data, and thought about data in general. In my mind, the implications, not necessarily the details, were some of the most profound in the history of computational papers. For the last few years, I've been working with the premise that there is an analogous information-minimization theory for all project information, not just data storage. So far I'm optimistic that this is true --- it's just not the same minimal format for different teams working on the same problem. It's people-dependent, not data-dependent. (Obligatory link: https://leanpub.com/info-ops https://leanpub.com/info-ops ) If true, and it looks like it is, it means that teams can have "analysis compilers" that work inside the build pipeline alongside all the other tools. This leads to "continuous information deployment"
- kthejoker2 8y ago
  As someone who serves as a product owner for analytics use cases, I can attest that the hardest problem in Agile is accurate qualitative and quantitative measurement of progress, value, and delivery. Definitely going to check out the book.
  - DanielBMarkham 8y ago
    To make it work, I had to propose a new working definition of what analysis is. The rough definition for most people is that analysis is all the "work around the work" If you're not coding you're doing analysis. I needed something more concrete than that, yet flexible enough to allow tractability on all the kinds of things teams do. Once I got that definition in place, the rest of the process of information minimization proceeded quite naturally. At that point you know what it is you're trying to minimize, ie, you have a "key" in Codd's terms. Then the trick is just minimize information and maximizing flow using key-based storage and transmission. (This also lets you look at whether your processes are working or not from a new, rational, and objective standpoint, which is a nice side-effect)
ZenPsycho 8y ago
relational databases are secretly supposed to be logic programming languages like prolog, each “table”, (relation) representing a predicate function. each row (tuple) representing a fact about the world that satisfies the predicate. with proper structure and labelling you can ask very sophisticated questions. somewhere along the line people forgot this and typically just dump spreadsheets in, to satisfy the fields in a form or an app screen, then think they need a graph database to represent relationships. it is unfortunate that SQL based systems became accidentally synonymous with the “relational model” as it actually mostly gets in the way of implementing the relational model. oh well.
- swingline-747 8y ago
  No, that's an misinformed, all-or-nothing, gross-mischaracterization. Data management systems are a toolbox. The core tenent, distilling ACID, of RDBMSes is referential integrity: that what a foreign key points to can't simply disappear and leave a dangling reference. If you have a database of customers and their orders, you want orders destroyed with that customer, and the referring customer can't simply vanish. There are several delete dependent policies, depending on the use-case. Key-Value stores (NoSQL) typically make no such guarantees. KVSes and Document store are fantasic for less critical and more nondeterministic data like user profiles, likes and tags... things that don't matter as much, as say, bank accounts transactions. Only a fool would use eventual consistency where it could be gamed by making maximum simulateneous withdrawls of say a $1000 at multiple locations. This is where atomicity and consistency are very important. Use the right underlying data guarantees for the task at hand... not like a consulting company whom tried to turn a DBMS into a NoSQL by having a single database table store everything. IIRC the columns looked like this: Key | Value | Type | Notes Good luck indexing that pile of slow, and say hello to full-table scans for nearly every fetch.
  - michaelmior 8y ago
    > more nondeterministic data What does that mean?
  - ZenPsycho 8y ago
    I am not entirely sure what this is in response to. did you hit reply on the wrong comment perchance?
JunkDNA 8y ago
If you’ve never read this paper, you owe it to yourself to give it a thorough read. It truly gives you a sense for the problems relational databases were attempting to solve with the tech of the day. Also note the strong mathematical underpinnings of many of the core ideas. It’s no wonder these ideas have withstood the test of time.
- mathattack 8y ago
  I recall the first training class I had where I connected relational tables with the theory I learned in school. Life never was the same after. On Codd we trust.
- gnufx 8y ago
  I've found it's generally worthwhile to read the original literature; especially so when it was all worked out, but the technology didn't support doing it properly and people have persisted with approximations and fudges needlessly (like some areas of science). The standard of writing may be better in the '60s and '70s literature than in recent stuff anyway.
redwood 8y ago
Amazing that something optimized for when storage was the bottleneck and shockingly expensive compared to people... is still the most popular model today (when people are the expensive and rare resource, with storage abundant and cheap).
lifepillar 8y ago
It’s a shame that no one in academia has bothered so far to curate a critical, annotated, edition of Codd’s collected works. His impact was immense (for good and for bad, e.g., nulls in SQL) and he foresaw many of the important issues of data management in the years to come (up to nowadays), including the relevance of distributed systems and the need for different ways of looking at data, as sets, as tables, as predicates, as graphs. Unfortunately, people who knew or worked with him have died or are getting old, so I’m losing hope…
commandlinefan 8y ago
It's nearly 50 years old and still the best model for storing and querying data. YesSQL FTW.

The key, the whole key, and nothing but the key.

mtVessel 8y ago
So help me, Codd.

This changed the way people coded, stored data, and thought about data in general. In my mind, the implications, not necessarily the details, were some of the most profound in the history of computational papers. For the last few years, I've been working with the premise that there is an analogous information-minimization theory for all project information, not just data storage. So far I'm optimistic that this is true --- it's just not the same minimal format for different teams working on the same problem. It's people-dependent, not data-dependent. (Obligatory link: https://leanpub.com/info-ops https://leanpub.com/info-ops ) If true, and it looks like it is, it means that teams can have "analysis compilers" that work inside the build pipeline alongside all the other tools. This leads to "continuous information deployment"

kthejoker2 8y ago
As someone who serves as a product owner for analytics use cases, I can attest that the hardest problem in Agile is accurate qualitative and quantitative measurement of progress, value, and delivery. Definitely going to check out the book.
- DanielBMarkham 8y ago
  To make it work, I had to propose a new working definition of what analysis is. The rough definition for most people is that analysis is all the "work around the work" If you're not coding you're doing analysis. I needed something more concrete than that, yet flexible enough to allow tractability on all the kinds of things teams do. Once I got that definition in place, the rest of the process of information minimization proceeded quite naturally. At that point you know what it is you're trying to minimize, ie, you have a "key" in Codd's terms. Then the trick is just minimize information and maximizing flow using key-based storage and transmission. (This also lets you look at whether your processes are working or not from a new, rational, and objective standpoint, which is a nice side-effect)

ZenPsycho 8y ago

relational databases are secretly supposed to be logic programming languages like prolog, each “table”, (relation) representing a predicate function. each row (tuple) representing a fact about the world that satisfies the predicate. with proper structure and labelling you can ask very sophisticated questions. somewhere along the line people forgot this and typically just dump spreadsheets in, to satisfy the fields in a form or an app screen, then think they need a graph database to represent relationships. it is unfortunate that SQL based systems became accidentally synonymous with the “relational model” as it actually mostly gets in the way of implementing the relational model. oh well.

swingline-747 8y ago
No, that's an misinformed, all-or-nothing, gross-mischaracterization. Data management systems are a toolbox. The core tenent, distilling ACID, of RDBMSes is referential integrity: that what a foreign key points to can't simply disappear and leave a dangling reference. If you have a database of customers and their orders, you want orders destroyed with that customer, and the referring customer can't simply vanish. There are several delete dependent policies, depending on the use-case. Key-Value stores (NoSQL) typically make no such guarantees. KVSes and Document store are fantasic for less critical and more nondeterministic data like user profiles, likes and tags... things that don't matter as much, as say, bank accounts transactions. Only a fool would use eventual consistency where it could be gamed by making maximum simulateneous withdrawls of say a $1000 at multiple locations. This is where atomicity and consistency are very important. Use the right underlying data guarantees for the task at hand... not like a consulting company whom tried to turn a DBMS into a NoSQL by having a single database table store everything. IIRC the columns looked like this: Key | Value | Type | Notes Good luck indexing that pile of slow, and say hello to full-table scans for nearly every fetch.
- michaelmior 8y ago
  > more nondeterministic data What does that mean?
- ZenPsycho 8y ago
  I am not entirely sure what this is in response to. did you hit reply on the wrong comment perchance?

JunkDNA 8y ago

If you’ve never read this paper, you owe it to yourself to give it a thorough read. It truly gives you a sense for the problems relational databases were attempting to solve with the tech of the day. Also note the strong mathematical underpinnings of many of the core ideas. It’s no wonder these ideas have withstood the test of time.

mathattack 8y ago
I recall the first training class I had where I connected relational tables with the theory I learned in school. Life never was the same after. On Codd we trust.
gnufx 8y ago
I've found it's generally worthwhile to read the original literature; especially so when it was all worked out, but the technology didn't support doing it properly and people have persisted with approximations and fudges needlessly (like some areas of science). The standard of writing may be better in the '60s and '70s literature than in recent stuff anyway.

redwood 8y ago

Amazing that something optimized for when storage was the bottleneck and shockingly expensive compared to people... is still the most popular model today (when people are the expensive and rare resource, with storage abundant and cheap).

lifepillar 8y ago

It’s a shame that no one in academia has bothered so far to curate a critical, annotated, edition of Codd’s collected works. His impact was immense (for good and for bad, e.g., nulls in SQL) and he foresaw many of the important issues of data management in the years to come (up to nowadays), including the relevance of distributed systems and the need for different ways of looking at data, as sets, as tables, as predicates, as graphs. Unfortunately, people who knew or worked with him have died or are getting old, so I’m losing hope…

commandlinefan 8y ago

It's nearly 50 years old and still the best model for storing and querying data. YesSQL FTW.