4 ms·

I think most are pragmatic and admit you may have to do denormalise for performance (e.g. I've read this in C.J. Date's books). I interpret their point as the n

by kpmah 10y ago

I think most are pragmatic and admit you may have to do denormalise for performance (e.g. I've read this in C.J. Date's books). I interpret their point as the need for denormalisation is not a flaw with the relational model, but a flaw in its implementation. The relational model does not specify an on-disk format.

calpaterson 10y ago
So rare to actually have to denormalise for performance today though. Query planners are miles ahead of where they were in the 80s and 90s. SSDs delivered an order of magnitude boost to performance. Rapidly expanding memories hugely increase the how much working set you can hold in memory. No one seems to talk about this but the "Moore's law has ended" meme does not apply to most database scenarios. Database servers are normally not CPU bound: they are bound by available memory and disk bandwidth and these are both still increasing.
- anonymous_fun 10y ago
  Antidotally, I started at a co 6 years ago where 100's of database servers were all spinning disks. It was common to have to trace down bad queries causing performance issues. Once the servers were upgraded with more ram and SSD's, the need to micromanage performance issues disappeared.
  - sgeneris 10y ago
    The sheer fact that much of the comments are about physical storage is an excellent validation of the article's claim that data professionals don't know and understand data fundamentals and the RDM.
Nokinside 10y ago
I think today, most often than not, the fault is between the keyboard and the chair. There is clear lack of domain knowledge. RDBMS's like PostgreSQL, Oracle, DB2, etc. can do much more than people expect them to do. Relational databases are relatively easy for anyone to start working with, but optimizing and tuning queries requires fair amount of domain knowledge. You'd be surprised to learn how little denormalization big financial company databasees have. They have people who are experts that do only databases. For I/O intensive work you may need transaction rows where data written together is stored together, but it's not denormalization, it's optimizing for I/O bottleneck. Query planners can do most of the work, if the person crafting queries has understanding of how they work. Carefully grafted queries and tuning memory layout for tables correctly is usually the right way to fix performance problems.
- sgeneris 10y ago
  Yes, I would be surprised. Because, unlike most products which only require training to work with, true RDBMSs require knowledge of data and relational fundamentals and such is scarce today, because education has been replaced by sheer training. How many database "experts" do you know today who have any background in logic and math, particularly those designing DBMSs? I've watched them for almost 50 years and they've become worse and worse, not better.

by kpmah 10y ago

calpaterson 10y ago

So rare to actually have to denormalise for performance today though. Query planners are miles ahead of where they were in the 80s and 90s. SSDs delivered an order of magnitude boost to performance. Rapidly expanding memories hugely increase the how much working set you can hold in memory. No one seems to talk about this but the "Moore's law has ended" meme does not apply to most database scenarios. Database servers are normally not CPU bound: they are bound by available memory and disk bandwidth and these are both still increasing.

anonymous_fun 10y ago
Antidotally, I started at a co 6 years ago where 100's of database servers were all spinning disks. It was common to have to trace down bad queries causing performance issues. Once the servers were upgraded with more ram and SSD's, the need to micromanage performance issues disappeared.
- sgeneris 10y ago
  The sheer fact that much of the comments are about physical storage is an excellent validation of the article's claim that data professionals don't know and understand data fundamentals and the RDM.

Nokinside 10y ago

I think today, most often than not, the fault is between the keyboard and the chair. There is clear lack of domain knowledge. RDBMS's like PostgreSQL, Oracle, DB2, etc. can do much more than people expect them to do. Relational databases are relatively easy for anyone to start working with, but optimizing and tuning queries requires fair amount of domain knowledge. You'd be surprised to learn how little denormalization big financial company databasees have. They have people who are experts that do only databases. For I/O intensive work you may need transaction rows where data written together is stored together, but it's not denormalization, it's optimizing for I/O bottleneck. Query planners can do most of the work, if the person crafting queries has understanding of how they work. Carefully grafted queries and tuning memory layout for tables correctly is usually the right way to fix performance problems.

sgeneris 10y ago
Yes, I would be surprised. Because, unlike most products which only require training to work with, true RDBMSs require knowledge of data and relational fundamentals and such is scarce today, because education has been replaced by sheer training. How many database "experts" do you know today who have any background in logic and math, particularly those designing DBMSs? I've watched them for almost 50 years and they've become worse and worse, not better.