6 ms·

I had great difficulty in controlling my mirth while I read the self-congratulatory article "Database Systems: Achievements and Opportunities" in the October, 1

by edw519 17y ago

I had great difficulty in controlling my mirth while I read the self-congratulatory article "Database Systems: Achievements and Opportunities" in the October, 1991, issue of the Communications,... I had little difficult in controlling my mirth when I realized that in 18 years, some things haven't changed. Linkbait, drama, and trolling all still look the same. As a designer of commercial manufacturing applications on IBM mainframes in the late 1960's and early 1970's... If you're going to wave your resume, make sure it's "wavable". If I had had a hand as a designer of commercial manufacturing applications on IBM mainframes in the late 1960's and early 1970's, I sure wouldn't brag about it. They were an excellent example of what not to do: so expensive, so difficult to deploy and use, and so ineffective, that the whole world rushed out to write better apps on mini-computers, and eventually, PCs. Ironically, the one thing they did do well was their relational database storage systems. If you owned a multi-million dollar AMAPS installation in 1978, the COBOL apps were soon worthless. The only thing salvageable was the TOTAL DBMS. I can categorically state that relational databases set the commercial data processing industry back at least ten years and wasted many of the billions of dollars that were spent on data processing. This conclusion is based upon what data? Maybe in 1991, you could bullshit the ACM without supporting data, but 2009 readers demand citings. Wikis & google have exposed the posers. Unfortunately, relational databases performed a task that didn't need doing; e.g., these databases were orders of magnitude slower than the "flat files" they replaced, Again, based upon what data? From what planet? Just because someone overnormalized a commercial database doesn't make it the fault of the underlying technology. That would be like saying, "That webpage sucks, therefore HTML sucks." Why were relational databases such a Procrustean bed? Because organizations, budgets, products, etc., are hierarchical... Organizations, budgets, products, etc. are data sources and sinks and can be structured any number of ways, including hierarchical. But the lifeblood of any business is its order flow and business processes which are almost always ideally suited to be relational; they're "linked" to almost everything else. Not everything has to be in 4th normal form, but flat files and hirerarchical data bases are almost always a poor stepchild to RDBMS for business flow. These databases could also respond quickly to "real-time" requests for information, because the data was readily accessible through pointers and hash tables--without performing "joins". I guess it's not really fair to "debate" with an OP from a generation ago. Even with Moore's Law, he would have had a hard time wrapping his head around where the real bottlenecks would be today. But one thing really hasn't changed that much: throughput has rarely been on the critical path. Why sacrifice data integrity, adherance to business rules, and effective delivery of user needs for a few microseconds? I remember routinely witnessing subsecond intercontinental response time on massive relational database installations as early as 1981. Why didn't OP? Oh, and RDBMS with pointers and hash tables have been around since 1965: http://en.wikipedia.org/wiki/Pick_operating_system http://en.wikipedia.org/wiki/Pick_operating_system and now also support object oriented technology: http://en.wikipedia.org/wiki/InterSystems_Cach%C3%A9 http://en.wikipedia.org/wiki/InterSystems_Cach%C3%A9 I shudder to think about the large number of man-years that were devoted during the 1970's and 1980's to "optimizing" relational databases to the point where they could remotely compete in the marketplace. I shudder to think about the large number of man-years lost by PHBs who read drivel like this and waste the time of people who do real work with initiatives derived from these drama-based conclusions. Database research has produced a number of good results, but the relational database is not one of them. From someone who has built a career rescuing so many manufacturers and distributors from flat file systems with relational database technology to whoever posted this: thanks for the laugh. I really needed it on a tough Monday.

maukdaddy 17y ago
PhDs are funny. Let's go shopping!
giardini 17y ago
Arguably not a single statement that you have made in your overly hasty post is true. To address a few: Henry Baker is no troll. His letter was written to the ACM in 1991. Baker was an established and respected computer scientist and iconoclast. To treat the letter as if it were written yesterday is foolish and disrespectful, In any case, had Baker's remarks been misguided or irrelevant then the ACM would not have published them. Baker was always in demand. No need to compare resumes: you likely would not rank well against him, "Flat files" were used often for batch processing. For sequential processing and merging/splitting/sorting of data they are usually faster than RDBMS. During that processing, temporary files might be created that are indexed, hashed, or sequential. Today network-style databases remain faster by at least an order of magnitude (and sometimes two) than relational databases. Ask any knowledgeable mainframer about the relative speeds of their network-style vs relational databases. The same holds for smaller computers too. Baker would have no problem "wrapping his head around" almost any problem. An archive of some of his research papers: http://home.pipeline.com/~hbaker1/ http://home.pipeline.com/~hbaker1/ You saw "subsecond intercontinental response time on massive relational database installations as early as 1981", when IIRC the only relational databases available was Oracle and possibly Ingres. My memories are that Oracle's early versions were performance dogs by any standard. But perhaps you can refresh our memories by citing performance statistics and showing us benchmarks. PICK was not a relational database. TOTAL was not a relational database. Your remarks remind me of how so often when an excellent programmer leaves a shop, the remaining developers diss his work and argue for rewriting all of his (excellent working) code. But to miss on almost every statement? Oh, my!
- chipsy 17y ago
  What bugs me is that even though Baker states in the comment that he proved a particular deficiency in the relational algebra(the expression of transitive closures), the assertions he uses to lead into this seem extremely vague: they(relational databases) made trivial problems obviously trivial, but did nothing to solve the really hard data processing problems... organizations, budgets, products, etc., are hierarchical; hierarchies require transitive closures for their "explosions"... He never describes concretely what an "explosion" is in a real situation - not in the comment, at least. I'll give the benefit of the doubt that perhaps he brings it up in some paper. But my understanding is that the relational algebra is useful primarily because of its linguistic properties - instead of every problem needing a customized data model and customized graph-traversal solution, it describes how to use a generic query syntax and joins everywhere. While this limits the theoretical power of the system and introduces order-of-magnitude slowdowns, it still solves a major subset of business problems, and it does it with a more compact description than the equivalent graph solution. And that, in turn, means that the theoretical limitations aren't necessarily going to matter for shipping and maintaining a solution, especially when working with a short-term view and small scale. Essentially, he banks on his credentials to turn an "academic" oriented statement into a "industry" one. But from the 70s onward he was writing papers on garbage collection, concurrency and PL theory, not going out and building systems in the industry. So my view is that he's being overly biased towards theoretical purity above practicality, a tendency that has led many an academic astray. (This is not a slam to academics and their pursuit of purity, though. It's a great way to advance the computing arts, and complements well the messy practicality of hackers.)

by edw519 17y ago

maukdaddy 17y ago

PhDs are funny. Let's go shopping!

giardini 17y ago

Arguably not a single statement that you have made in your overly hasty post is true. To address a few: Henry Baker is no troll. His letter was written to the ACM in 1991. Baker was an established and respected computer scientist and iconoclast. To treat the letter as if it were written yesterday is foolish and disrespectful, In any case, had Baker's remarks been misguided or irrelevant then the ACM would not have published them. Baker was always in demand. No need to compare resumes: you likely would not rank well against him, "Flat files" were used often for batch processing. For sequential processing and merging/splitting/sorting of data they are usually faster than RDBMS. During that processing, temporary files might be created that are indexed, hashed, or sequential. Today network-style databases remain faster by at least an order of magnitude (and sometimes two) than relational databases. Ask any knowledgeable mainframer about the relative speeds of their network-style vs relational databases. The same holds for smaller computers too. Baker would have no problem "wrapping his head around" almost any problem. An archive of some of his research papers: http://home.pipeline.com/~hbaker1/ http://home.pipeline.com/~hbaker1/ You saw "subsecond intercontinental response time on massive relational database installations as early as 1981", when IIRC the only relational databases available was Oracle and possibly Ingres. My memories are that Oracle's early versions were performance dogs by any standard. But perhaps you can refresh our memories by citing performance statistics and showing us benchmarks. PICK was not a relational database. TOTAL was not a relational database. Your remarks remind me of how so often when an excellent programmer leaves a shop, the remaining developers diss his work and argue for rewriting all of his (excellent working) code. But to miss on almost every statement? Oh, my!

chipsy 17y ago
What bugs me is that even though Baker states in the comment that he proved a particular deficiency in the relational algebra(the expression of transitive closures), the assertions he uses to lead into this seem extremely vague: they(relational databases) made trivial problems obviously trivial, but did nothing to solve the really hard data processing problems... organizations, budgets, products, etc., are hierarchical; hierarchies require transitive closures for their "explosions"... He never describes concretely what an "explosion" is in a real situation - not in the comment, at least. I'll give the benefit of the doubt that perhaps he brings it up in some paper. But my understanding is that the relational algebra is useful primarily because of its linguistic properties - instead of every problem needing a customized data model and customized graph-traversal solution, it describes how to use a generic query syntax and joins everywhere. While this limits the theoretical power of the system and introduces order-of-magnitude slowdowns, it still solves a major subset of business problems, and it does it with a more compact description than the equivalent graph solution. And that, in turn, means that the theoretical limitations aren't necessarily going to matter for shipping and maintaining a solution, especially when working with a short-term view and small scale. Essentially, he banks on his credentials to turn an "academic" oriented statement into a "industry" one. But from the 70s onward he was writing papers on garbage collection, concurrency and PL theory, not going out and building systems in the industry. So my view is that he's being overly biased towards theoretical purity above practicality, a tendency that has led many an academic astray. (This is not a slam to academics and their pursuit of purity, though. It's a great way to advance the computing arts, and complements well the messy practicality of hackers.)