4 ms·

An intriguing topic, nice try, but the author failed to explain the reasons why programmers hate databases. It's definitely not because we don't know the relati

by mojuba 19y ago

An intriguing topic, nice try, but the author failed to explain the reasons why programmers hate databases. It's definitely not because we don't know the relational theory (I beg your pardon, but screw relational theory, it's as ugly as DBs themselves). One problem I can think of is that I'm forced to mix two languages in one story (a program, that is). It's like writing an essay in English over-peppered with a lot of Latin phrases and sayings. In web development it gets even worse: as if my favorite programming language X plus SQL wasn't enough, I mix up HTML, CSS and JavaScript - that's at least 5 languages in total! Another problem is that from a programmer's perspective databases are, put simply, arrays possibly with pointers. The only reason we can't declare these structures as arrays/lists with pointers is that data can grow infinitely big and my favorite language X can't handle that. In other words, using databases is a classical example of premature optimization that comes at a price of overcomplicated code. Yes, databases are plain arrays and even the best relational advocate can't prove the opposite (or I'd be glad to hear arguments actually). Upd: persistence and a possibility to share data between processes is another reason we use databases, but I'm sure there are more elegant ways of doing this.

jimm 19y ago
I agree with some of your points, but you need to study data structures a bit more. Just because a simple table structure could be implemented as an array doesn't mean that database tables are equivalent to arrays. A primary key is not the same as array index: it can be a type other than integer, and it can be non-consecutive. Indexing columns (including but not limited to the primary key) provides searches that happen in less than linear time. You can't do that with a plain array. Another thing database give you is ACID storage. Sure, you could reproduce some of that using thread-level locking or some other locking mechanism, but then what you have isn't a simple array any more.
- mojuba 19y ago
  You can't do that with a plain array I can do that with hash arrays. The primary key is my primary hash, other keys are just extra hash maps pointing to the main array. I understand, perhaps none of the existing languages can fully replace the DB functionality, but I suspect nobody is even thinking about that because - hey, there are plenty of nice DBMSs around, why bother reinventing them? but then what you have isn't a simple array any more True, but again, it's a matter of having an infrastructure that handles sharing for you. Windows Registry, however awkward and Microsoft'ish, does just that, as as example.
  - nostrademons 19y ago
    "I can do that with hash arrays. The primary key is my primary hash, other keys are just extra hash maps pointing to the main array." A.k.a an in-memory table with hash indexes. MySQL has them. ;-) You're still missing out on a lot of functionality (even besides transactions/persistence/concurrency). For example, what if you want all records where the creation date is between a certain range of days? Or how about the maximum value of a certain column? How about joins? A conventional database can retrieve these from the indexes, but your multiple-hashes system would require a full table scan.
    - mojuba 19y ago
      All you just said is map/reduce, possibly optimized for large data chunks internally. In fact if we create a proper infrastructure, "indexing" can be done automatically and transparently. And I never said arrays should be in memory entirely, even though I declare them as arrays in my language.
      - brlewis 19y ago
        SQL indexes work transparently. Creating them automatically would not be good, since they have a cost for write operations.
Kaizyn 19y ago
The relational model is simply the application of set theory and predicate logic to data management. Are you saying that these two branches of math/logic are ugly or that you don't understand them well enough to have a decent opinion?
olavk 19y ago
Databases are not arrays, although they may be implemented as arrays. The whole point of the relational model is that the logical model is abstracted from the implementation. So if e.g. the DBA decides that it would be more efficient to store a table as an array of columns rather than an array of rows, he can do that without changing the logical model.
nostrademons 19y ago
I don't understand the resistance to using multiple languages in one project. Every time you use an API, you're effectively using a new language. The syntax may be familiar, but the vocabulary is completely different. That's the point - through libraries and DSLs, we extend the capabilities of the base language so it's more suitable to the problem domain. Take a look at the following SQLAlchemy code: select([users.c.first_name + ' ' + users.c.last_name, users.c.email, games.c.title], from_obj=[users.join(games, users.c.user_id == games.c.creator_id], and_(games.c.creation_date == datetime.datetime.now() - datetime.timedelta(3), or_(users.c.email.like('%@aol.com'), users.c.email.like('%@hotmail.com')))) Can you really say that's easier to read than this? SELECT users.first_name + ' ' + users.last_name, users.email, games.title FROM users INNER JOIN games ON (users.user_id = games.game_id) WHERE games.creation_date >= DATE_SUB(NOW(), INTERVAL 3 DAY) AND (users.email LIKE '%@aol.com' OR users.email LIKE '%@hotmail.com') SQLAlchemy is a really cool library - it basically lets you write all your SQL queries in 100% Pure Python. But I doubt I'd use it on my own projects, because I've learned to be suspicious of libraries whose only purpose is to make some other tool look like a language I already know. Invariably there are reasons why the original language looked the way it did, and the reimplementation as a library becomes incredibly clunky as things get more complicated. I find I'm better off learning the original language instead.
- olavk 19y ago
  My theory is that there is two kinds of people. Those that like everything to be in the same language and environment, and those that like to combine lots of different languages where each language is optimized for a specific task.
  - nostrademons 19y ago
    Could be. I'm reminded of the two competing camps for large-scale systems design. The UNIX philosophy is "small pieces loosely joined": each component does one thing well, and the system just provides a common metaphor and architecture for them to work together (eg. pipes and line-oriented text). The Windows philosophy is "big chunks tightly coupled": everything just works out of the box, but if you need to do something the designers didn't think of, you're out of luck. Kinda like "I'll do it myself; show me how" vs. "Do it for me." Nearly every computing system can be placed somewhere along this continuum. In operating systems, we've got MacOS (pre-X) and Windows on one side and UNIX on the other. In languages, we've got Scheme and Smalltalk on the UNIX side and Java and .NET on the other (ironically, Scheme and Smalltalk have a "small pieces loosely coupled" architecture internally, but are big monolithic chunks when it comes to the outside world). For IDEs, Emacs is all the way over on the UNIX side, vim not far behind, Eclipse is in the middle, and IntelliJ and Visual Studio all the way on the Windows side. For JavaScript libraries, YUI/Mootools/Prototype are all on the Windows side and JQuery is on the UNIX side, though it's moving closer to the Windows philosophy. Among web frameworks, Rails and Django are on the Windows side with Pylons on the UNIX side. The web itself is based on the UNIX philosophy, but many of the largest websites follow the Windows philosophy. I'm not so certain that people fit rigidly into one category though. Up until about 3-4 years ago, I was firmly in the Windows camp: I did Java programming only, I did it on a Windows machine, I used JBuilder and Eclipse to do it. Then I started branching out, mostly because I started learning about all this stuff that Java wouldn't let me get at. I think many people may follow a similar path: "do it for me" until you want to do something that the designers didn't think of, and then you have no choice to move to a "do it yourself; we'll show you how" environment.
bootload 19y ago
"... One problem I can think of is that I'm forced to mix two languages in one story .." Do you mean embedded? that I agree with. Embedded sucks but there are lots of techniques (abstraction) you can use to avoid them. Unavoidable is writing web based applications without using several languages. Though I can think of some lisp apps that do not (news). Do you lisp?
- mojuba 19y ago
  No, I don't lisp, but I know you can nicely embed HTML directly into your lisp code with macros. I saw an example here at YCnews earlier and was struck by it, that was beautiful. No doubt, you can do that with SQL too, although that's just a cosmetic change. Even if your code looks lispish, you still think in DB terms - tables, relations.
gregjor 19y ago
I'm the author of the original article, and I admit that I failed to explain all of the reasons programmers don't like databases -- you came up with one I didn't list (don't want to use a lot of languages) and demonstrated one of the reasons I did list: Who Needs Theory? Your analogy with writing an essay is off target. If writing a program is like writing an essay in English, the SQL/database part of the program are like mathematical formulas in the essay. You could use English instead of mathematical notation but it would be far uglier, less precise, and more prone to error and misunderstanding. In a typical web application (what I've been doing for over ten years) the database part is usually the only part that can be proved correct. I can't help you overcome your misconceptions about relational theory or relational databases, or do anything about your opinion that relational theory is ugly. But you are just wrong arguing that relational databases are glorified arrays full of pointers and examples of premature optimization. The "best relational advocate" is probably Chris Date, and if you take the time to read his books you'll at least discover proof that you are wrong.
- mojuba 19y ago
  Languages: we may be easily misled by analogies, but still, I'd try to defend my point: you are forced to use SQL only and only because the DB infrastructure can't understand your (presumably imperative) way of describing operations on large data sets. But you are just wrong arguing that relational databases are glorified arrays full of pointers and examples of premature optimization. I think the relational concepts are just a subset of what you can do with various kinds of collections in imperative languages - arrays, sets, maps. For example, I can have an array of records describing users, each of which incorporates a set of groups a user belongs to. In DB terms that's 3 tables (users, groups and membership) with one-to-many relations between them. Either way, we are describing the same concept of users and their membership in various groups. So why bother having another language?

by mojuba 19y ago

jimm 19y ago

I agree with some of your points, but you need to study data structures a bit more. Just because a simple table structure could be implemented as an array doesn't mean that database tables are equivalent to arrays. A primary key is not the same as array index: it can be a type other than integer, and it can be non-consecutive. Indexing columns (including but not limited to the primary key) provides searches that happen in less than linear time. You can't do that with a plain array. Another thing database give you is ACID storage. Sure, you could reproduce some of that using thread-level locking or some other locking mechanism, but then what you have isn't a simple array any more.

mojuba 19y ago
You can't do that with a plain array I can do that with hash arrays. The primary key is my primary hash, other keys are just extra hash maps pointing to the main array. I understand, perhaps none of the existing languages can fully replace the DB functionality, but I suspect nobody is even thinking about that because - hey, there are plenty of nice DBMSs around, why bother reinventing them? but then what you have isn't a simple array any more True, but again, it's a matter of having an infrastructure that handles sharing for you. Windows Registry, however awkward and Microsoft'ish, does just that, as as example.
- nostrademons 19y ago
  "I can do that with hash arrays. The primary key is my primary hash, other keys are just extra hash maps pointing to the main array." A.k.a an in-memory table with hash indexes. MySQL has them. ;-) You're still missing out on a lot of functionality (even besides transactions/persistence/concurrency). For example, what if you want all records where the creation date is between a certain range of days? Or how about the maximum value of a certain column? How about joins? A conventional database can retrieve these from the indexes, but your multiple-hashes system would require a full table scan.
  - mojuba 19y ago
    All you just said is map/reduce, possibly optimized for large data chunks internally. In fact if we create a proper infrastructure, "indexing" can be done automatically and transparently. And I never said arrays should be in memory entirely, even though I declare them as arrays in my language.
    - brlewis 19y ago
      SQL indexes work transparently. Creating them automatically would not be good, since they have a cost for write operations.

Kaizyn 19y ago

The relational model is simply the application of set theory and predicate logic to data management. Are you saying that these two branches of math/logic are ugly or that you don't understand them well enough to have a decent opinion?

olavk 19y ago

Databases are not arrays, although they may be implemented as arrays. The whole point of the relational model is that the logical model is abstracted from the implementation. So if e.g. the DBA decides that it would be more efficient to store a table as an array of columns rather than an array of rows, he can do that without changing the logical model.

nostrademons 19y ago

I don't understand the resistance to using multiple languages in one project. Every time you use an API, you're effectively using a new language. The syntax may be familiar, but the vocabulary is completely different. That's the point - through libraries and DSLs, we extend the capabilities of the base language so it's more suitable to the problem domain. Take a look at the following SQLAlchemy code: select([users.c.first_name + ' ' + users.c.last_name, users.c.email, games.c.title], from_obj=[users.join(games, users.c.user_id == games.c.creator_id], and_(games.c.creation_date == datetime.datetime.now() - datetime.timedelta(3), or_(users.c.email.like('%@aol.com'), users.c.email.like('%@hotmail.com')))) Can you really say that's easier to read than this? SELECT users.first_name + ' ' + users.last_name, users.email, games.title FROM users INNER JOIN games ON (users.user_id = games.game_id) WHERE games.creation_date >= DATE_SUB(NOW(), INTERVAL 3 DAY) AND (users.email LIKE '%@aol.com' OR users.email LIKE '%@hotmail.com') SQLAlchemy is a really cool library - it basically lets you write all your SQL queries in 100% Pure Python. But I doubt I'd use it on my own projects, because I've learned to be suspicious of libraries whose only purpose is to make some other tool look like a language I already know. Invariably there are reasons why the original language looked the way it did, and the reimplementation as a library becomes incredibly clunky as things get more complicated. I find I'm better off learning the original language instead.

olavk 19y ago
My theory is that there is two kinds of people. Those that like everything to be in the same language and environment, and those that like to combine lots of different languages where each language is optimized for a specific task.
- nostrademons 19y ago
  Could be. I'm reminded of the two competing camps for large-scale systems design. The UNIX philosophy is "small pieces loosely joined": each component does one thing well, and the system just provides a common metaphor and architecture for them to work together (eg. pipes and line-oriented text). The Windows philosophy is "big chunks tightly coupled": everything just works out of the box, but if you need to do something the designers didn't think of, you're out of luck. Kinda like "I'll do it myself; show me how" vs. "Do it for me." Nearly every computing system can be placed somewhere along this continuum. In operating systems, we've got MacOS (pre-X) and Windows on one side and UNIX on the other. In languages, we've got Scheme and Smalltalk on the UNIX side and Java and .NET on the other (ironically, Scheme and Smalltalk have a "small pieces loosely coupled" architecture internally, but are big monolithic chunks when it comes to the outside world). For IDEs, Emacs is all the way over on the UNIX side, vim not far behind, Eclipse is in the middle, and IntelliJ and Visual Studio all the way on the Windows side. For JavaScript libraries, YUI/Mootools/Prototype are all on the Windows side and JQuery is on the UNIX side, though it's moving closer to the Windows philosophy. Among web frameworks, Rails and Django are on the Windows side with Pylons on the UNIX side. The web itself is based on the UNIX philosophy, but many of the largest websites follow the Windows philosophy. I'm not so certain that people fit rigidly into one category though. Up until about 3-4 years ago, I was firmly in the Windows camp: I did Java programming only, I did it on a Windows machine, I used JBuilder and Eclipse to do it. Then I started branching out, mostly because I started learning about all this stuff that Java wouldn't let me get at. I think many people may follow a similar path: "do it for me" until you want to do something that the designers didn't think of, and then you have no choice to move to a "do it yourself; we'll show you how" environment.

bootload 19y ago

"... One problem I can think of is that I'm forced to mix two languages in one story .." Do you mean embedded? that I agree with. Embedded sucks but there are lots of techniques (abstraction) you can use to avoid them. Unavoidable is writing web based applications without using several languages. Though I can think of some lisp apps that do not (news). Do you lisp?

mojuba 19y ago
No, I don't lisp, but I know you can nicely embed HTML directly into your lisp code with macros. I saw an example here at YCnews earlier and was struck by it, that was beautiful. No doubt, you can do that with SQL too, although that's just a cosmetic change. Even if your code looks lispish, you still think in DB terms - tables, relations.

gregjor 19y ago

I'm the author of the original article, and I admit that I failed to explain all of the reasons programmers don't like databases -- you came up with one I didn't list (don't want to use a lot of languages) and demonstrated one of the reasons I did list: Who Needs Theory? Your analogy with writing an essay is off target. If writing a program is like writing an essay in English, the SQL/database part of the program are like mathematical formulas in the essay. You could use English instead of mathematical notation but it would be far uglier, less precise, and more prone to error and misunderstanding. In a typical web application (what I've been doing for over ten years) the database part is usually the only part that can be proved correct. I can't help you overcome your misconceptions about relational theory or relational databases, or do anything about your opinion that relational theory is ugly. But you are just wrong arguing that relational databases are glorified arrays full of pointers and examples of premature optimization. The "best relational advocate" is probably Chris Date, and if you take the time to read his books you'll at least discover proof that you are wrong.

mojuba 19y ago
Languages: we may be easily misled by analogies, but still, I'd try to defend my point: you are forced to use SQL only and only because the DB infrastructure can't understand your (presumably imperative) way of describing operations on large data sets. But you are just wrong arguing that relational databases are glorified arrays full of pointers and examples of premature optimization. I think the relational concepts are just a subset of what you can do with various kinds of collections in imperative languages - arrays, sets, maps. For example, I can have an array of records describing users, each of which incorporates a set of groups a user belongs to. In DB terms that's 3 tables (users, groups and membership) with one-to-many relations between them. Either way, we are describing the same concept of users and their membership in various groups. So why bother having another language?