3 ms·

Thank you! I've bookmarked this reply, so that the next 20 times I see ardit33's argument made I can just link to it rather than pounding out my own long-winde

by dfranke 19y ago

Thank you! I've bookmarked this reply, so that the next 20 times I see ardit33's argument made I can just link to it rather than pounding out my own long-winded reply or just beating my head against my desk until it doesn't hurt any more. (The reply that I would have written if you didn't respond first would probably have emphasized the amount of code required rather than performance.)

edw519 19y ago
lol I could have made any number of technical arguments. We'd be pissing at each other all day. Instead, I cited a few things that concern those who pay the bills. As hackers engaging in good technical debate here, it's too easy to lose sight of those whom we serve.
- dfranke 19y ago
  It's frustrating how many smart people agree with ardit33's argument, though. I've even heard the same thing from spez. When you're mostly writing CRUD queries, you can get away with just treating the DB as a hash table, but it still means you're probably doing a lot of things inefficiently. The database needs of the company I work for now are just run-of-the-mill web app stuff, but still it's just another day at the office when I write an SQL query that could easily translate to 100 LOC worth of hash table operations. Here's a real example from our forum code that I've used as a job interview question: each user is member of some set of groups (groups in the UNIX sense), and each forum requires membership in all of some set of groups in order to be allowed to post. Given a userid, return the set forums to which that user is allowed to post. You can do it with a single, concise SQL statement containing nested existential quantifiers, and the query plan it generates is very fast.
  - mojuba 19y ago
    Let's say I have the entire structure in memory, interlinked with pointers. The solution in pseudocode: allowed_forums = [] // empty set for each group in user.groups allowed_forums |= group.forums // union That's all. I can probably write a compiler that translates this to SQL if necessary. Nice thing is that when I write this piece I don't care about how exactly data is stored and accessed. Now let's compare it with your SQL solution. You see, mine so simple, it's not even an interview question :)
    - dfranke 19y ago
      1. Your solution is incorrect. I said you need to be a member of all the groups associated with a forum, not one of them. How would your code handle a public forum (one with no groups associated with it)? 2. Suppose you have one million users. Then you're sucking up an awful lot of RAM just for this tiny piece of functionality. 3. Your pseudocode makes a lot of convenient assumptions about what functionality is already provided in standard libraries and in the accessor methods for user.groups and group.forums. Let's see it written in a real language. 4. You say you don't care about how data is stored and accessed, but given the right set of reasonable assumptions, I can conclude that the running time of your algorithm is O(n^3). The beauty of relational databases is that you usually really don't have to care. The query optimizer is pretty good at figuring it out for you. EDIT: nostrademons, I saw that peer reply before you deleted it. That was just spookily similar :-)
      - mojuba 19y ago
        1. Again, easier than you think: allowed_forums = [] for each forum in forums if forum.groups <= user.groups // set comparison, like in Pascal allowed_forums |= forum // include an item 2. Let' say, I have a system that will do swapping and optimization for me. 3. My pseudocode is an assumption that it is possible to build some language/environment that would replace databases, only in theory so far. 4. Ok, maybe I'm going to use the "beauty of databases" in the implementation for my language. The goal is to eliminate SQL (and useless interview questions at the same time :)
        dfranke 19y ago
        1. Ok, that looks correct now. 2. There's a word for such a system. It's called a database :-). 3/4. Ok, sounds good. But like many others in this discussion, you're confounding SQL with the relational model. Your pseudocode above goes slightly outside the relational model because it's written in imperative style, but can be adapted to it easily enough just by replacing your for loop with set-builder notation and maybe a unary union operator. Sure, SQL has warts. It was originally designed as a language extension for COBOL for chrissakes. But the language you're inventing is just a prettier syntax for basically the same semantics.

by dfranke 19y ago

edw519 19y ago

lol I could have made any number of technical arguments. We'd be pissing at each other all day. Instead, I cited a few things that concern those who pay the bills. As hackers engaging in good technical debate here, it's too easy to lose sight of those whom we serve.

dfranke 19y ago
It's frustrating how many smart people agree with ardit33's argument, though. I've even heard the same thing from spez. When you're mostly writing CRUD queries, you can get away with just treating the DB as a hash table, but it still means you're probably doing a lot of things inefficiently. The database needs of the company I work for now are just run-of-the-mill web app stuff, but still it's just another day at the office when I write an SQL query that could easily translate to 100 LOC worth of hash table operations. Here's a real example from our forum code that I've used as a job interview question: each user is member of some set of groups (groups in the UNIX sense), and each forum requires membership in all of some set of groups in order to be allowed to post. Given a userid, return the set forums to which that user is allowed to post. You can do it with a single, concise SQL statement containing nested existential quantifiers, and the query plan it generates is very fast.
- mojuba 19y ago
  Let's say I have the entire structure in memory, interlinked with pointers. The solution in pseudocode: allowed_forums = [] // empty set for each group in user.groups allowed_forums |= group.forums // union That's all. I can probably write a compiler that translates this to SQL if necessary. Nice thing is that when I write this piece I don't care about how exactly data is stored and accessed. Now let's compare it with your SQL solution. You see, mine so simple, it's not even an interview question :)
  - dfranke 19y ago
    1. Your solution is incorrect. I said you need to be a member of all the groups associated with a forum, not one of them. How would your code handle a public forum (one with no groups associated with it)? 2. Suppose you have one million users. Then you're sucking up an awful lot of RAM just for this tiny piece of functionality. 3. Your pseudocode makes a lot of convenient assumptions about what functionality is already provided in standard libraries and in the accessor methods for user.groups and group.forums. Let's see it written in a real language. 4. You say you don't care about how data is stored and accessed, but given the right set of reasonable assumptions, I can conclude that the running time of your algorithm is O(n^3). The beauty of relational databases is that you usually really don't have to care. The query optimizer is pretty good at figuring it out for you. EDIT: nostrademons, I saw that peer reply before you deleted it. That was just spookily similar :-)
    - mojuba 19y ago
      1. Again, easier than you think: allowed_forums = [] for each forum in forums if forum.groups <= user.groups // set comparison, like in Pascal allowed_forums |= forum // include an item 2. Let' say, I have a system that will do swapping and optimization for me. 3. My pseudocode is an assumption that it is possible to build some language/environment that would replace databases, only in theory so far. 4. Ok, maybe I'm going to use the "beauty of databases" in the implementation for my language. The goal is to eliminate SQL (and useless interview questions at the same time :)
      - dfranke 19y ago
        1. Ok, that looks correct now. 2. There's a word for such a system. It's called a database :-). 3/4. Ok, sounds good. But like many others in this discussion, you're confounding SQL with the relational model. Your pseudocode above goes slightly outside the relational model because it's written in imperative style, but can be adapted to it easily enough just by replacing your for loop with set-builder notation and maybe a unary union operator. Sure, SQL has warts. It was originally designed as a language extension for COBOL for chrissakes. But the language you're inventing is just a prettier syntax for basically the same semantics.