4 ms·

This kind of rubbish really needs to stop. Just because you don't agree with or understand their choices does not mean that the majority of "NoSQL guys" are ign

This kind of rubbish really needs to stop. Just because you don't agree with or understand their choices does not mean that the majority of "NoSQL guys" are ignorant or uneducated. Some of the biggest companies e.g. Twitter, Foursquare, Google, Amazon all rely on NoSQL. The real issue I see is that by dismissing NoSQL as only for fools RDBMS developers are failing to see why they are popular to begin with. Take PostgreSQL for example and how difficult it is to shard/replicate compared to CouchDB or MongoDB. This is an area PostgreSQL should see as an opportunity for improvement. But dismissing huge groups of people as uneducated just makes you seem uneducated.

gaius 14y ago
I think we did "sharding" with relational databases... Back in the 80s. Then we got fast hash joins and partitioning and it turns out that the disadvantages of sharding just aren't worth it. The NoSQL crew will figure this out too around 2030 :-p
- taligent 14y ago
  I guess so will the likes of Facebook, Google, Twitter, eBay etc. Must be amazing to know more about databases than all of the world's biggest companies put together.
  - gaius 14y ago
    Why? The guys at FB or Google are just guys like you and me, don't believe the hype, they are not superheroes or gurus despite their much talked about interview process (the Google interviews I did were a walk in the park compared to GS, btw). Some will know more than me sure, but some'll know less.
    - taligent 14y ago
      Those were only a few examples. I can keep listing companies that use sharding. The idea that it is some 1980's approach to scaling databases is laughable.
      - PommeDeTerre 14y ago
        You're right, it isn't a 1980s approach. I recall hearing about it in the mid 1970s, and even then it was probably originally devised in the 1960s.
    - jeremyjh 14y ago
      Just guys like you or me except they build systems with billions of page views a month.
      - gaius 14y ago
        At my last job, we averaged 60k pages/sec, on what the NoSQL guys would call a "legacy" database, and there was plenty of headroom. It's not rocket science, just engineering. Companies like Google love to weave a mystique around what they do, it's in their interests after all, to convince their investors that what they do is magic. But I'm from back in the day when we were the same way about millions of pages per month, then per week, then per day...
        deleted 14y ago
        [deleted]
        deleted 14y ago
        [deleted]
        einhverfr 14y ago
        On good hardware there were PostgreSQL instances running a billion queries a day back in the 9.0 days. Often these are actual accounting apps where reporting matters and so NoSQL would be a very poor fit. For example the French government uses it to distribute welfare program dispersements. The Wisconsin Courts also uses PostgreSQL at loads in the billion per day range. I know there are larger instances out there. Now a days we are talking about thousands of concurrent users and up to 350k reads/sec on high end machines.
        gaius 14y ago
        Yes there is a lurking iceberg of things that aren't in the public eye like Facebook or Google, and that's where the really intense and interesting stuff is happening. I wonder what the guys at Visa make of all the hype around these websites, when they were doing these volumes all along... At said 60k pages/sec job, you know who we looked up to for databases? Starbucks. Walmart. McDonalds. 'Cos we'd seen what they do, and anyone who thinks Twitter is impressive, well it'd blow their minds.
- einhverfr 14y ago
  Be a little careful with this level of disdain. Sharding may have been available in the 1980's, but what it lead to in some products is quite amazing. Consider Teradata's clustering ability which is sort of like sharding your database but without the disadvantages typically associated with it. Postgres-XC now offers something similar as a semi-fork of PostgreSQL. Basically what we are talking about here is a two-tier database layer where storage and coordination are separated, and two phase commit is used between these two. Thus the coordination tier can enforce referential integrity between storage nodes if necessary and thus allow write-extensibility. This isn't something without uses. For high-end, high-write-load databases, serving very large amounts of traffic (think airline reservations), this has been a typical approach for quite some time. The fundamental problem though is that once you give up on local consistency over a given domain, you cannot have any guarantee of global consistency. The current relational approaches (Postgres-XC and Teradata) both enforce ACID compliance. BASE doesn't offer any consistency guarantee and therefore it is only good for throw-away data.
  - gaius 14y ago
    Oh absolutely, but what you're talking about there, people do with CICS today, and that's even older than the 80s. CICS is a technology I have a lot of respect for. But my point is - when I need to use something like that, I know that's what I'm using. I don't imagine that it's some new invention. Hell, a lot of what the NoSQL guys think they're inventing, IBM did back then too - IMS.
    - einhverfr 14y ago
      "Those who don't understand Unix are condemned to reinvent it, poorly." – Henry Spencer
deleted 14y ago
[deleted]
frio 14y ago
This is a valid point. I dismissed NoSQL for years, and still (largely) do (I prefer to have a schema, documented data structures, deal with relational data etc.)... but I finally gave Mongo a go the other day, and am actually quite impressed. For a simple endpoint I can throw arbitrary persistent data in (rather than the filesystem), it's let me get a couple of proof-of-concept projects out the door much quicker. Note that I'm not intending to comment on its scalability or how appropriate it is for other projects, just that I agree with you: dismissing it out of hand is hubris.
pessimizer 14y ago
The majority of NoSQL guys are not Twitter, Foursquare, Google, or Amazon, and Twitter, Foursquare, Google, and Amazon all have plenty of people experienced enough in traditional RDBMS to tell them what things would probably be better in them. Most NoSQL guys I meet have little to no experience in SQL. Plenty have lots.
- taligent 14y ago
  The plural of anecdote is not data. You know that right ? In my case every NoSQL developer I've met had a lot of experience in SQL as well. But do you have ask why so called inexperienced users are choosing NoSQL in the first place ? Hint: it's because most RDBMS are ridiculously complex and inflexible.
  - reinhardt 14y ago
    The last line sounds pretty familiar after replacing "NoSQL" with "PHP" and "RDBMS" with "decent languages".
    - taligent 14y ago
      I completely agree. This whole NoSQL/RDBMS situation is programming language debate being played out over again. And just like not every programming language works for every situation. Neither does every database.
      - einhverfr 14y ago
        I agree. BTW, the most lucid description of when to use relational vs non-relational models is not all that new. You can read it here: http://db.cs.berkeley.edu/papers/Informix/www.informix.com/informix/corpinfo/zines/whitpprs/illuswp/wave.htm http://db.cs.berkeley.edu/papers/Informix/www.informix.com/i... In this model, there is virtually no difference between the complex data/no query db's of the paper then and NoSQL today in terms of use case. Interesting that a one-time executive of Informix was able to present such a long-lasting discussion of db technologies.....

gaius 14y ago

I think we did "sharding" with relational databases... Back in the 80s. Then we got fast hash joins and partitioning and it turns out that the disadvantages of sharding just aren't worth it. The NoSQL crew will figure this out too around 2030 :-p

taligent 14y ago
I guess so will the likes of Facebook, Google, Twitter, eBay etc. Must be amazing to know more about databases than all of the world's biggest companies put together.
- gaius 14y ago
  Why? The guys at FB or Google are just guys like you and me, don't believe the hype, they are not superheroes or gurus despite their much talked about interview process (the Google interviews I did were a walk in the park compared to GS, btw). Some will know more than me sure, but some'll know less.
  - taligent 14y ago
    Those were only a few examples. I can keep listing companies that use sharding. The idea that it is some 1980's approach to scaling databases is laughable.
    - PommeDeTerre 14y ago
      You're right, it isn't a 1980s approach. I recall hearing about it in the mid 1970s, and even then it was probably originally devised in the 1960s.
  - jeremyjh 14y ago
    Just guys like you or me except they build systems with billions of page views a month.
    - gaius 14y ago
      At my last job, we averaged 60k pages/sec, on what the NoSQL guys would call a "legacy" database, and there was plenty of headroom. It's not rocket science, just engineering. Companies like Google love to weave a mystique around what they do, it's in their interests after all, to convince their investors that what they do is magic. But I'm from back in the day when we were the same way about millions of pages per month, then per week, then per day...
      - deleted 14y ago
        [deleted]
      - deleted 14y ago
        [deleted]
      - einhverfr 14y ago
        On good hardware there were PostgreSQL instances running a billion queries a day back in the 9.0 days. Often these are actual accounting apps where reporting matters and so NoSQL would be a very poor fit. For example the French government uses it to distribute welfare program dispersements. The Wisconsin Courts also uses PostgreSQL at loads in the billion per day range. I know there are larger instances out there. Now a days we are talking about thousands of concurrent users and up to 350k reads/sec on high end machines.
        gaius 14y ago
        Yes there is a lurking iceberg of things that aren't in the public eye like Facebook or Google, and that's where the really intense and interesting stuff is happening. I wonder what the guys at Visa make of all the hype around these websites, when they were doing these volumes all along... At said 60k pages/sec job, you know who we looked up to for databases? Starbucks. Walmart. McDonalds. 'Cos we'd seen what they do, and anyone who thinks Twitter is impressive, well it'd blow their minds.
einhverfr 14y ago
Be a little careful with this level of disdain. Sharding may have been available in the 1980's, but what it lead to in some products is quite amazing. Consider Teradata's clustering ability which is sort of like sharding your database but without the disadvantages typically associated with it. Postgres-XC now offers something similar as a semi-fork of PostgreSQL. Basically what we are talking about here is a two-tier database layer where storage and coordination are separated, and two phase commit is used between these two. Thus the coordination tier can enforce referential integrity between storage nodes if necessary and thus allow write-extensibility. This isn't something without uses. For high-end, high-write-load databases, serving very large amounts of traffic (think airline reservations), this has been a typical approach for quite some time. The fundamental problem though is that once you give up on local consistency over a given domain, you cannot have any guarantee of global consistency. The current relational approaches (Postgres-XC and Teradata) both enforce ACID compliance. BASE doesn't offer any consistency guarantee and therefore it is only good for throw-away data.
- gaius 14y ago
  Oh absolutely, but what you're talking about there, people do with CICS today, and that's even older than the 80s. CICS is a technology I have a lot of respect for. But my point is - when I need to use something like that, I know that's what I'm using. I don't imagine that it's some new invention. Hell, a lot of what the NoSQL guys think they're inventing, IBM did back then too - IMS.
  - einhverfr 14y ago
    "Those who don't understand Unix are condemned to reinvent it, poorly." – Henry Spencer

deleted 14y ago

[deleted]

frio 14y ago

This is a valid point. I dismissed NoSQL for years, and still (largely) do (I prefer to have a schema, documented data structures, deal with relational data etc.)... but I finally gave Mongo a go the other day, and am actually quite impressed. For a simple endpoint I can throw arbitrary persistent data in (rather than the filesystem), it's let me get a couple of proof-of-concept projects out the door much quicker. Note that I'm not intending to comment on its scalability or how appropriate it is for other projects, just that I agree with you: dismissing it out of hand is hubris.

pessimizer 14y ago

The majority of NoSQL guys are not Twitter, Foursquare, Google, or Amazon, and Twitter, Foursquare, Google, and Amazon all have plenty of people experienced enough in traditional RDBMS to tell them what things would probably be better in them. Most NoSQL guys I meet have little to no experience in SQL. Plenty have lots.

taligent 14y ago
The plural of anecdote is not data. You know that right ? In my case every NoSQL developer I've met had a lot of experience in SQL as well. But do you have ask why so called inexperienced users are choosing NoSQL in the first place ? Hint: it's because most RDBMS are ridiculously complex and inflexible.
- reinhardt 14y ago
  The last line sounds pretty familiar after replacing "NoSQL" with "PHP" and "RDBMS" with "decent languages".
  - taligent 14y ago
    I completely agree. This whole NoSQL/RDBMS situation is programming language debate being played out over again. And just like not every programming language works for every situation. Neither does every database.
    - einhverfr 14y ago
      I agree. BTW, the most lucid description of when to use relational vs non-relational models is not all that new. You can read it here: http://db.cs.berkeley.edu/papers/Informix/www.informix.com/informix/corpinfo/zines/whitpprs/illuswp/wave.htm http://db.cs.berkeley.edu/papers/Informix/www.informix.com/i... In this model, there is virtually no difference between the complex data/no query db's of the paper then and NoSQL today in terms of use case. Interesting that a one-time executive of Informix was able to present such a long-lasting discussion of db technologies.....