8 ms·

RDBMSs are as powerful as ever and are still improving rapidly. If you look at "what's new" in Oracle 12c, SQL Server 2016, VoltDB and Vertica; you can see they

by gfody 11y ago

RDBMSs are as powerful as ever and are still improving rapidly. If you look at "what's new" in Oracle 12c, SQL Server 2016, VoltDB and Vertica; you can see they're not maintenance mode. I think calling them fossils that should be burned down to fuel the next thing is way off. IMO the main thing RDBMSs have failed to do is to capture the new generation of developers who don't have the desire or the patience to learn SQL, relational concepts, various modeling techniques, and whatever sharding or shared nothing architectural fundamentals just to scale out. This generation wants to "npm install magic-database", write everything in javascript, and push to the cloud without ever having to think about scaleout. This isn't necessarily a problem with RDBMSs but RDBMSs could address it. [1] https://docs.oracle.com/database/121/NEWFT/chapter12102.htm https://docs.oracle.com/database/121/NEWFT/chapter12102.htm [2] https://msdn.microsoft.com/en-us/library/bb510411.aspx https://msdn.microsoft.com/en-us/library/bb510411.aspx [3] https://docs.voltdb.com/ReleaseNotes https://docs.voltdb.com/ReleaseNotes [4] https://community.dev.hpe.com/t5/Vertica-Blog/What-s-New-in-Vertica-7-2-2/ba-p/235496 https://community.dev.hpe.com/t5/Vertica-Blog/What-s-New-in-...

Retra 11y ago
>IMO the main thing RDBMSs have failed to do is to capture the new generation of developers who don't have the desire or the patience to learn [...] As one of this 'new generation of developers' (who happens to also work on industry database internals), I would say that I've been enormously unimpressed by the amount of specialized knowledge needed to implement an optimized, portable, flexible, scalable database. Do you know what tables in your DB need indexes and why? Do you know how to optimally set up your partitions? Do you know why your one query runs fast for a week and then suddenly runs slow, and what you can do about it? The database API is just too low-level for most people's comfort zone. Most people who want to use a database are just trying to stuff organized bits somewhere and be able to get them out quickly. Databases are smart, but the admin still has to learn and learn and learn -- all this esoteric, domain specific knowledge -- just to store some damn bits. Learning to use a database feels like learning machine code in that if you suck at it, you're going to pay the price of not having the best performance. Databases market themselves as data storage applications, but they are complex programming APIs at heart. People turn to databases to quickly solve data storage problems, not because they want to interface with some unsexy, cryptic API.
- qewrffewqwfqew 11y ago
  Is any of that really less the case with NoSQL?
  - oneloop 11y ago
    Yes. The need for indices is easier to identify on NoSQL: if you want to grab a key in O(1) then you need an index. However this simplicity comes not from eschewing SQL but because the queries you typically demand are simpler.
    - creshal 11y ago
      I'm not sure I follow that line of argument. If plain SQL is too low-level for you there's a wealth of ORMs that abstract away all the nitty-gritty details.
      - virmundi 11y ago
        Up front: I'm thinking about leaving NoSQL because I hate its memory model. SQL requires joins to get a "document" back. In order to properly optimize that process, you'll want an index. Most DBs don't do that (if I understand them correctly). See 1 for details about PG. NoSQLs like Mongo lack joins. As a result you pack most of your joinable data into a document. When you need some manner of linked relation, you can store either the key to the collection, or a URI that goes through your API to that collection. In either case, you're pegging against an indexed key (probably the PK). http://dba.stackexchange.com/questions/53809/need-for-indexes-on-foreign-keys http://dba.stackexchange.com/questions/53809/need-for-indexe...
        deleted 11y ago
        [deleted]
        jasondc 11y ago
        https://docs.mongodb.org/manual/reference/operator/aggregation/lookup/ https://docs.mongodb.org/manual/reference/operator/aggregati...
eva1984 11y ago
SQL is easy. I believe in the life a developer, you will have to run into SQL, here and there which is inevitable. But we have a lot of other tools now, RDBMS is just one of them. Plus the problems everybody having are vastly different those days, the solution varies with them. Can RDBMS used to address them? Probably. But RDBMS are beasts themselves, the amount of effort to optimize it for your specific requirement, might not look worthy compared to just setup a NoSQL/NewSQL solution that has been built purposefully to perform under certain scenarios. I believe the new gen of devs, are just being more pragmatic and unconstrained about the data storage solutions the have in the toolkits, and pick whatever they seem fit.
- sqldba 11y ago
  SQL is easy. ORM is not - except through a framework which will make you forget about the SQL at all.
  - luckystarr 11y ago
    ORMs are a trap. They are like some sugary drink which makes you fat and you only realize it when it's too late. They prevent harnessing the power of SQL by providing seemingly convenient shortcuts. Only later (when it's too late and you already invested heavily in it) it becomes clear that one would have been better off not using an ORM but using SQL directly.
    - kidmenot 11y ago
      That's a great analogy, I'm totally with you on this one. ORMs are painful, I only use solutions which help me reducing the amount of boilerplate code to map data to domain objects (Dapper being a great example in .NET land), but I really prefer writing my queries and seeing them at a glance (however ugly they may look when inlined) without having to actually run the app. Plus, SQL produced by things like Entity Framework is not really readable with all those "extent1", "extent2" and so on. In addition, ORMs tie you to the lowest common denominator so if you want to take full advantage of, say, jsonb in PostgreSQL you're out of luck. I mean, in principle they're great, and they will theoretically allow you to switch database engines but, according to my limited experience, that is not such a frequent occurrence. The thing is, they come with too many hidden costs, and I never bought the "no need to know any SQL" argument to begin with, as I really think a decent command of it is part of the fundamental abilities of a developer.
nickbauman 11y ago
Disagree. For starters it's clear the traditional (legacy?) RDMS community has yet to come to terms with the realities of the CAP theorem. Google, Facebook, etc, does not, nor will they ever use an Oracle database to run its business. That's a huge pile of data being ignored.
- walshemj 11y ago
  I bet their backend billing/revenue sytems do.
- gfody 10y ago
  Facebook has one of the largest Vertica installations in the world.
  - nickbauman 10y ago
    All while Facebook being the birthplace of Cassandra.
cwyers 11y ago
It's not generational. I don't think developers these days are any lazier or less gifted than developers at any other times. We're at a point in history where it's much, much cheaper at a certain point to double the number of computers you have than it is to make all of your computers twice as fast. SQL databases, especially the ones we have right now, were built with the idea that you'd have one large database server -- replication and such exists but it's an extension of the single-server model. Things like NoSQL databases and Hadoop are meant to provide a model that supports horizontal scaling better than existing relational databases do. As they mature, they are starting to take on more and more features from relational databases. And SQL databases are taking on more and more features that let them scale horizontally. I suspect at some point in the future we'll see a lot more convergence.
- gfody 11y ago
  I didn't mean to imply that the new generation of devs are lazy and stupid. They (and you apparently) are influenced by the perception that RDBMSs are relics not designed with modern hardware in mind. This perception is wrong. Vertica and VoltDB were both designed from the beginning to be distributed. SQL Server and Oracle were not designed that way in the beginning but incorporated those capabilities over time. All of these are incredibly sophisticated, modern, high performance RDBMSs. None of them can be used effectively without some effort to acquire relational concepts, modeling, SQL, etc. This is where the "database community" has lost ground to the prevailing NoSQL solutions IMO, by not making it easier for new devs to plug and play. One example of the sort of thing that could make RDBMSs more plug-and-play is Vertica's flex tables - a schema-free, super flexible blob of a table that lets you move forward without having to completely define your relational model.
manigandham 11y ago
I'd like to point out http://www.memsql.com http://www.memsql.com as one of the better NewSQL implementations with a solid distributed design that actually works and scales.

by gfody 11y ago

Retra 11y ago

>IMO the main thing RDBMSs have failed to do is to capture the new generation of developers who don't have the desire or the patience to learn [...] As one of this 'new generation of developers' (who happens to also work on industry database internals), I would say that I've been enormously unimpressed by the amount of specialized knowledge needed to implement an optimized, portable, flexible, scalable database. Do you know what tables in your DB need indexes and why? Do you know how to optimally set up your partitions? Do you know why your one query runs fast for a week and then suddenly runs slow, and what you can do about it? The database API is just too low-level for most people's comfort zone. Most people who want to use a database are just trying to stuff organized bits somewhere and be able to get them out quickly. Databases are smart, but the admin still has to learn and learn and learn -- all this esoteric, domain specific knowledge -- just to store some damn bits. Learning to use a database feels like learning machine code in that if you suck at it, you're going to pay the price of not having the best performance. Databases market themselves as data storage applications, but they are complex programming APIs at heart. People turn to databases to quickly solve data storage problems, not because they want to interface with some unsexy, cryptic API.

qewrffewqwfqew 11y ago
Is any of that really less the case with NoSQL?
- oneloop 11y ago
  Yes. The need for indices is easier to identify on NoSQL: if you want to grab a key in O(1) then you need an index. However this simplicity comes not from eschewing SQL but because the queries you typically demand are simpler.
  - creshal 11y ago
    I'm not sure I follow that line of argument. If plain SQL is too low-level for you there's a wealth of ORMs that abstract away all the nitty-gritty details.
    - virmundi 11y ago
      Up front: I'm thinking about leaving NoSQL because I hate its memory model. SQL requires joins to get a "document" back. In order to properly optimize that process, you'll want an index. Most DBs don't do that (if I understand them correctly). See 1 for details about PG. NoSQLs like Mongo lack joins. As a result you pack most of your joinable data into a document. When you need some manner of linked relation, you can store either the key to the collection, or a URI that goes through your API to that collection. In either case, you're pegging against an indexed key (probably the PK). http://dba.stackexchange.com/questions/53809/need-for-indexes-on-foreign-keys http://dba.stackexchange.com/questions/53809/need-for-indexe...
      - deleted 11y ago
        [deleted]
      - jasondc 11y ago
        https://docs.mongodb.org/manual/reference/operator/aggregation/lookup/ https://docs.mongodb.org/manual/reference/operator/aggregati...

eva1984 11y ago

SQL is easy. I believe in the life a developer, you will have to run into SQL, here and there which is inevitable. But we have a lot of other tools now, RDBMS is just one of them. Plus the problems everybody having are vastly different those days, the solution varies with them. Can RDBMS used to address them? Probably. But RDBMS are beasts themselves, the amount of effort to optimize it for your specific requirement, might not look worthy compared to just setup a NoSQL/NewSQL solution that has been built purposefully to perform under certain scenarios. I believe the new gen of devs, are just being more pragmatic and unconstrained about the data storage solutions the have in the toolkits, and pick whatever they seem fit.

sqldba 11y ago
SQL is easy. ORM is not - except through a framework which will make you forget about the SQL at all.
- luckystarr 11y ago
  ORMs are a trap. They are like some sugary drink which makes you fat and you only realize it when it's too late. They prevent harnessing the power of SQL by providing seemingly convenient shortcuts. Only later (when it's too late and you already invested heavily in it) it becomes clear that one would have been better off not using an ORM but using SQL directly.
  - kidmenot 11y ago
    That's a great analogy, I'm totally with you on this one. ORMs are painful, I only use solutions which help me reducing the amount of boilerplate code to map data to domain objects (Dapper being a great example in .NET land), but I really prefer writing my queries and seeing them at a glance (however ugly they may look when inlined) without having to actually run the app. Plus, SQL produced by things like Entity Framework is not really readable with all those "extent1", "extent2" and so on. In addition, ORMs tie you to the lowest common denominator so if you want to take full advantage of, say, jsonb in PostgreSQL you're out of luck. I mean, in principle they're great, and they will theoretically allow you to switch database engines but, according to my limited experience, that is not such a frequent occurrence. The thing is, they come with too many hidden costs, and I never bought the "no need to know any SQL" argument to begin with, as I really think a decent command of it is part of the fundamental abilities of a developer.

nickbauman 11y ago

Disagree. For starters it's clear the traditional (legacy?) RDMS community has yet to come to terms with the realities of the CAP theorem. Google, Facebook, etc, does not, nor will they ever use an Oracle database to run its business. That's a huge pile of data being ignored.

walshemj 11y ago
I bet their backend billing/revenue sytems do.
gfody 10y ago
Facebook has one of the largest Vertica installations in the world.
- nickbauman 10y ago
  All while Facebook being the birthplace of Cassandra.

cwyers 11y ago

It's not generational. I don't think developers these days are any lazier or less gifted than developers at any other times. We're at a point in history where it's much, much cheaper at a certain point to double the number of computers you have than it is to make all of your computers twice as fast. SQL databases, especially the ones we have right now, were built with the idea that you'd have one large database server -- replication and such exists but it's an extension of the single-server model. Things like NoSQL databases and Hadoop are meant to provide a model that supports horizontal scaling better than existing relational databases do. As they mature, they are starting to take on more and more features from relational databases. And SQL databases are taking on more and more features that let them scale horizontally. I suspect at some point in the future we'll see a lot more convergence.

gfody 11y ago
I didn't mean to imply that the new generation of devs are lazy and stupid. They (and you apparently) are influenced by the perception that RDBMSs are relics not designed with modern hardware in mind. This perception is wrong. Vertica and VoltDB were both designed from the beginning to be distributed. SQL Server and Oracle were not designed that way in the beginning but incorporated those capabilities over time. All of these are incredibly sophisticated, modern, high performance RDBMSs. None of them can be used effectively without some effort to acquire relational concepts, modeling, SQL, etc. This is where the "database community" has lost ground to the prevailing NoSQL solutions IMO, by not making it easier for new devs to plug and play. One example of the sort of thing that could make RDBMSs more plug-and-play is Vertica's flex tables - a schema-free, super flexible blob of a table that lets you move forward without having to completely define your relational model.

manigandham 11y ago

I'd like to point out http://www.memsql.com http://www.memsql.com as one of the better NewSQL implementations with a solid distributed design that actually works and scales.