6 ms·

>I think NoSql, especially things like Mongo, got popular because it is super easy to program with javascript. I chose Mongo for a project recently. I have man

>I think NoSql, especially things like Mongo, got popular because it is super easy to program with javascript. I chose Mongo for a project recently. I have many years experience with SQL databases, and I think SQL will become more of a niche in the future. Here are the reasons: Everybody uses an ORM with a SQL database. You can pretend they don't and everyone is writing raw sql, but they aren't. This is basically a big, complicated, and slow piece of software that tries to make a SQL database into something else. A NoSQL database is like a native version of that. It starts off with you being able to define collections and schemas in code and so on. SQL is a shit way to query a database. I used to write massive queries that took an hour or so to write just so that the non-programming people could put them into analytics software that didn't support any other methods of input. While I was writing these queries, I was just thinking to myself "I can write this in code in a few minutes instead of an hour or more". The fact is that programming languages are much better at querying databases than SQL. Joins are slow and complicated, and ORMs are slow and complicated, so you can never actually use any of these things in a high-traffic environment. As you say, not many people will reach these levels of traffic, but it is still a factor. Most of the comments about "I moved from mongo to postgres" lately are first-time programmers who didn't know any of the basic concepts of databases. They then discovered patterns that the SQL database forced on them and declared that mongo sucks when in reality, they just didn't know what they were doing. >One of the problems of many stacks is that the frameworks wrap general purpose languages over SQL, which, is not really a good idea, SQL is a vastly more capable language for dealing with relational data and layers built over the top often dumb down the database. This is not true at all. Programming languages are vastly superior at querying a database. Just go write a complex query and compare it to the one in linq or whatever you use. >Microsoft at one stage had Linq to SQL which was quite good..... but they killed it :) They have entity framework, which uses linq and is the same thing. It is very slow though.

richmarr 8y ago
> This is basically a big, complicated, and slow piece of software that tries to make a SQL database into something else. Just to nitpick a little, but I think it's a worthwhile distinction, ORMs don't try to make a SQL database something else... either literally or philosophically. To use them in any more than a trivial way you still to understand RDBMSs. They just make the queries less verbose and the output more convenient to work with.
- 189723954 8y ago
  You are just defining the limits of "something else" and then saying it doesn't do that. If your data looks completely differently to how the database outputs it, I don't think you can say it is just a trivial difference. Both the structure of a query and its output use different concepts to sql. Using linq, a query will look like: db.Products.Include(p => p.manufacturers).Include(p => p.parts).where(p => p.name == 'bike').ToList(); in sql, it is SELECT * from products, manufacturers, parts, JOIN ...... ON ......... The linq query syntax makes it seem like you are just plucking a Product out of a database that has manfufacturer and a list of parts as part of its object. It is an object, not a row and table based structure. Then the output itself is also an object, in a completely different structure to what the database gave to you. People can use an ORM and not really know how SQL works if they never bother to learn. It is presenting them with an object-based database. In a NoSQL database like Mongo, you either have to literally store the Product with its parts and manufacturer, or you take them separately from the database and put them back together in code. This requires no abstraction. It is exactly what is happening.
  - dang 8y ago
    Could you please stop creating new accounts for every few comments you post? We ban accounts that do this, and it's in the site guidelines: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html. It's particularly abusive that you used multiple accounts in this same thread. HN is a community. Obviously you don't have to use your real name, but if users don't have some consistent identity for others to relate to, we may as well have no usernames and no community at all. That would be quite a different kind of forum. Anonymity is fine, and throwaways for a specific purpose are ok—just not routinely. https://hn.algolia.com/?sort=byDate&dateRange=all&type=comment&storyText=false&prefix=false&page=0&query=by:dang%20community%20identity https://hn.algolia.com/?sort=byDate&dateRange=all&type=comme...
- threeseed 8y ago
  > ORMs don't try to make a SQL database something else This is simply not true. Many ORMs are designed to abstract away SQL and RDBMS concepts entirely. They deal in objects and object graphs and output SQL which can be quite disjointed from the object model e.g. many-many relationships.
deleted 8y ago
[deleted]
CuriousSkeptic 8y ago
Linq works for trivial left join project style of queries. But for anything actually using the features of modern sql-databases it comes short quickly. As a example try to express something like “sum(x) over (partition by y order by z)” I do agree that SQL the syntax leaves a lot to be desired, and a proper relational language with a syntax optimized for actual development, and even better, optimized for 6NF style databases, would be awesome, but linq is not by any stretch sufficient to be that
- jeffreyrogers 8y ago
  My understanding is everyone uses 3NF in practice though, so what would a 6NF database get you practically? I see SQL the way I see things like Linux: it's got some weird things that you'd design differently if you could do it over again, but overall it's pretty good and probably not worth the effort to switch.
  - CuriousSkeptic 8y ago
    6NF gives flexibility since every thing is decoupled until you bring it in. It makes evolving the schema easier, makes optimizations easier. I also have a hunch that with such a focus it would be easier to find an interesting design space for new relational programming models since it’s, in a sense, “purer”. But I guess it depends on the use case. For normal oltp style save/fetch entity 3NF makes sense. But it kind of makes sense in the way an ORM would make sense. Which makes you question if you really need relational database at all.
matwood 8y ago
> Everybody uses an ORM with a SQL database. I'm ripping ORMs out of any backend code and using things like jOOQ. > I was just thinking to myself "I can write this in code in a few minutes instead of an hour or more".... > Joins are slow and complicated You know what's really slow? Creating joins in code. > This is not true at all. Programming languages are vastly superior at querying a database. Just go write a complex query and compare it to the one in linq or whatever you use. It makes zero sense to pull a bunch of records back from the db using multiple network calls to join and then filter. Let the db do its job.
- mmt 8y ago
  > It makes zero sense to pull a bunch of records back from the db using multiple network calls to join and then filter. Let the db do its job. But then it wouldn't be distributed processing! :) Seriously, though, consider it for a moment.. this pattern has similar features to something like Hadoop. The data comes from storage nodes (database server and, hopefully, their read replicas) and goes to processing nodes (app server) to have the work done and is then new data is written back out over the network to storage nodes and replicated across the network (to the replica/slave database servers). If the data volume is particularly low or the compute load (CPU and/or RAM) is particularly high, the distributed method would make intuitive sense. I haven't seen it yet, however.
  - matwood 8y ago
    Touché. You're right, at some point the data must be joined and filtered. My point was to let the tool do its job :)
  - jandrewrogers 8y ago
    Doing distributed joins correctly requires an architectural/technical capability that most distributed database engines don't have: decentralized parallel orchestration. If you have this, you can do joins even with very high data volumes efficiently given good parallel scheduling algorithms. Most databases are designed such that there is a single point of control that declaratively schedules all data flows required to execute the query; this scales poorly for operations like joins, never mind recursive joins, which is why you don't see it. Letting individual database nodes dynamically schedule and orchestrate their own data flows with each other, essentially allowing each node in the parallel system construct its own execution plan in relation to other nodes as it goes along, does not fit within the "giant distributed file system" paradigm that most distributed systems are based on. People who design codes for supercomputers are often familiar with parallel orchestration idioms that work at extremely large scales but it hasn't crossed over into ordinary distributed database engines. (This is also a good litmus test for what makes a database "parallel" as distinct from "distributed".) Most distributed database architectures are much more centralized than they need to be, particularly around control of execution planning, and this limits their expressiveness. It is quite difficult to hack together a distributed join that performs better than a centralized one without good support for parallel orchestration.
orf 8y ago
> The fact is that programming languages are much better at querying databases than SQL. They really are not though. Sure, perhaps for simple single table scans, bu nothing beats the pure optimizing potential of SQL. Try joining three tables in JS neatly and fast. It's a bit terse, but you're lying to yourself if you think hodgepodge mess of ad-hoc JavaScript written to poorly replicate a single specific query is in any way better than what we have now.
flavio81 8y ago
>Everybody uses an ORM with a SQL database. That's not true. >While I was writing these queries, I was just thinking to myself "I can write this in code in a few minutes instead of an hour or more" Well, no, because SQL is a 4th generation language of higher level than most general-purpose languages. It is a domain-specific language focused on database handling. A simple SELECT with a few joins and indexes involved encompasses a query execution plan that would be a few hundreds of code in a regular programming language. >I chose Mongo for a project recently. Storing relational data on a document store is a bad idea. And if you need a document store, Mongo is pretty much a bad option.
- ruiquelhas 8y ago
  Why is Mongo a bad option for a document store? Serious question.
collyw 8y ago
> Joins are slow and complicated, and ORMs are slow and complicated, so you can never actually use any of these things in a high-traffic environment. As you say, not many people will reach these levels of traffic, but it is still a factor. I have to clean up crap written by people like you. Go learn how to use a relational database properly instead of writing tosh like this and crap buggy code in the application layer.

richmarr 8y ago

> This is basically a big, complicated, and slow piece of software that tries to make a SQL database into something else. Just to nitpick a little, but I think it's a worthwhile distinction, ORMs don't try to make a SQL database something else... either literally or philosophically. To use them in any more than a trivial way you still to understand RDBMSs. They just make the queries less verbose and the output more convenient to work with.

189723954 8y ago
You are just defining the limits of "something else" and then saying it doesn't do that. If your data looks completely differently to how the database outputs it, I don't think you can say it is just a trivial difference. Both the structure of a query and its output use different concepts to sql. Using linq, a query will look like: db.Products.Include(p => p.manufacturers).Include(p => p.parts).where(p => p.name == 'bike').ToList(); in sql, it is SELECT * from products, manufacturers, parts, JOIN ...... ON ......... The linq query syntax makes it seem like you are just plucking a Product out of a database that has manfufacturer and a list of parts as part of its object. It is an object, not a row and table based structure. Then the output itself is also an object, in a completely different structure to what the database gave to you. People can use an ORM and not really know how SQL works if they never bother to learn. It is presenting them with an object-based database. In a NoSQL database like Mongo, you either have to literally store the Product with its parts and manufacturer, or you take them separately from the database and put them back together in code. This requires no abstraction. It is exactly what is happening.
- dang 8y ago
  Could you please stop creating new accounts for every few comments you post? We ban accounts that do this, and it's in the site guidelines: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html. It's particularly abusive that you used multiple accounts in this same thread. HN is a community. Obviously you don't have to use your real name, but if users don't have some consistent identity for others to relate to, we may as well have no usernames and no community at all. That would be quite a different kind of forum. Anonymity is fine, and throwaways for a specific purpose are ok—just not routinely. https://hn.algolia.com/?sort=byDate&dateRange=all&type=comment&storyText=false&prefix=false&page=0&query=by:dang%20community%20identity https://hn.algolia.com/?sort=byDate&dateRange=all&type=comme...
threeseed 8y ago
> ORMs don't try to make a SQL database something else This is simply not true. Many ORMs are designed to abstract away SQL and RDBMS concepts entirely. They deal in objects and object graphs and output SQL which can be quite disjointed from the object model e.g. many-many relationships.

deleted 8y ago

[deleted]

CuriousSkeptic 8y ago

Linq works for trivial left join project style of queries. But for anything actually using the features of modern sql-databases it comes short quickly. As a example try to express something like “sum(x) over (partition by y order by z)” I do agree that SQL the syntax leaves a lot to be desired, and a proper relational language with a syntax optimized for actual development, and even better, optimized for 6NF style databases, would be awesome, but linq is not by any stretch sufficient to be that

jeffreyrogers 8y ago
My understanding is everyone uses 3NF in practice though, so what would a 6NF database get you practically? I see SQL the way I see things like Linux: it's got some weird things that you'd design differently if you could do it over again, but overall it's pretty good and probably not worth the effort to switch.
- CuriousSkeptic 8y ago
  6NF gives flexibility since every thing is decoupled until you bring it in. It makes evolving the schema easier, makes optimizations easier. I also have a hunch that with such a focus it would be easier to find an interesting design space for new relational programming models since it’s, in a sense, “purer”. But I guess it depends on the use case. For normal oltp style save/fetch entity 3NF makes sense. But it kind of makes sense in the way an ORM would make sense. Which makes you question if you really need relational database at all.

matwood 8y ago

> Everybody uses an ORM with a SQL database. I'm ripping ORMs out of any backend code and using things like jOOQ. > I was just thinking to myself "I can write this in code in a few minutes instead of an hour or more".... > Joins are slow and complicated You know what's really slow? Creating joins in code. > This is not true at all. Programming languages are vastly superior at querying a database. Just go write a complex query and compare it to the one in linq or whatever you use. It makes zero sense to pull a bunch of records back from the db using multiple network calls to join and then filter. Let the db do its job.

mmt 8y ago
> It makes zero sense to pull a bunch of records back from the db using multiple network calls to join and then filter. Let the db do its job. But then it wouldn't be distributed processing! :) Seriously, though, consider it for a moment.. this pattern has similar features to something like Hadoop. The data comes from storage nodes (database server and, hopefully, their read replicas) and goes to processing nodes (app server) to have the work done and is then new data is written back out over the network to storage nodes and replicated across the network (to the replica/slave database servers). If the data volume is particularly low or the compute load (CPU and/or RAM) is particularly high, the distributed method would make intuitive sense. I haven't seen it yet, however.
- matwood 8y ago
  Touché. You're right, at some point the data must be joined and filtered. My point was to let the tool do its job :)
- jandrewrogers 8y ago
  Doing distributed joins correctly requires an architectural/technical capability that most distributed database engines don't have: decentralized parallel orchestration. If you have this, you can do joins even with very high data volumes efficiently given good parallel scheduling algorithms. Most databases are designed such that there is a single point of control that declaratively schedules all data flows required to execute the query; this scales poorly for operations like joins, never mind recursive joins, which is why you don't see it. Letting individual database nodes dynamically schedule and orchestrate their own data flows with each other, essentially allowing each node in the parallel system construct its own execution plan in relation to other nodes as it goes along, does not fit within the "giant distributed file system" paradigm that most distributed systems are based on. People who design codes for supercomputers are often familiar with parallel orchestration idioms that work at extremely large scales but it hasn't crossed over into ordinary distributed database engines. (This is also a good litmus test for what makes a database "parallel" as distinct from "distributed".) Most distributed database architectures are much more centralized than they need to be, particularly around control of execution planning, and this limits their expressiveness. It is quite difficult to hack together a distributed join that performs better than a centralized one without good support for parallel orchestration.

orf 8y ago

> The fact is that programming languages are much better at querying databases than SQL. They really are not though. Sure, perhaps for simple single table scans, bu nothing beats the pure optimizing potential of SQL. Try joining three tables in JS neatly and fast. It's a bit terse, but you're lying to yourself if you think hodgepodge mess of ad-hoc JavaScript written to poorly replicate a single specific query is in any way better than what we have now.

flavio81 8y ago

>Everybody uses an ORM with a SQL database. That's not true. >While I was writing these queries, I was just thinking to myself "I can write this in code in a few minutes instead of an hour or more" Well, no, because SQL is a 4th generation language of higher level than most general-purpose languages. It is a domain-specific language focused on database handling. A simple SELECT with a few joins and indexes involved encompasses a query execution plan that would be a few hundreds of code in a regular programming language. >I chose Mongo for a project recently. Storing relational data on a document store is a bad idea. And if you need a document store, Mongo is pretty much a bad option.

ruiquelhas 8y ago
Why is Mongo a bad option for a document store? Serious question.

collyw 8y ago

> Joins are slow and complicated, and ORMs are slow and complicated, so you can never actually use any of these things in a high-traffic environment. As you say, not many people will reach these levels of traffic, but it is still a factor. I have to clean up crap written by people like you. Go learn how to use a relational database properly instead of writing tosh like this and crap buggy code in the application layer.