3 ms·
Some initial thoughts from looking at the website and documentation... 1. The main reason I want SQL is for relational data. The lack of joins basically makes
by danpalmer 13y ago
Some initial thoughts from looking at the website and documentation...
1. The main reason I want SQL is for relational data. The lack of joins basically makes this a "NoSQL" database in every respect except the query language being something resembling SQL. I'm fairly sure ANSI SQL requires support for joins.
2. It does auto-sharding, but I don't know how. The documentation doesn't specify how the data is sharded, and this is quite important. Range-based sharding, like MongoDB uses by default, is often not what users want (depends on use-case), so if it's that, we need to know. Whatever it is, there are trade-offs with different approaches and that's something users need to take account of.
3. You can't change the shard cluster size after initially sharding. I assume this is a planned feature, but until then, it's probably not ready for production use.
4. You can only shard on the primary key, if you have a primary key.
5. The configuration for the number of replicas is confusing, and appears to not be very configurable.
A brief read makes this seem to be basically MongoDB from its early days, with many of the disadvantages, but some advantages like custom analysers which appear to replace Mongo's map reduce, that can be queried by an SQL-like syntax.
When I saw this, I thought it was going to be a relational database that did auto-sharding and replication. That would have been great. Unfortunately it's not. It might become that a few years down the line, but right now, I'm not inspired by it.
- rburhum 13y ago> I'm fairly sure ANSI SQL requires support for joins Yup. The older SQL-92 spec BNF definition for SELECT clauses: http://savage.net.au/SQL/sql-92.bnf.html#query%20specification http://savage.net.au/SQL/sql-92.bnf.html#query%20specificati...
- ddorian43 13y agoits actually sql on top of elasticsearch
- coolsunglasses 13y agoThat is bad. I've done a lot of work with Elasticsearch[1] and while it's a great search engine, it is NOT a primary source of truth or something you want to trust not to lose your data. [1]: Built a startup's product on top of it and have written an open source client library for it.
- ddorian43 13y agowhy ? at least i remember they didn't recommend it for database until the backup funcionality was done
- coolsunglasses 13y agoMost people aren't good enough at backups for me to consider that enough to mitigate the risks. The data needs to go somewhere other than ES before you considered it "saved".
- jodok 13y agothanks for your summary and valuable feedback. a few notes from the crate team: 1. JOINs are on our roadmap. we might never reach full JOIN support anytime soon (INNER, LEFT OUTER, RIGHT OUTER, FULL OUTER and CROSS), but simple use-cases (1:n), equal joins shouldn't be too hard. 2. after re-reading our documentation on sharding i need to admit that we need to improve it. we mention it briefly here https://crate.io/docs/current/sql/ddl.html#sharding https://crate.io/docs/current/sql/ddl.html#sharding, more will be added asap 3. if we have growing datasets we typically work with parted tables. we just pushed that change just a few days ago (https://github.com/crate/crate/blob/master/docs/sql/partitioned_tables.txt https://github.com/crate/crate/blob/master/docs/sql/partitio...), not yet in the RPM build. 4. no, you can specify routing https://crate.io/docs/current/sql/ddl.html#routing https://crate.io/docs/current/sql/ddl.html#routing 5. agreed, that the doc can be improved. and we're at the very beginning https://crate.io/docs/current/sql/ddl.html#replication https://crate.io/docs/current/sql/ddl.html#replication - rack awareness,... there are quite some replica settings on our roadmap. i'd be happy to hear your most important additions to replica policies. i promise, we're working hard and try to be there faster than in a few years :)
- girvo 13y agoEven if you can get basic JOINs going, then I can see your tool becoming super useful in a lot of my use cases :)
- danpalmer 13y agoThanks for the reply! Basic JOIN support would be good, but I think it is misleading to advertise Crate as having SQL support, because I think most users would assume that the SQL everyone knows, ANSI SQL, would be supported, which it definitely isn't yet. As I said, sharding is mentioned, but no details about how this is actually implemented. A potential user could check in the code, but realistically this is unlikely as many users have probably not given a huge amount of thought to how the routing is done to different shards. It's a really important issue, because if you're doing range-based sharding like MongoDB does by default, that changes the kind of key that it should be partitioned on. With regards to point 4, your documentation says "If a primary key constraint is defined, the routing column definition can be omitted or must match a primary key column." I read this as if there is a primary key defined, the routing column must match it, or can just not be specified, or more simply, you can only shard on the primary key. If this isn't the case, I think this needs re-wording. In terms of replication, the explanation of the ranges of replicas is confusingly worded. I was wondering what the use-case for this is? Surely the idea of replicas is to determine how many node failures you want to support, and then set the count at the minimum number required to support this so as not to waste resources. Also, if you set a range, how does Crate determine where in that range to set the number of replicas? Is it as many as possible?