5 ms·

Congratulations on the funding! I don't understand under what specific use case ArangoDB works best; the comparisons section lists Cassandra and Neo4j, and my

Congratulations on the funding! I don't understand under what specific use case ArangoDB works best; the comparisons section lists Cassandra and Neo4j, and my understanding was Cassandra was for something like chat apps and Neo4j was for something like GIS analytics. Enlighten me? Also, how convertible is a proprietary query language like AQL/CQL to SQL? Is it fully declarative, and version completely independently of the database core?

graetzer 8y ago
You should certainly be able to use ArangoDB as a replacement for Cassandra or Neo4j. Check out the SQL / AQL comparison page to get a quick overview: https://www.arangodb.com/why-arangodb/sql-aql-comparison/ https://www.arangodb.com/why-arangodb/sql-aql-comparison/
jsteemann 8y ago
Regarding the question on the query language, AQL is fully declarative. In this respect it is like SQL. However, there are a few differences between AQL and SQL: * SQL is an all-purpose database management and querying language. It is very complex and heavy-weight as it has to solve a lot of different problems, e.g. data definition, data retrieval and manipulation, stored procedures etc. AQL is much more lightweight, as its purpose is querying and manipulating database data. Any data definition or database administration commands are not part of AQL, but can be achieved using other, dedicated commands/APIs. * for data retrieval and manipulation, the functionality of SQL and AQL do overlap a lot, but they use different keywords for the similar things. Still simple SQL queries can be converted to AQL easily and vice versa. There are some specialized parts of AQL, such as graph traversals and shortest path queries, for which may be no direct equivalent in SQL. AQL is versioned along with the database core, as sometimes features are added to AQL which the database core must also support and vice versa. However, during further development of AQL and the database core, one of the major goals is to keep it always downwards-compatible, meaning that existing AQL queries are expected to work and behave identically in newer versions of the database (but ideally run faster or are better optimized there).
- PaulHoule 8y ago
  Aql is similar to n1so from couchbase. Imagine if SQL was designed around Json instead of.rows and that is what aql is.
  - jsteemann 8y ago
    Somewhat at least... N1QL tries to be more close to SQL in terms of keywords and such, whereas the AQL approach was to pick different keywords than SQL. Apart from the difference in keywords I tend to agree.
- yingw787 8y ago
  Okay, I like how backwards compatibility is preserved. I worked with mongoDB at my previous company and we ended up not being able to migrate to mongoDB 3.x. I think it was because we forked 'eve-mongoengine' and couldn't merge upstream changes, which ended up forcing us to version the entire stack through the database at the same time, which passed the threshold of feasibility. We were absolute idiots, but I still think a data warehouse should be idiot-proof, which is why I like SQL. I read through the documentation for ArangoDB and I would be concerned about the lack of native strict type definitions and referencing in AQL, as well as the dearth of type availability in ArangoDB in general. Is this a design decision related to not supporting data/database administration, or something to be added later to the roadmap? It sounds like if you support write-intensive paths through the database, it would be considered an OLTP database for some OLTP workloads; do you publish TPC-C benchmarks anywhere? What about resource utilization? Is there a particular reason to support JavaScript first? Is it because Swagger has JavaScript-first support, or a different reason?
  - jsteemann 8y ago
    ArangoDB is a schema-less database. There is currently no support for schemas or schema validation on the database core level, but it may be added later, because IMHO it is a very sensible feature. When that is in place, AQL may also be extended to get more strict about the types used in queries. However, IMHO that should only be enforced if there is a schema present. To keep things simple and manageable, we originally started with AQL just being a language for querying the database. It was extended years ago to support data manipulation operations. I don't exclude the possibility that at some point it will support database administration or DDL commands, however, I am just one of the developers and not the product manager. And you are right about the main use case being OLTP workloads. For OLAP use cases, dedicated analytical databases (with fixed data schemas) are probably superior, because they can run much more specialized and streamlined operations on the data. To my best knowledge we never published any TPC benchmark results somewhere. I think it's possible to implement TPC-C even without SQL, however, implementing the full benchmark is a huge amount of work, so we never did...
jsteemann 8y ago
ArangoDB is a multi-model database so it tries to target several use cases. It provides functionality working with key-values, documents, graphs and fulltext indexing/searching. It provides some flexibility in the sense that it does not force you into a specific way of working with the data. For example, it does not force you to treat each use case as a graph use case. This is in contrast to some other specialized databases, which excel at their specific area, but also force you to completely adopt the type of data-modeling they support.
- dmix 8y ago
  So what’s the downside of using this DB which does it all vs using a specialized DB? Scaling, performance, etc?
  - jsteemann 8y ago
    Specialized databases have the advantage of being, well, specialized... For example, a specialized OLAP database which knows about the schema of the data can employ much more streamlined storage and query operators, so it should have a "natural" performance advantage. However, a very specialized database may later lock you in to something, and in case you need something different, you will end up with running multiple different special-purpose databases. Not saying this is necessarily bad (or good), but it is at least one aspect to consider how many different databases to you want to operate & manage in your stack.
    - dmix 8y ago
      Interesting. Pretty much every startup I've worked for has run 2-3 databases. Usually Redis plus some search (typically Elastic now). I could see this making that easier.
  - janemanos 8y ago
    Think we have to be a bit more precise here. ArangoDB supports documents, key/value, and graph. It is not really optimized for large timeseries use cases which might need windowing or other features. Influx or Timescale might provide better characteristics here. However, for the supported data models we found a way to combine them quite efficiently. Many search engines access data stored in JSON format. Hence integrating a search engine like ArangoSearch as an additional layer on top of the existing data models is no magic but makes a lot of sense. Allowing to combine models with search is then rather an obvious task for us.
PeterCorless 8y ago
As an aside, Apache Cassandra CQL is now used by a growing number of wire-compliant databases: • Cassandra • Scylla (full disclosure: I work for ScyllaDB) • DataStax Enterprise (DSE) • Cosmos DB • Yugabyte If you want to learn more about CQL: https://docs.scylladb.com/using-scylla/cql/ https://docs.scylladb.com/using-scylla/cql/ If you want to learn how it's different, here's a quick StackOverflow answer: https://stackoverflow.com/a/19140553/6995180 https://stackoverflow.com/a/19140553/6995180
PeterCorless 8y ago
Also, Cassandra, and Cassandra-like databases (like Scylla) are capable of far more than 'chat apps.' There are a lot of IoT, adtech, and other use cases. I just published this blog today: https://www.scylladb.com/2019/03/14/from-sap-to-scylla-tracking-the-fleet-at-gps-insight/ https://www.scylladb.com/2019/03/14/from-sap-to-scylla-track... (Apologies for coming in sideways to this thread. Hat's off to ArangoDB, and all in the NoSQL arena who are pushing the envelope in terms of new Big Data solutions.)

graetzer 8y ago

You should certainly be able to use ArangoDB as a replacement for Cassandra or Neo4j. Check out the SQL / AQL comparison page to get a quick overview: https://www.arangodb.com/why-arangodb/sql-aql-comparison/ https://www.arangodb.com/why-arangodb/sql-aql-comparison/

jsteemann 8y ago

Regarding the question on the query language, AQL is fully declarative. In this respect it is like SQL. However, there are a few differences between AQL and SQL: * SQL is an all-purpose database management and querying language. It is very complex and heavy-weight as it has to solve a lot of different problems, e.g. data definition, data retrieval and manipulation, stored procedures etc. AQL is much more lightweight, as its purpose is querying and manipulating database data. Any data definition or database administration commands are not part of AQL, but can be achieved using other, dedicated commands/APIs. * for data retrieval and manipulation, the functionality of SQL and AQL do overlap a lot, but they use different keywords for the similar things. Still simple SQL queries can be converted to AQL easily and vice versa. There are some specialized parts of AQL, such as graph traversals and shortest path queries, for which may be no direct equivalent in SQL. AQL is versioned along with the database core, as sometimes features are added to AQL which the database core must also support and vice versa. However, during further development of AQL and the database core, one of the major goals is to keep it always downwards-compatible, meaning that existing AQL queries are expected to work and behave identically in newer versions of the database (but ideally run faster or are better optimized there).

PaulHoule 8y ago
Aql is similar to n1so from couchbase. Imagine if SQL was designed around Json instead of.rows and that is what aql is.
- jsteemann 8y ago
  Somewhat at least... N1QL tries to be more close to SQL in terms of keywords and such, whereas the AQL approach was to pick different keywords than SQL. Apart from the difference in keywords I tend to agree.
yingw787 8y ago
Okay, I like how backwards compatibility is preserved. I worked with mongoDB at my previous company and we ended up not being able to migrate to mongoDB 3.x. I think it was because we forked 'eve-mongoengine' and couldn't merge upstream changes, which ended up forcing us to version the entire stack through the database at the same time, which passed the threshold of feasibility. We were absolute idiots, but I still think a data warehouse should be idiot-proof, which is why I like SQL. I read through the documentation for ArangoDB and I would be concerned about the lack of native strict type definitions and referencing in AQL, as well as the dearth of type availability in ArangoDB in general. Is this a design decision related to not supporting data/database administration, or something to be added later to the roadmap? It sounds like if you support write-intensive paths through the database, it would be considered an OLTP database for some OLTP workloads; do you publish TPC-C benchmarks anywhere? What about resource utilization? Is there a particular reason to support JavaScript first? Is it because Swagger has JavaScript-first support, or a different reason?
- jsteemann 8y ago
  ArangoDB is a schema-less database. There is currently no support for schemas or schema validation on the database core level, but it may be added later, because IMHO it is a very sensible feature. When that is in place, AQL may also be extended to get more strict about the types used in queries. However, IMHO that should only be enforced if there is a schema present. To keep things simple and manageable, we originally started with AQL just being a language for querying the database. It was extended years ago to support data manipulation operations. I don't exclude the possibility that at some point it will support database administration or DDL commands, however, I am just one of the developers and not the product manager. And you are right about the main use case being OLTP workloads. For OLAP use cases, dedicated analytical databases (with fixed data schemas) are probably superior, because they can run much more specialized and streamlined operations on the data. To my best knowledge we never published any TPC benchmark results somewhere. I think it's possible to implement TPC-C even without SQL, however, implementing the full benchmark is a huge amount of work, so we never did...

jsteemann 8y ago

ArangoDB is a multi-model database so it tries to target several use cases. It provides functionality working with key-values, documents, graphs and fulltext indexing/searching. It provides some flexibility in the sense that it does not force you into a specific way of working with the data. For example, it does not force you to treat each use case as a graph use case. This is in contrast to some other specialized databases, which excel at their specific area, but also force you to completely adopt the type of data-modeling they support.

dmix 8y ago
So what’s the downside of using this DB which does it all vs using a specialized DB? Scaling, performance, etc?
- jsteemann 8y ago
  Specialized databases have the advantage of being, well, specialized... For example, a specialized OLAP database which knows about the schema of the data can employ much more streamlined storage and query operators, so it should have a "natural" performance advantage. However, a very specialized database may later lock you in to something, and in case you need something different, you will end up with running multiple different special-purpose databases. Not saying this is necessarily bad (or good), but it is at least one aspect to consider how many different databases to you want to operate & manage in your stack.
  - dmix 8y ago
    Interesting. Pretty much every startup I've worked for has run 2-3 databases. Usually Redis plus some search (typically Elastic now). I could see this making that easier.
- janemanos 8y ago
  Think we have to be a bit more precise here. ArangoDB supports documents, key/value, and graph. It is not really optimized for large timeseries use cases which might need windowing or other features. Influx or Timescale might provide better characteristics here. However, for the supported data models we found a way to combine them quite efficiently. Many search engines access data stored in JSON format. Hence integrating a search engine like ArangoSearch as an additional layer on top of the existing data models is no magic but makes a lot of sense. Allowing to combine models with search is then rather an obvious task for us.

PeterCorless 8y ago

As an aside, Apache Cassandra CQL is now used by a growing number of wire-compliant databases: • Cassandra • Scylla (full disclosure: I work for ScyllaDB) • DataStax Enterprise (DSE) • Cosmos DB • Yugabyte If you want to learn more about CQL: https://docs.scylladb.com/using-scylla/cql/ https://docs.scylladb.com/using-scylla/cql/ If you want to learn how it's different, here's a quick StackOverflow answer: https://stackoverflow.com/a/19140553/6995180 https://stackoverflow.com/a/19140553/6995180

PeterCorless 8y ago

Also, Cassandra, and Cassandra-like databases (like Scylla) are capable of far more than 'chat apps.' There are a lot of IoT, adtech, and other use cases. I just published this blog today: https://www.scylladb.com/2019/03/14/from-sap-to-scylla-tracking-the-fleet-at-gps-insight/ https://www.scylladb.com/2019/03/14/from-sap-to-scylla-track... (Apologies for coming in sideways to this thread. Hat's off to ArangoDB, and all in the NoSQL arena who are pushing the envelope in terms of new Big Data solutions.)