4 ms·

The entire offering looks very compelling. Unfortunately I couldn't get a sense of operational efficiency or performance of ArangoDB in the wild. I couldn't con

The entire offering looks very compelling. Unfortunately I couldn't get a sense of operational efficiency or performance of ArangoDB in the wild. I couldn't convince our CTO if we adopted this product, would we be trading the extra development agility of the mixed models: graph, aql and kv for an overly complex babysitting problem. Can anyone here speak about their experience using ArangoDb in a multi tenant SaaS product? How is it to manage your own cluster, backups, etc?

roflc0ptic 8y ago
I lead our database team for a while, and picked ArangoDB. We started to regret it pretty soon thereafter. While I was there, we found that our cluster would just die. I think they got put fix in for that, though my memory is hazy. After I left, they came to really regret it. My team was working at scale, and basically found themselves doing QA work, trouble shooting with the Arango team. To their credit, the Arango crew was extremely responsive and helpful. Maybe they've fixed things up since then; it's been a year and a half. At this point, I would hard pass on any database whose name doesn't start with PostgreSQL. Just got burned too hard. Graph databases ostensibly let you write queries that would otherwise be unwieldy, but it turns out PostgreSQL's `recursive` keyword lets you achieve roughly the same things, sans having to learn a whole new query language.
- mr_toad 8y ago
  > PostgreSQL's `recursive` keyword lets you achieve roughly the same things Or you could run GraphQL on top of PostgreSQL. Just because you can write anything in SQL doesn’t mean you should.
  - pluma 8y ago
    GraphQL has nothing to do with graph queries though. GraphQL is called that because it was created as a query language for Facebook's "social graph". It actually doesn't provide any graph operations or recursion (i.e. you explicitly tell it how many levels deep you want to go). You can provide a GraphQL interface on top of any backend or database, though.
- sv12l 8y ago
  Do you mind telling the issues you faced with ArangoDB? Coz we are evaluating few options currently and Arango is one of them.
  - roflc0ptic 8y ago
    I mentioned that the cluster would freeze. The other issue I remember vividly (less important, but irksome) was that the join query `for i in collection1: for j in collection2: filter ... ` executed way, way slower than the equivalent query: `col1 = for i in collection1 for j in collection2: filter ... ` I kept in touch with one of the directors, and after I left he mentioned a couple of things - they found that doing joins returning large amounts of data (maybe 10k records, IIRC) was prohibitively slow. They also found that under certain conditions, with a certain amount of data in the database, it would crash. He didn't ever describe the conditions. They switched to couchbase, and have reported being happy with it.
    - jsteemann 8y ago
      Being one of the developers of ArangoDB, I would like to use the chance to reply to this as well. I think there have been various issues with the cluster stability 1.5 years ago, and since then we have put great efforts into making the database much more robust and faster. Many man-years have been dedicated to this since 2017. 1.5 years ago we were shipping release 3.1, which is out of service already. Since then, we have released * ArangoDB 3.2: this release provided the RocksDB storage engine, which improves parallelism and memory management compared to our traditional mostly-memory storage engine * ArangoDB 3.3: with a new deployment mode (active failover), plus ease-of-use and replication improvements (e.g. cross-datacenter replication) * ArangoDB 3.4: latest release, for which we put great emphasis on performance improvements, namely for the RocksDB storage engine (which now also is the default engine in ArangoDB) In all of the above releases we also worked on improving AQL query execution plans, in order to make queries perform faster in both single server and cluster deployments. Working on the query optimizer and query execution plan improvements is obviously a never-ending task, and not only did we achieved a lot here since 2017, but we still have a lot of ideas for further improvements in this area. So there are more improvements to be expected for the following releases. All that said, I think it is clear now that my intention is to show that things should have improved a lot compared to the situation 1.5 y ago, and that we will always be working hard to make ArangoDB a better product.
- josiahpeters 8y ago
  Thanks for the insight, that was my fear. Our decision came down to the notion of "no one ever got fired for recommending SQL Server or PostgreSQL"
- ifcologne 8y ago
  Yes, we had cluster stability issues 1.5 years ago. That has changed, cluster stability and performance has the top priority and we invest a lot to improve the developer and devops experience with every release. Now, e.g. with K8s deployments or the arangodb starter, it's much easier to run and maintain clusters. Hope you find the time to give it a second try.

roflc0ptic 8y ago

I lead our database team for a while, and picked ArangoDB. We started to regret it pretty soon thereafter. While I was there, we found that our cluster would just die. I think they got put fix in for that, though my memory is hazy. After I left, they came to really regret it. My team was working at scale, and basically found themselves doing QA work, trouble shooting with the Arango team. To their credit, the Arango crew was extremely responsive and helpful. Maybe they've fixed things up since then; it's been a year and a half. At this point, I would hard pass on any database whose name doesn't start with PostgreSQL. Just got burned too hard. Graph databases ostensibly let you write queries that would otherwise be unwieldy, but it turns out PostgreSQL's `recursive` keyword lets you achieve roughly the same things, sans having to learn a whole new query language.

mr_toad 8y ago
> PostgreSQL's `recursive` keyword lets you achieve roughly the same things Or you could run GraphQL on top of PostgreSQL. Just because you can write anything in SQL doesn’t mean you should.
- pluma 8y ago
  GraphQL has nothing to do with graph queries though. GraphQL is called that because it was created as a query language for Facebook's "social graph". It actually doesn't provide any graph operations or recursion (i.e. you explicitly tell it how many levels deep you want to go). You can provide a GraphQL interface on top of any backend or database, though.
sv12l 8y ago
Do you mind telling the issues you faced with ArangoDB? Coz we are evaluating few options currently and Arango is one of them.
- roflc0ptic 8y ago
  I mentioned that the cluster would freeze. The other issue I remember vividly (less important, but irksome) was that the join query `for i in collection1: for j in collection2: filter ... ` executed way, way slower than the equivalent query: `col1 = for i in collection1 for j in collection2: filter ... ` I kept in touch with one of the directors, and after I left he mentioned a couple of things - they found that doing joins returning large amounts of data (maybe 10k records, IIRC) was prohibitively slow. They also found that under certain conditions, with a certain amount of data in the database, it would crash. He didn't ever describe the conditions. They switched to couchbase, and have reported being happy with it.
  - jsteemann 8y ago
    Being one of the developers of ArangoDB, I would like to use the chance to reply to this as well. I think there have been various issues with the cluster stability 1.5 years ago, and since then we have put great efforts into making the database much more robust and faster. Many man-years have been dedicated to this since 2017. 1.5 years ago we were shipping release 3.1, which is out of service already. Since then, we have released * ArangoDB 3.2: this release provided the RocksDB storage engine, which improves parallelism and memory management compared to our traditional mostly-memory storage engine * ArangoDB 3.3: with a new deployment mode (active failover), plus ease-of-use and replication improvements (e.g. cross-datacenter replication) * ArangoDB 3.4: latest release, for which we put great emphasis on performance improvements, namely for the RocksDB storage engine (which now also is the default engine in ArangoDB) In all of the above releases we also worked on improving AQL query execution plans, in order to make queries perform faster in both single server and cluster deployments. Working on the query optimizer and query execution plan improvements is obviously a never-ending task, and not only did we achieved a lot here since 2017, but we still have a lot of ideas for further improvements in this area. So there are more improvements to be expected for the following releases. All that said, I think it is clear now that my intention is to show that things should have improved a lot compared to the situation 1.5 y ago, and that we will always be working hard to make ArangoDB a better product.
josiahpeters 8y ago
Thanks for the insight, that was my fear. Our decision came down to the notion of "no one ever got fired for recommending SQL Server or PostgreSQL"
ifcologne 8y ago
Yes, we had cluster stability issues 1.5 years ago. That has changed, cluster stability and performance has the top priority and we invest a lot to improve the developer and devops experience with every release. Now, e.g. with K8s deployments or the arangodb starter, it's much easier to run and maintain clusters. Hope you find the time to give it a second try.