4 ms·

I think this is an unfair comparison as RedisGraph keeps everything in memory (graph index + graph entities) for the sake of performance, AFAIK Neo utilise disk

I think this is an unfair comparison as RedisGraph keeps everything in memory (graph index + graph entities) for the sake of performance, AFAIK Neo utilise disks.

antirez 9y ago
This makes sense however even on-disk stores written in Java sometimes have non trivial memory usage characteristics. ElasticSearch is an example, so I would check what happens in the real world. Well of course an in memory system will always end having a memory usage proportional to the graph size...
- yehosef 9y ago
  For many use cases, ES has great memory characteristics because the doc values, which is now default. https://www.elastic.co/guide/en/elasticsearch/reference/current/doc-values.html https://www.elastic.co/guide/en/elasticsearch/reference/curr... As an example, I have the HN data set on my laptop - about 15 GB of data and I have the max mem set to 3GB (heap is usually ~1) and I can search, aggregate, etc. very quickly without memory problems.
  - antirez 9y ago
    Sure I don't want to say that on-disk systems always need to have the whole set of pages in memory, but depending on the performances that you want to obtain, at least a percentage of the pages will need to be on memory, so if one has the illusion that: in-memory systems -> memory usage proportional to the amount of data. in-disk systems -> fixed memory usage, just more disk, this is usually not true.
swilly2 9y ago
In addition to my previous comment I would like to add that RedisGraph memory usage is proportional to the size of the graph stored, AFAIK Neo utilizes disk as the main storage, however the Java VM may have objects in memory in order to traverse the graph, so may also use some memory. I've never tested the memory usage difference between the two, but it should be an interesting test to perform. Generally speaking it seems like a common practice now days to load large portions of your data from disk to memory for performance reasons obviously, no one is willing to pay the penalty hit of disk seek when it comes to real time data querying.

I think this is an unfair comparison as RedisGraph keeps everything in memory (graph index + graph entities) for the sake of performance, AFAIK Neo utilise disks.

antirez 9y ago

This makes sense however even on-disk stores written in Java sometimes have non trivial memory usage characteristics. ElasticSearch is an example, so I would check what happens in the real world. Well of course an in memory system will always end having a memory usage proportional to the graph size...

yehosef 9y ago
For many use cases, ES has great memory characteristics because the doc values, which is now default. https://www.elastic.co/guide/en/elasticsearch/reference/current/doc-values.html https://www.elastic.co/guide/en/elasticsearch/reference/curr... As an example, I have the HN data set on my laptop - about 15 GB of data and I have the max mem set to 3GB (heap is usually ~1) and I can search, aggregate, etc. very quickly without memory problems.
- antirez 9y ago
  Sure I don't want to say that on-disk systems always need to have the whole set of pages in memory, but depending on the performances that you want to obtain, at least a percentage of the pages will need to be on memory, so if one has the illusion that: in-memory systems -> memory usage proportional to the amount of data. in-disk systems -> fixed memory usage, just more disk, this is usually not true.

swilly2 9y ago

In addition to my previous comment I would like to add that RedisGraph memory usage is proportional to the size of the graph stored, AFAIK Neo utilizes disk as the main storage, however the Java VM may have objects in memory in order to traverse the graph, so may also use some memory. I've never tested the memory usage difference between the two, but it should be an interesting test to perform. Generally speaking it seems like a common practice now days to load large portions of your data from disk to memory for performance reasons obviously, no one is willing to pay the penalty hit of disk seek when it comes to real time data querying.