3 ms·

One question I really have is: for the exact same dataset, how much RAM is used to handle it with MongoDB, Redis, MySQL, or even a column-based store. If someo

One question I really have is: for the exact same dataset, how much RAM is used to handle it with MongoDB, Redis, MySQL, or even a column-based store. If someone has such a benchmark already, I'd love to read it.

otterley 16y ago
There's no way to know, really. When you mmap() backing stores into process memory as MongoDB and Redis do, the OS takes care of paging activity for you. And they get to compete with other processes on the system for memory pages. InnoDB (MySQL) allows you to fix its buffer pool in RAM, so it makes decisions as to which pages to buffer instead of the OS. Sometimes it makes better decisions, sometimes it doesn't.
- thibaut_barrere 16y ago
  I had a couple of ideas: given a machine (or a VM) with an amount of RAM which is enough to handle all the db in memory (eg: 8GB for a 512MB dataset), maybe it's possible to track the OS overall used RAM and have a rough idea ? One other way would be to have a more intimate knowledge of how the stores work and make a computation: for instance when some column stores, you can compute the bytes taken by column if you know how the inner engine works.
  - otterley 16y ago
    You can use the VSZ of the process (from ps(1)) as a rough approximation of memory utilization. This is what the process maximally could address (without mapping or allocating any additional memory) at the moment of measurement, and may include pages that are in fact not in memory at the time (because they might be references to pages that are on files or swap). RSS tells you what is currently "resident" (i.e. in RAM) but doesn't include pages that are in files or swap. Personally, I'm much more interested in flows (paging and I/O activity) than stocks (how many pages a process has allocated). If I'm comparing two competing platforms with the same durability guarantees on identical hardware using identical data access patterns, the one with the lower paging/swapping activity wins. It's not always the one with the smaller memory utilization!
- nkurz 16y ago
  While MongoDB does this, I don't think that Redis takes this approach. In theory, it manages the memory itself, reading from disk into its own cache. In practice, I worry that it might end up competing with the system, leading to "full memory" like conditions far before memory is actually exhausted. Thus my interest in how it performs in a simple real life test like this.
  - antirez 16y ago
    No mmap in Redis indeed.
    - otterley 16y ago
      My mistake.
  - otterley 16y ago
    On Linux, at least, that's probably better than mmap()ing pages, since you get better control over when the backing store is synced. Nevertheless, the risk still exists, as you suggest, that Redis's memory pages could be involuntarily swapped to disk due to memory pressure caused by competing processes.
gvp 16y ago
The big problem with this question is "what's the same dataset"? The same dataset can be structured multiple different ways. In fact, if your data structure in MongoDB is the same as it is in MySQL, then you're probably structuring it incorrectly. By the same measure, what are you then testing for? Are you testing for lots of reads, lots of writes? What type of reads? How many servers are you testing across? MongoDB makes it really easy to denormalize a data structure, so it's easy to "cheat" on the 99% query and perform poorly on the 1% query. This is probably what you want from a real-world performance stand-point, but you have to be really careful about factoring that into your test weightings. Either way, I'd love to read such a benchmark as well. But to do any justice to the sheer complexity of the task, it probably needs to be a short book.

otterley 16y ago

There's no way to know, really. When you mmap() backing stores into process memory as MongoDB and Redis do, the OS takes care of paging activity for you. And they get to compete with other processes on the system for memory pages. InnoDB (MySQL) allows you to fix its buffer pool in RAM, so it makes decisions as to which pages to buffer instead of the OS. Sometimes it makes better decisions, sometimes it doesn't.

thibaut_barrere 16y ago
I had a couple of ideas: given a machine (or a VM) with an amount of RAM which is enough to handle all the db in memory (eg: 8GB for a 512MB dataset), maybe it's possible to track the OS overall used RAM and have a rough idea ? One other way would be to have a more intimate knowledge of how the stores work and make a computation: for instance when some column stores, you can compute the bytes taken by column if you know how the inner engine works.
- otterley 16y ago
  You can use the VSZ of the process (from ps(1)) as a rough approximation of memory utilization. This is what the process maximally could address (without mapping or allocating any additional memory) at the moment of measurement, and may include pages that are in fact not in memory at the time (because they might be references to pages that are on files or swap). RSS tells you what is currently "resident" (i.e. in RAM) but doesn't include pages that are in files or swap. Personally, I'm much more interested in flows (paging and I/O activity) than stocks (how many pages a process has allocated). If I'm comparing two competing platforms with the same durability guarantees on identical hardware using identical data access patterns, the one with the lower paging/swapping activity wins. It's not always the one with the smaller memory utilization!
nkurz 16y ago
While MongoDB does this, I don't think that Redis takes this approach. In theory, it manages the memory itself, reading from disk into its own cache. In practice, I worry that it might end up competing with the system, leading to "full memory" like conditions far before memory is actually exhausted. Thus my interest in how it performs in a simple real life test like this.
- antirez 16y ago
  No mmap in Redis indeed.
  - otterley 16y ago
    My mistake.
- otterley 16y ago
  On Linux, at least, that's probably better than mmap()ing pages, since you get better control over when the backing store is synced. Nevertheless, the risk still exists, as you suggest, that Redis's memory pages could be involuntarily swapped to disk due to memory pressure caused by competing processes.

gvp 16y ago

The big problem with this question is "what's the same dataset"? The same dataset can be structured multiple different ways. In fact, if your data structure in MongoDB is the same as it is in MySQL, then you're probably structuring it incorrectly. By the same measure, what are you then testing for? Are you testing for lots of reads, lots of writes? What type of reads? How many servers are you testing across? MongoDB makes it really easy to denormalize a data structure, so it's easy to "cheat" on the 99% query and perform poorly on the 1% query. This is probably what you want from a real-world performance stand-point, but you have to be really careful about factoring that into your test weightings. Either way, I'd love to read such a benchmark as well. But to do any justice to the sheer complexity of the task, it probably needs to be a short book.