5 ms·
mmap'd databases are really quick to implement. I implemented both row and column orientated databases. The traders and quants loved it - and adoption took off
by vii 6y ago
mmap'd databases are really quick to implement. I implemented both row and column orientated databases. The traders and quants loved it - and adoption took off after we built a web interface that let you see a whole day and also zoom into exact trades with 100ms load times for even the most heavily traded symbols.
The benefits of mmaping and in general POSIX filesystem atomic properties are quick implementation, where you don't have to worry about buffer management. The filesystem and disk block remapping layer (in SSD or even HDDs now) are radically more efficient when data are given to them in contiguous large chunks. This is difficult to control with mmap where the OS may write out pages at its whim. However, even using advanced Linux system calls like mremap and fallocate, which try to improve the complexity of changing mappings and layout in the filesystem, eventually this lack of control over buffers will bite you.
And then when you look at it, the kernel (with help from the processor TLB) has to maintain complex data-structures to represent the mappings and their dirty/clean states. Accessing memory is not O(1) even when it is in RAM. Making something better tuned to a database than the kernel page management is a significant hurdle but that's where there are opportunities.
- bluestreak 6y agothank you for sharing! The core of memory management is abstracted away. All of the query execution logic is unaware of the source of memory pointer. That said we are still learning and really appreciate your feedback. There are some places where we could not beat aggregation of julia, but the delta wasn't very big. This could have been down to mapped memory. We will definitely try things with direct memory too!
- vii 6y agoThe databases I implemented experimented with various ways to compile queries. Turns out that the JVM can run quite fast. I feel like LLVM (Julia) is likely to be able to be better for throughput and definitely better for predictability of performance. If you know layouts and sizes, then your generated code can run really fast - using SIMD and not checking bounds is a win. Hugepages would greatly reduce pagetable bookkeeping, but obviously may magnify writes. Wish I could have tried that!
- bluestreak 6y agoOur best performance currently is in C++ code and LLVM is something we are considering using to compute expressions, such as predicates and select clause items. This is most likely to be way faster than what we can currently do in Java. What I would like to know if LLVM can optimize all the way to AVX512? We also need to experiment with hugepages. The beauty is that if read and write are separated - there is no issue with writes. They can still use 4k pages!
- polskibus 6y agoPlease consider going through Andy Pavlo's superb video courses on building databases. http://www.cs.cmu.edu/~pavlo/ http://www.cs.cmu.edu/~pavlo/ He discusses many common pitfalls and architectural decisions that impact database quality and performance, including map and why it's bad.