6 ms·
Show HN: A Database Written in C
Hey hackernews! I'd like to share a hobby and passion project I've been working on for almost a year now. It's an open-source storage engine (underlying component of a database(s)) similar to that of LevelDB/RocksDB but written entirely in C.
The storage engine is called TidesDB.
TidesDB is written entirely in C. It's designed as a fast, transactional key-value storage engine built on a log-structured merge-tree (LSM-tree) architecture.
My journey with TidesDB began nearly 2 years ago whilst I was experimenting with various data structures and databases in Go. When I encountered the log-structured merge tree concept, I was initially overwhelmed by its complexity after reviewing other implementations.
However, after studying the original paper, I realized I could potentially simplify the design by focusing on just a 2-level approach(memory level and a disk level). This was challenging at first, and I scrapped many ideas along the way, but over time I discovered how powerful this design could potentially be.
The beauty of this design/architecture is its extensibility. Since the disk level contains many SSTables (Sorted String Tables), we can efficiently pair and merge them in various ways - whether in parallel for speed or incrementally to minimize resource impact.
What began as a challenging learning process has I believed evolved into a unique engine design and library.
You can check out TidesDB code here: https://github.com/tidesdb/tidesdb https://github.com/tidesdb/tidesdb
Website here (Architecture-design, documentation and more): https://tidesdb.com https://tidesdb.com
There are a variety of FFI libraries in the works for C++, GO, Python, Lua and more!
Currently TidesDB is nearing its first major release, we are still in beta development.
I'd love to get your thoughts, questions and all :)
- timfi 2y ago[flagged]
- alexpadula 2y agoTitles are hard on hackernews :S It's a skill I swear! "DB projects being implemented in C is nothing new, e.g., PostgreSQL is also dritten C." I get it may seem like something not new but it's a challenge which I thought would bring more attention to the post itself. I appreciate your comment.
- Rochus 2y agoThat's a cool project, thanks for sharing! Does string sorting also consider unicode (or did I get SSTables wrong)?
- alexpadula 2y agoHey! Thank you very much for the kind words. The system currently sorts comparing raw byte values directly. So if I'm understanding correctly, no it doesn't handle unicode sorting. If key's were stored as strings and compared that way, yes. :)
- Rochus 2y agoOk, I see. I implemented an object database based on the Sqlite backend some time ago for CrossLine and other applications, which includes ideas of MUMPS and also uses unicode collating algorithms implemented in Qt, but it's c++, see e.g. https://github.com/rochus-keller/Udb/blob/master/Idx.cpp https://github.com/rochus-keller/Udb/blob/master/Idx.cpp.
- alexpadula 2y agoOu!! really checking it out.
- deleted 2y ago[deleted]
- alexpadula 2y agoI must say just off the bat and a few minutes reviewing, I'm extremely impressed. I stared the repository to play with it further later.
- Rochus 2y agoI have been considering for some time whether I should change the backend. LSM trees and a much simpler implementation than, for example, RocksDb seem attractive.
- ivanimprv 2y agotried tidesdb with python and it's really fast. a transactional key-value database written in c with a modern lsm-tree architecture. worth checking out
- alexpadula 2y agoThank you for the kind words Ivan and your work on the Python FFI.