5 ms·
This is a good write up. Many years ago I decided to code/program a log based db, to see how far can I go. It was supposed to be Light-weight (Berkley DB was
by ardit33 5y ago
This is a good write up. Many years ago I decided to code/program a log based db, to see how far can I go. It was supposed to be Light-weight (Berkley DB was the inspiration). It had a working stream to file for all changes, and a compaction thread in the background.
What I learned... creating the main/basic features of non-relational database (log based), is easy... but it is the edge cases that hit you hard and dealing with them creates about 90% of the code.
It was a fun exercise from which I learned a lot.
- killingtime74 5y agoHey if you don't mind what are some examples of edge cases?
- ardit33 5y agoSaturation of CPU resources Making sure the in-memory cache doesn't get too big Recovery when something goes wrong (computer loses power) Recovery of data when there is a partial HDD failure Compaction creating performance problems Handling of duplicate requests/out of order request etc.. etc... Basically performance and readability and data consistency are hard, especially for a database that has a lot of concurrent users... My toy one was super fast (I don't remember the stats on top of my head), and was good enough for good weather conditions, but I realized, in order to make it actually production ready, there was so much more to do and it was not worth for me to spend more on it, since there are so many good options out there, and SQL-Lite was becoming a de-facto standard on mobile (both iOS and Android adopted it then).