3 ms·
1. Memory map a file 2. Manage access to it through a server There you go, you know have a database.
by mgaunard 4y ago
1. Memory map a file
2. Manage access to it through a server
There you go, you know have a database.
- orf 4y ago1. Draw some circles 2. Draw the rest of the owl https://www.pillarsofimpact.com/post/draw-the-owl https://www.pillarsofimpact.com/post/draw-the-owl
- mmphosis 4y agoGood to now.
- remram 4y ago> Manage access to it through a server This has real "draw the rest of the owl" energy. How to make a DBMS? 1. Get a file 2. Make a DBMS
- jandrewrogers 4y agoMemory mapping is a poor way to make a database[0], not all storage can be memory mapped, and this doesn't provide any of the functionality you expect of a database like indexing and concurrency control. [0] https://db.cs.cmu.edu/mmap-cidr2022/ https://db.cs.cmu.edu/mmap-cidr2022/
- jfrisby 4y agoI saw a headline about the book on Reddit and made a joke to a friend of mine: "How much you wanna bet it uses mmap?" Well... It uses mmap.
- mgaunard 4y agoThe concurrency control is managed by the server that manages access. All databases that are any different than these two points are just bad.
- mkl95 4y ago1. Build your own database There you go!
- icedchai 4y agoI worked on a commercial product that did just this, except it also had a "transaction log" component. All access through the server was logged, and the log could be used for replay/recovery. While it worked for them for thousands of simultaneous clients, it was not a general purpose solution. There were no real indexes: the "indexes" (hash tables, really) were built in memory at startup.
- akamaozu 4y agoi don't understand what you mean by no real indexes why does generating indexes at start-up not count as having indexes? (asking because i do this all the time)
- icedchai 4y agoI suppose I should've said "persistent" indexes. There was also only a single type supported: hash table. And it was hard coded to 2 fields. You had to change the code (which was C) to change the database. This system was rarely restarted, so in practice it didn't matter what it did at startup, as long as it didn't take more than a few minutes, but it did place some limitations on data size. (This was a 32-bit system and everything was memory mapped.)
- mamcx 4y ago> 1. Memory map a file Complex when you want to > 2. Manage access to it through a server A database is only "easy" IF: - Append only - No real "delete" or "updates" just to reiterate the above. - Only Sequential scan - Only need simple iterator-per-row - No maintain secondary stuff like indexes, so not need to coordinate changes - No concurrency - Fit in RAM, and I mean in few MB - No need to deal with SQL, use his own DSL (sql is so bad! so much weird stuff!, but is ok to have something sql-ish like LINQ) - No need to deal with recursive data types, only scalars - Is only embebed - No need auth or security validations Ok, after making this list, I sure forgot some other tips to make this easy!
- mgaunard 4y agoThere is no requirement to have the file fit in RAM to mmap it, that's the point. Obviously you'd have a file per column (or index), and use directories to represent tables. This is the correct way of doing it and yet so few databases do it.
- mamcx 4y agoYes, I understood, but "mmap a file" that should be a database is not "easy", because now you can deal with segfaults and other weird things that are out the normal playbook. Of course, depending on the other things of the list this is or not a major issue. Is more about how combining several ideas leads to a easy or complex implementation.
- preseinger 4y ago"correct" is not an objective measure, it's a function of use case mmap is not a panacea, it improves specific access patterns by incurring specific costs, it's definitely not true that mmap is the right choice for all databases
- akamaozu 4y agowhy is this so heavily down-voted? it feels correct, even though it's not a complete guide (reliably persisting changes to disk is a big part of what dbs do, but is missing here)