5 ms·
I'm increasingly tempted to try my hand at making a toy OS - or at least a kernel. There's a few things I desperately want an operating system to do: 1. Have e
by josephg 3y ago
I'm increasingly tempted to try my hand at making a toy OS - or at least a kernel. There's a few things I desperately want an operating system to do:
1. Have either database-like transactions or (at a minimum) write barriers for file operations. The current filesystem semantics are terrible for databases and other applications which need to not corrupt their data after a crash. fsync() is too heavy, because you can't issue subsequent writes while waiting for fsync. Write barriers have better ergonomics (so, less complex, less buggy database implementations). And they're finer grained - so databases can also run much faster. Seriously, I could rant for hours about this. It drives me nuts.
2. Use event queue like semantics for subscriptions. There's dozens of OS APIs in which the OS exposes some data to applications where the data changes over time. Eg, filesystem watching, USB device insertion/removal, Bluetooth, watching a network device, or a socket, or APIs for htop to see the set of active processes. Each of these APIs has an entirely bespoke - and oftentimes idiosyncratic - API to get changes to that data over time. Sometimes you need to poll and parse a file in procfs. Sometimes there's syscall based APIs. And some of the subscription APIs are just broken sometimes, and they'll fail to notify you about changes sometimes.
I want to try replacing all of that with a simple, unified API which lets you say "tell me the state of <kernel resource X> and tell me when it changes". The kernel then replies with the object's current state and I get messages notifying my application about changes. We should be able to use the same API for every type of kernel object.
It would also be interesting to try making all userspace programs be wasm bundles and run them in ring0. But I suspect that will run slower on modern hardware compared to using native binaries and context switching. ... Maybe. It might depend on the application.
I think this stuff could be done in linux. Event queue style subscriptions would work especially well with io_uring. But I'm kind of excited by the idea of just noodling myself and seeing where I end up. Making stuff is fun.
- actionfromafar 3y agoSome of the issues can be improved "hackily" by for instance giving a database software exclusive access to two discrete disks - one for the db file, one for the journal.
- shiroiuma 3y agoWhy is this "hacky"? I've long wondered why databases need to be on filesystems, other than simply convenience. If you think about it, a filesystem is, itself, basically a database (not a relational one of course). So it seems like you could improve performance a lot by eliminating the filesystem layer and going straight to direct disk access. Also, how this would be done would probably change depending on whether you're using a spinning-rust HD or an SSD, or a RAID array.
- josephg 3y agoLots - maybe most - applications need to essentially run their own mini database to store user data. I obviously don't want a partition per application, or per word document. Abstracting away your hard disk is literally the job of your filesystem.
- shiroiuma 3y agoI don't mean that some little SQLite DB for your web browser's storage and settings needs its own partition; that's obviously going too far. I mean for really huge databases, where an entire server is dedicated to running that DB and it has high performance needs.
- josephg 3y agoFair enough. But I am including all of those little databases. I agree with you - you could hack around this for a big database by putting it on a dedicated partition (even though you shouldn't need to). But I'm also thinking about applications with little databases that need to live inside the filesystem. They shouldn't need to embed sqlite in order to survive a crash. A different syscall API would allow you to write a small, fast, efficient database engines (like redis or mysql) without needing all of the complex, performance gobbling tricks that these databases currently have. A better filesystem API would help databases everywhere.
- 3y ago
- gjvc 3y agoa great place to start is https://wiki.osdev.org/Expanded_Main_Page https://wiki.osdev.org/Expanded_Main_Page
- sph 3y agoDo it. Ignore whoever says it's too complicated. You probably won't supplant Linux with your toy OS for your own research, but for a software engineer, writing a OS is the closest thing we have to the exhilarating feeling of seeing your code affect real hardware. You haven't lived until a bug of your own causes a triple-fault exception that causes a literal system shutdown. I started this career writing my own tiny OS in my teens (https://github.com/1player/klesh https://github.com/1player/klesh) and for 20 years I've been wanting to go back, with all the experience I have accumulated. Just don't make it another UNIX, please. We have enough of those already ;)
- josephg 3y ago> Just don't make it another UNIX, please. We have enough of those already ;) Hah thanks for the encouragement! Don't worry, there's no danger on that front. I'm pretty thoroughly bored of the "everything is a file" motif. I do like unix's idea of making large programs out of small programs that can be composed together. But unix's raw byte streams make a terrible messaging bus.
- sph 3y agoYou're on the right path! My area of interest is actually making messaging (a-la-Smalltalk/Erlang) the root idea everything is built upon, rather than the file stream. The file metaphor made sense in the 1960, not in an ultra-networked world of heterogeneous CPUs. IMO, communication should be the basis of computing. Which also means inventing a language to build this novel paradigm. As Dan Ingalls said, an operating system is everything that doesn't fit in a language. I am looking forward to finding the time for this long and quixotic quest :-)
- tonyg 3y agoYou might enjoy http://synit.org/ http://synit.org/, perhaps. Heavily inspired by both Smalltalk and Erlang. (It's my own wee quixotic project btw)
- 3y ago
- NoboruWataya 3y agoI am very interested in this tutorial for building an OS for the Raspberry Pi in Rust: https://github.com/rust-embedded/rust-raspberrypi-OS-tutorials https://github.com/rust-embedded/rust-raspberrypi-OS-tutoria... I'd love to try it out when (if ever) I have the time.
- twic 3y agoSo soft updates and kqueue? :)
- josephg 3y agoI'm not sure you mean by soft updates but - kqueue is really just a messaging queue. I want a data subscription system, so I can get a kernel object and get a stream of updates which can be applied to the snapshot in sequence. kqueue is very limited in terms of what you can subscribe to, and things like filesystem watching usually requires subsequent filesystem calls to see whats actually changed. And that in turn introduces race conditions if the data is changed again before your fs syscalls make it through. I think there are cleaner APIs out there. I'd like to see if I can find one.
- saagarjha 3y agoOne problem with 2. is that subscribing to changes isn't always good enough, since if you go to act on the changes things may update out from under you regardless. Like, if you react to filesystem events, if a new file is created you might have it get deleted before you even register that it was added and do whatever you wanted with it.
- vacuity 3y agoI feel like it should be possible to say "I want to reserve access to this resource when something important happens on it". Or just "I want to reserve access to this resource". Hopefully eliminates TOCTOU races. I guess then there have to be different modes to balance scalability/flexibility.
- saagarjha 3y agoYou can definitely do that, it just means that userspace can now stall access to things unless you're very careful :) Like, one model (often used by EDR, which is why everyone hates them) is that every operation must be "authorized" so it doesn't go through until some evaluation happens. But this means the kernel has to ask a program for every operation, which is slow.
- electroly 3y agoWhen people complain about antivirus and endpoint security applications slowing their computer down, this is what they're talking about. This leads to user-visible delays (which, to them, seem to be random) as the reserver does what it wants right after the user has done something to the file. Many (most? ALL?) apparent Explorer stalls are really Windows Defender locking and scanning the file right after you did something to it.
- throw10920 3y ago> It would also be interesting to try making all userspace programs be wasm bundles and run them in ring0. But I suspect that will run slower on modern hardware compared to using native binaries and context switching. If you have all of your programs reside in the same virtual memory space (which is the implication with running them in ring 0), then you get a unique performance advantage in that you don't have to invalidate the TLB. I have no idea how much the amortized performance impact of virtual memory is, unfortunately, so the gains might be trivial, but who knows.
- josephg 3y agoYeah - if you run things in the kernel you don't need to do TLB flushes or context switches - which can be very expensive. But thats all offset by the fact that wasm code runs slower inherently, because it needs to bounds checks everywhere. I suspect that programs which make a lot of kernel calls would run faster in the kernel in wasm, and programs that are more compute / memory bound will run faster as native processes. It'd be fun to play around and find where the tradeoffs lie for different applications.
- dhruvrajvanshi 3y agoSorry for being daft, but doesn't this imply that TLB is now shared between all processes? Which means that the cost of flushing the TLB is amortized across context switches? Processes are gonna evict TLB entries from other processes at unpredictable intervals. I'm not sure if I understood you correctly.
- josephg 3y agoIf all programs run in ring 0, you can just run them all in the same virtual memory space. Then you don’t need TLB flushes at all. Of course, one bad program could bring the whole system down but that’s the price you pay. But if programs were written in wasm, that problem goes away. Wasm prevents bad memory accesses by doing memory protection in software. (Every array or pointer access is bounds checked).
- Findecanor 3y ago> I want to try replacing all of that with a simple, unified API which lets you say "tell me the state of <kernel resource X> and tell me when it changes". Fuchsia's "Signals" are designed in that direction. You get a "handle" to a resource, and the resource has a 32-bit integer with signal bits that you could subscribe to. The meaning of each bit is different for each type of resource. (In general operating system terminology, "Signal" means just a message without any payload that would have required any memory allocation. There are many ways to do it, not just the Unix way. Message ports and other abstractions are sometimes built on top of types of signals)
- josephg 3y agoOh I didn't know that! Thanks - I'll take a look.
- dataflow 3y ago> In general operating system terminology, "Signal" means just a message without any payload that would have required any memory allocation. Do you have a link for this? It's the first time I'm seeing signals described in terms of lack of memory allocation. I always thought the word "signal" basically meant user-mode interrupts - asynchronously hijacking a thread.
- JohnFen 3y ago> I'm increasingly tempted to try my hand at making a toy OS - or at least a kernel. I strongly encourage you to do this! I think writing your own OS is one of the things every dev should do at least once, just for the educational value. The others are writing a game and writing a compiler and linker.
- nvm0n2 3y agoI think macOS/Darwin does have write barriers like you want.
- cbdumas 3y ago> Have either database-like transactions or (at a minimum) write barriers for file operations. Go one step further and have the kernel expose a much more structured view of storage to user space by default. 99.9% of user space programs I've written don't want to deal with file IO, they want a database, or some other way for arbitrary in memory objects to be persisted and shared between processes. There's so much room to improve on the 1970s era IO primitives we use by default.
- didntcheck 3y agoIronically my understanding is that several operating systems around that time did do that, then Unix came and said "a file is just a bag of bytes in a hierarchical namespace" and we've seemingly never reconsidered
- nerdponx 3y agoI can imagine that there's a major lack of flexibility inherent in a record-based filesystem. Nowadays we have file formats that introduce a record format on top of the "bag of bytes" for when you want records, but you aren't required to have records of any particular shape and size (or have any records at all) if they don't work for your needs.
- snuxoll 3y agoNotable major player here was IBM on the AS/400, you had libraries containing objects of various types (such as PGM/Program, FILE/'file', *DTAQ/Data Queue, etc). 'file' objects were effectively database tables, and even things like source code for programs were stored as such. Eventually IBM added the IFS (integrated file system) to provide a more traditional POSIX-adjacent filesystem that was in fact just a bunch of bits, and it feels weird and out of place with the rest of the system as a result.
- bakul 3y ago> The current filesystem semantics are terrible for databases There was at least one Unix startup (IIRC it was Tolerant Systems) in mid 80s that applied transactional semantics to filesystem operations. I did some contract work for them and found the performance was absolutely abysmal and the system had some deadlocks. Now may be their implementation was not great but I suspect the converse is also true: database semantics are terrible for filesystems :-)
- josephg 3y agoThere was at least one database from the mid 80s with abysmal performance and deadlocks too. But plenty of modern databases are fast and correct. The fact this one startup 40 years ago didn’t solve the problem doesn’t say a lot about whether I’ll be able to. Honestly the fact there are some dead bodies on the mountain makes it more fun to give it a go and see how I do. To be frank, judging by the code I’ve seen over the years, most engineers don’t have a lot of experience or skill at making software run fast.
- bakul 3y agoProof is in the pudding etc.! I'm certainly curious how well your design & implementation will turn out. Good luck with your project!!
- brcmthrowaway 3y agoThis seems a total waste of time when you could be learning LLMs