5 ms·
Writing an operating system kernel from scratch
- lordleft 1y agoI will never not find this kind of project incredibly impressive. It’s interesting to think that Linux, after all, is really just the kernel — and yet getting that work done paved the way to getting an open source version of Unix installed on billions of machines. Great stuff!
- hollerith 1y agoIt is equally valid to say that Stallman's starting to write a C compiler and Unix utilities (in 1984 whereas the Linux project started in late 1991) paved the way to getting an open source version of Unix installed on billions of machines.
- kimixa 1y agoI agree - there's a number of kernels that were "open source" and released at a similar time enough time to linux (e.g. 386BSD in '92) that I could see any of those winning the "community battle" and taking that space instead, but no real credible "development toolchain" equivalent until decades later. Though I'm unsure how differing licenses might have affected this - I suspect that really early in it's development the "copyleft" nature of the GPL Linux didn't make as much of a difference, as from what I remember most commercial uses of Linux didn't come until it had already gained significant momentum.
- deleted 1y ago[deleted]
- treyd 1y agoThe copyleft nature was essential to good driver support. It set it up such that for corporations making drivers the easiest path was to get the driver upstreamed. There was a bunch of hoops they could have gone through to avoid that (as many did, like Nvidia) but that became a sorta-default. Copyleft encourages a collaborative relationship between entities because it makes trying to play it close to the chest with IP involve more legal effort (if it's possible at all).
- kimixa 1y agoYes, I can see that stalling development as (at best) it turns into a pile of private forks rather than a cohesive project, but from what I remember that was already after Linux had "won" the "Open Source Kernel" race. Commercial support for Linux was... Sparse... before the early 2000s.
- Quekid5 1y ago> [Stallman/GNU] getting an open source version of Unix installed on billions of machines. Agreed, funnily enough GNU tools/compilers also ended up getting installed on a lot of proprietary UNIXes because proprietary UNIX was mostly shit (in user space!). At least most of the ones I had the misfortune to have to work on.
- hollerith 1y agoI first came across GNU tools on NeXTSTEP, which wasn't too bad. If Stallman had started with a kernel, there would be very few people who had the legal right to run any utilities or apps on the new kernel whereas GNU's utilities and apps (e.g., Emacs) were immediately useful (i.e., without breaking any copyright law or violating any software license) to a large population, namely, anyone with an account on a proprietary Unix system, which explains why Stallman chose to start with the userland.
- amszmidt 1y ago> If Stallman had started with a kernel, there would be very few people who had the legal right to run any utilities or apps on the new kernel That is really not true, one of the most important things when it comes to the GNU project and the whole Free Software movement is the ability to run _any_ program, be it non-free software or free software. This has been parroted for more than 40 years now ...
- hollerith 1y agoThat's true in isolation, but is missing vital context. In the 1980s a user of proprietary Unix usually had no way to make the ls utility or a C compiler or a text editor run on an new Unix-like kernel. And even if he had the technical means to do it, he probably did not have the legal right, i.e., it would have been a violation of copyright law. In the Unix world, there was no tradition of binary compatibility: you couldn't just take a binary compiled for NeXTSTEP and run it on Solaris or on Linux: you needed to recompile, which means you needed a C compiler and the source code for the program you want to run, and most users of Unix in the 1980s didn't have a practical way to get the source code (and if he did have access to the source code, using to to port the program to a new kernel was probably a copyright violation). Stallman could have tried to start a tradition of binary compatibility in the Unix world. That is one of the strategies he could have chosen for the GNU project: i.e., he could have picked a proprietary Unix, Solaris, say, and make sure the kernel of his GNU system could run all (or most) binaries that run on Solaris. But that strategy ran the risk that the owner of Solaris might have sued to stop this and the courts might have sided with owner, requiring the GNU project to shut down, pay the owner (Sun) a lot of money or start over with some other strategy.
- diath 1y agoIt's even more funny and amusing when you remember that in the initial release mail for Linux, Torvals said "just a hobby, won't be big and professional like gnu". https://groups.google.com/g/comp.os.minix/c/dlNtH7RRrGA/m/SwRavCzVE7gJ https://groups.google.com/g/comp.os.minix/c/dlNtH7RRrGA/m/Sw...
- chuckadams 1y agoHe's also referred to Linux as "the GNU Emacs of all terminal emulators".
- Y_Y 1y ago> It's now the GNU Emacs of all terminal emulators. > (Linus Torvalds, regarding the fact that Linux started off as a terminal emulator.) http://neil.franklin.ch/Jokes_and_Fun/Linux_Quotes.html http://neil.franklin.ch/Jokes_and_Fun/Linux_Quotes.html That's the best reference I can find, but even if it's totally legit it doesn't make any sense to me.
- throwaway82332 1y agoPerhaps it's a reference to how Emacs does "everything" and some has also joked that Emacs is an OS with a text editor
- flykespice 1y agoBack then Emacs was considered what we call in modern software 'bloatware': resource hungry, big and full of feature creep. BTW, in early versions of Slackware emacs was bundled in its own software category due to its huge size when selecting which packages to install on first boot.
- chuckadams 1y agoProbably even had its own floppies too. Remember that install method?
- Someone 1y ago> I will never not find this kind of project incredibly impressive I wouldn’t call it incredibly impressive. The path on how to write a minimal multi-tasking kernel has been beaten decades ago. Writing a kernel that can boot and do a few things is ‘just’ a matter of being somewhat smart and have some perseverance. Doing it for RISC-V complicates things a bit compared to x86, but there, too, the information about initialising the hardware often is easily obtained (for example: https://wiki.osdev.org/RISC-V_Meaty_Skeleton_with_QEMU_virt_board https://wiki.osdev.org/RISC-V_Meaty_Skeleton_with_QEMU_virt_...; I wouldn’t know whether this project used that) I think the author agrees, given (FTA) that they wrote “This is a redo of an exercise I did for my undergraduate course in operating systems” It’s work, may be nice work, but I think everybody with a degree in software engineering could do this. Yes, the OS likely will have bugs, certainly will have rough edges, but getting something that can multi-process, with processes shielded from each other by a MMU isn’t hard anymore.
- d_sem 1y agoIs there a book one can read to learn how to create one?
- esafak 1y agohttps://en.wikipedia.org/wiki/Operating_Systems:_Design_and_Implementation https://en.wikipedia.org/wiki/Operating_Systems:_Design_and_...
- deleted 1y ago[deleted]
- andrehacker 1y agoThat is an excellent recommendation. For Operating Systems anything Andy Tanenbaum did is world class. This made me look up what he has been up to, there is a 2023 edition of "Modern Operating Systems" which covers cloud virtualization and Android along with everything that came before, hm, tempting.
- lifeline82 1y ago[dead]
- WD-42 1y agoZig really is amenable to OS development. And so is RISC-V. I started this same exercise except with x86 first. I quickly got annoyed with all the legacy boilerplate required. RISC-v doesn’t have any of that. It’s so much faster to get up and running. Here’s my bad zig: https://github.com/Fingel/aeros-v https://github.com/Fingel/aeros-v
- toast0 1y ago> with x86 first. I quickly got annoyed with all the legacy boilerplate required. IMHO, if you use a reasonable bootloader, you don't have too much boilerplate. Multiboot loaders do leave you in real mode, and most people want to be in protected mode, so you have to set up some tables and do a jump, but that's not that much boilerplate. There's a bit more stuff if you disable legacy interrupt controllers (which you probably want to) but it seems to me being able to maybe run on a regular pc is worth it (caveats about console interfaces apply... my hobby OS needs bios boot and uses some vga things that I found aren't well supported... serial console is easier, but lots of computers don't have a serial port either)
- ethin 1y agoYou still have to set up the GDT, TSS, all that other legacy stuff that isn't actually used but it's still there. And if you set it up incorrectly, your punished for it with triple faults and the like, even if nobody even uses anything like the GDT or TSS these days.
- convolvatron 1y agothat is a pain. oswiki is a good resource, and there are also a huge number of examples. but it still takes a day. context save and restore and interrupts is another speed bump. but its no different than any other development banging your head against the wall exercise, and its pretty rewarding to have smoothed all that over and go to town on bespoke scheduler design.
- saulpw 1y ago
- chris_wot 1y agoHow does one get their hands on low-cost RISC hardware?
- Rzor 1y agoThere's this $10 board on Aliexpress called Milk-V Duo S. It's been popping up on my recs every now and then. Looks interesting. https://aliexpress.com/w/wholesale-Milk%2525252dV-Duo-S.html https://aliexpress.com/w/wholesale-Milk%2525252dV-Duo-S.html Taken directly from its description: >Milk-V Duo S is an upgraded model of Duo with an upgraded SG2000 master with a larger 512MB RAM and extended IO capability >Onboard WI-FI6/BT5(Model Milk-V-Duo-S-512M-Basic/Milk-V-Duo-S-512M-eMMC does not have this function) >USB 2.0 HOST port >100 Mbps Ethernet Portwith PoE Support (via PoE HAT) >Dual MIPI CSI with ISP >The device also supports switch between RISC-V and ARM boot via a switch
- hackerInnen 1y agoI think there are dozens of boards by now, but what I found interesting and pledged to is this: https://www.kickstarter.com/projects/starfive/visionfive-2-lite-unlock-risc-v-sbc-at-199/description https://www.kickstarter.com/projects/starfive/visionfive-2-l... I don't have a first version visionfive2 myself, but i heard good things about it and the eco-system sounds like it's growing. There are still rough edges. Lets hope they ship soon (tm) Edit: What I currently have for my personal projects is a PolarFire SoC Discovery Kit. That's a quad-core RISC-V system with FPGA embedded. Maybe too expensive and not for everyone (130 bucks, fun fact: The dev board is cheaper than the chip itself). https://www.microchip.com/en-us/development-tool/MPFS-DISCO-KIT https://www.microchip.com/en-us/development-tool/MPFS-DISCO-... Also the microchip documentation, toolchain and so forth feel really crappy and old, but once you get used to it, it's not that bad and actually getting to run bare-metal risc-v code is easy. There are easy examples for linux and bare-metal
- yencabulator 1y agoMaybe just run an emulator on your x86/apple, for now? It'll probably be faster than actual hardware, and it will for sure be faster to develop a kernel with. https://www.qemu.org/docs/master/system/target-riscv.html https://www.qemu.org/docs/master/system/target-riscv.html
- ktosobcy 1y ago> El acceso a la presente dirección IP ha sido bloqueado en cumplimiento de lo dispuesto en la Sentencia de 18 de diciembre de 2024, dictada por el Juzgado de lo Mercantil nº 6 de Barcelona en el marco del procedimiento ordinario (Materia mercantil art. 249.1.4)-1005/2024-H instado por la Liga Nacional de Fútbol Profesional y por Telefónica Audiovisual Digital, S.L.U. > https://www.laliga.com/noticias/nota-informativa-en-relacion-con-el-bloqueo-de-ips-durante-las-ultimas-jornadas-de-laliga-ea-sports-vinculadas-a-las-practicas-ilegales-de-cloudflare https://www.laliga.com/noticias/nota-informativa-en-relacion... Of FFS... one thing is using http but another this one…
- flykespice 1y ago...what? a football organization in Spain has the power to restrict internet access to the entire country?
- lioeters 1y agoI heard about this situation in Spain where Cloudflare (and maybe others) are blocked by mistake due to being associated with some sites streaming football. How are people getting around this problem, I guess VPN? I mean, some of the wrongly blocked IPs are needed for work I imagine.
- fransje26 1y ago> initiated by the National Professional Football League and Telefónica Audiovisual Digital, S.L.U. These people are criminals..
- NaOH 1y agoSubmitted separately by the author: https://news.ycombinator.com/item?id=45236479 https://news.ycombinator.com/item?id=45236479 Copying their comment to here: I have redone the classical exercise of writing a tiny OS kernel with time sharing, which manages a couple of user threads. My goal was to experiment specifically on RISC-V + OpenSBI. Additionally, I wanted to explore Zig a little bit, so that was the language used instead of the traditional C, but it should be straightforward how to do the same experiment in either C or Rust. It's definitely very rough around the edges, and it's more of an experiment and an intro for people who want to go through step 0 of learning OS kernel development and computer architecture. Nevertheless, I hope it is still a fun experimental thing to play with over the weekend! The full walkthrough and the GitHub link are available at the link posted!
- kristoff_it 1y agoFor those who want a "packaged" experience doing this same kind of work, there's "Operating System in 1000 Lines of Code" (https://operating-system-in-1000-lines.vercel.app/en/ https://operating-system-in-1000-lines.vercel.app/en/), I followed it a while ago in Zig (which meant translating C code snippets to Zig) and had a good time. I have the code and VODs here: https://github.com/kristoff-it/kristos/ https://github.com/kristoff-it/kristos/
- fransje26 1y agoWhat would be the level of Zig knowledge needed to do this exercise? Or, asked differently, with no knowledge of Zig, what would be a realistic approach to trying this out? (Assuming an interest in learning Zig, with a background in C++)
- ozgrakkurt 1y agoI could easily understand stdlib code of zig after building some stuff for 2 days. Would be different for everyone but it is fairly easy
- kristoff_it 1y agoIt's definitely possible to hit the ground running you have some knowledge of systems programming. I would say that the Zig version is much easier to understand than the C version provided by the book because the Zig one uses `packed structs` instead of bit operations in some places, everything else is roughly the same.
- Isamu 1y agoAlways a fun and interesting thing to try. Also go ahead and try hard things like your own cryptography. The advice “don’t roll your own crypto” just means don’t DEPLOY something that is not battle tested, because it is very easy to make BIG mistakes in that area. Always feel free to experiment and explore. We need more operating systems, more choice.
- jackdoe 1y agonice! i love how accessible riscv isa is, the docs are amazing, there are tons of examples, and tons of emulators, the uncompressed machine code is quite readable. i am writing a book for my daughter and am also working on a small os with forth and timesharing https://punkx.org/projekt0/book/part1/os.html https://punkx.org/projekt0/book/part1/os.html and i dont think i could've done it on x86, learning both forth and riscv assembly on the way, its super fun. honestly if you ever wanted to make a toy os from scratch there is no better time to start than now (besides maybe 1980), get a cheap riscv like rp2350 and just upload the relevant sections of the docs to claude and it will help you if you get stuck.
- balder1991 1y agoI just checked the prices for my country and I’m surprised how cheap this is. I’m 90% sure I’m buying one just to play around with.
- jackdoe 1y agoand the datasheet is amazing https://datasheets.raspberrypi.com/rp2350/rp2350-datasheet.pdf https://datasheets.raspberrypi.com/rp2350/rp2350-datasheet.p... there is an issue that language models have seen enormous amounts of rp2040 code and docs, and it they get properly confused a lot, so you should just pdf print relevant pages and use them in-context.