14 ms·
I spent 5 years writing my own operating system
- mathgladiator 5y agoNeat. Well done.
- devetec 5y ago...wow. Congratulations!
- sebyx07 5y agoyou deserve at least 1k votes on hn
- halfer53 5y agoThanks mate :D
- deleted 5y ago[deleted]
- dane-pgp 5y agoCan it run tcc and self-host?
- HenryKissinger 5y agoMore importantly, can it track and collect commercially useful data on its consu... err, users? Can it display desktop ads and make it impossible for the users to turn them off? And does it require an Internet connection and a registered account with the author?
- discmonkey 5y agoJoin the linux clan. It's the one true way.
- undfg 5y agoThis is not relevant, original, or funny.
- obnauticus 5y agoI think it’s pretty funny
- antonzabirko 5y agoBoo
- imjustsaying 5y agoIt's all three
- hungryforcodes 5y agoIt's weird anyways and that makes life interesting.
- roody15 5y agolol :)
- halfer53 5y agoAuthor here, this project is just for fun, nothing more than that. The OS is designed for MIPS architecture at this stage, I don't think tcc supports MIPS at the moment.
- shakna 5y agoThere have definitely been forks of tcc that have added in MIPS support. Depending which one you end up with (debian is pretty far diverged from the mob branch), I'm pretty sure it should be available.
- cbzehner 5y agoIncredible! What was the hardest part? What did you learn that you weren’t expecting?
- halfer53 5y agoThe hardest part is debugging concurrency and weird scheduling bug. It's hard to debug because I can't reproduce it, which is quite frustrating. Over time, I found that making educated guesses, or running the code in my brain, it a lot more efficient than debugging every line of the code.
- sn41 5y agoYou are right. Great work! One somewhat related theoretical observation about concurrency is in some article by Dijkstra (I don't remember the reference right now): he says that debugging using traces (essentially printf) does not work for concurrency, since it is projecting multidimensional data (data present at the same time in multiple processes) unnecessarily linearized onto a single dimension (a sequence of printfs) and then trying to make sense of what is happening. It may not work, even if you print timestamps. His view was to promote theoretical proofs of correctness of concurrent code, rather than debugging, but to me at least, this is much more difficult.
- aquark 5y agoVery true, and not only do print statements inherently serialized threads, they also change the timing so significantly that probably your bug disappears anyway.
- erwincoumans 5y agoI would expect blazing fast in-memory logging with thread-id and time stamps, so timings aren't affected (much).
- grishka 5y agoAs someone who wrote and debugged a lot of concurrent code, here's another advice: log everything. Log as much as you could in the part where you think the bug is. Log every line that's run if you have to. You'll then skim through the log file looking for any unexpected patterns. This approach works better than using a debugger, even on a single-core system, because these kinds of bugs tend to be hard to reproduce and take many iterations. You don't want it to hit a breakpoint a zillion times before it finally shows itself. And another one, tangential to what you said. Read your code line by line and ask yourself "what would break if a context switch happens right here" for each line.
- obsequiosity 5y agoBut does it let you talk with God?
- 0x77617472 5y agoAnd is there a language like HolyC?
- Dma54rhs 5y agoImagine thinking privileged white code bro like you can even imagine what does it mean to get out of material comfort zone of capitalism. Time to take a look at the mirror and very hard, there's no reason for speaking up nowadays.
- deleted 5y ago[deleted]
- deleted 5y ago[deleted]
- erhk 5y agoRequest: Make a blog post on the experience. What does the heatmap of changes look like over 5 years? I cant imagine you burnt the candle on both ends for that whole time. What is the final lines of code? What blockers did you not anticipate that killed a lot of time? Hardest problem? I think you've got a treasure of interesting retrospectives to share :)
- halfer53 5y agoThanks, that's a good point. I'll definitely do that
- beebs93 5y agoYeah, good questions that I'd also love to see see answers to! My favorites for retro-like discussions (if you think it will be beneficial): 1. What did you worry about the most in the beginning that later turned out to be silly/inconsequential? 2. What did you intentionally put off in the beginning thinking it wasn't a big deal that you later regretted not spending more time on?
- mmaunder 5y agoAgreed. Not many devs tackle Mount Everest, so we’re interested in your journey.
- aiisjustanif 5y ago
- jacquesm 5y agoCongratulations. I did the same thing in the early 90's and I still think it is the project that I learned by far the most from. I never released it because I think its time has passed but it was a fun exercise. Debugging the early stages of such development is super hard, especially if you are doing it on bare metal instead of on a VM. Do you have a plan behind this or was it just to scratch an itch?
- halfer53 5y agoThanks. The OS is roughly 90% compliant with POSIX.1, but there's still a lot of work to make it fully compliant. That would probably be my next goal. This is a hobby project, I will probably spend 1-4 horus every other week to work on this. I am working on optimising memory usage of the file system which is almost completed in a separate branch.
- jacquesm 5y agoNeat. POSIX compliance is a tough nut to crack for some of the features that look pretty innocent on the outside, they can dictate huge amounts of under-the-hood architectural and conceptual elements. My project revolved around a 32 bit capable clone of QnX, which at the time had only been released in a 16 bit version. The company that I did regular consulting for had a bunch of requirements that the stock QnX could not fulfill so in a fit of madness I decided to roll my own. By the time it was finished QnX had finally released their own 32 bit OS. Keep at it, I'm really curious how what you are making will end up.
- bregma 5y agoThat's funny because QNX now only supports 64-bit systems and hasn't done 16-bit systems for decades. You must be (relatively) ancient. Disclaimer: I currently work for the company that owns the QNX IP and would also qualify as (relatively) ancient.
- jacquesm 5y agoYes, I'm ancient :) 56 on this end. This work was done in the early 90's.
- 46756e 5y agoAs someone who wants to get into systems programming, this is very inspiring.
- akkartik 5y agoI'm curious about the hardware it runs on, and didn't immediately find answers on Google. Could you talk about what hardware is on a usual system and what device drivers need to do for screen, serial port, disk drives, etc.? Does the CPU use floating point? The reason I ask: for the past six months I've been dipping my toes in OS development for x86 with BIOS, SVGA and ATA interfaces: https://github.com/akkartik/mu https://github.com/akkartik/mu And I'm way out of my depth. I have this suspicion that x86 might be one of the more difficult environments for device drivers, so I'm curious to hear what other hardware looks like.
- halfer53 5y agoI agree, X86 is too complex with its legacy hardware and obscure instructions and environments. It might not be a good starting place to develope hobby projects. The OS is running in a customised RISV hardware in MIPS for educational purposes, you can read more here https://dl.acm.org/doi/pdf/10.1145/1275462.1275470 https://dl.acm.org/doi/pdf/10.1145/1275462.1275470 The simulator https://github.com/halfer53/rexsimulator https://github.com/halfer53/rexsimulator The BIOS has already been written for the serial ports, etc. So it is quite easy for me to interact with the devices in the OS.
- akkartik 5y agoThis is really cool, to the extent that I'm surprised I've never heard of it. Are universities still using it in coursework?
- halfer53 5y agoYeah they are still using to teach computer systems and operating system course
- lovelyviking 5y agoHow hard would it be to run it on RaspberryPi ? (let's say RaspberryPi 400, the one that is within keyboard case)
- kebman 5y agoAwesome work! Very inspiring! :D
- kumarjsingh6 5y agoinitial commit https://github.com/halfer53/winix/commit/e3cd6ecb37511801840408b6ba353544c795720d https://github.com/halfer53/winix/commit/e3cd6ecb37511801840...
- teleforce 5y agoIn the era when system research is dying [1], the only saving graces are the Nix/Guix for OS management improvement and perhaps eBPF for OS performance enhancement. The Winix OS is a breath of fresh air in term of potential OS directions. Like they have always said, timing is everything and probably this is the project similar to Apple's Newton project that have many goods in it but just born several years too early. Winix targets RISC architecture and with RISC-V taking off exponentially at the very moment, having RISC biased OS will definitely provide edges and advantages for the platform, similar to x86 quirks to Linux, and Linux taking advantages when x86-32 and x86-64 took off. This year when Linus was asked for the best Linux achievement compared to other OS, Linus has pointed out the innovative Linux based lock-free filesystem [2]. Winix has built-in innovative POSIX compatible in-memory file-system (IMFS) by default. Imagine an OS with IMFS that's also natively compatible with the increasingly popular Arrow and TileDB in-memory format. With Terabyte (TB) RAM computers becoming the norm in the near future this can easily be the fastest OS with the state-of-the-art filesystem around. Fuschia is another latest OS on-the-block but by focusing on the mobile rather than desktop it will probably optimized for the former, unlike Winix. [1]https://tianyin.github.io/misc/irrelevant.pdf https://tianyin.github.io/misc/irrelevant.pdf [2]https://www.tag1consulting.com/blog/interview-linus-torvalds-linux-and-git https://www.tag1consulting.com/blog/interview-linus-torvalds...
- halfer53 5y agoLol, I like your description of the project, sounds much more promising than mine
- jackcviers3 5y agoWoz, meet Steve.
- captainredbeard 5y agoSock puppet?
- amelius 5y agoAre you talking about kernels or OSes? There is a huge difference in terms of opportunities for further research.
- ashdev 5y agoI just generally have tremendous respect for someone who even attempts to build an OS from scratch. You kept at it and didn't give up. Great work.
- ZephyrBlu 5y agoHey, another Kiwi! The about section says "A UNIX-style Operating System for the Waikato RISC Architecture Microprocessor (WRAMP)". Did you study at Waikato? I'm curious what you think about tech in general in NZ as well. Compared to the US it feels like there's practically no community :/.
- tjpnz 5y agoAnother Kiwi here too. First I've ever heard of the "Waikato RISC Architecture" but am curious to learn more. I'm not the author but my view of the NZ tech sector is that it's pretty much stagnant. I ended up taking my skills elsewhere as did most of the people I remain in contact with from my UC days. It seems successive governments have put all their eggs into the agricultural export market. That's a real shame.
- ZephyrBlu 5y agoThat's basically what I thought, but it sucks to hear that. Whereabouts did you go from NZ? I'm working remotely right now and want to leave NZ after travel opens up. Compared to other places in the world NZ seems terrible for a young, single guy in tech.
- tjpnz 5y agoI went to Tokyo. Not where people typically go but it's a fun place to live. Salaries are about on par with Australia and Western Europe provided you have some experience.
- 0x90nz 5y agoIf you want to learn more there's a website [1] for the WRAMP architecture which goes into a fair bit of detail. The manual specifically is what students are (or at least were when I was at Waikato) given as a resource to understand it. The toolchain and simulator are also available on GitHub [2], though it looks like the author has their own fork from an older version. A few years ago the the project was ported from a custom board to an off-the-shelf FPGA dev board and the simulator was renamed from "rexsim"/"RexSimulator" to "wsim", hence the difference in name. [1] https://wramp.wand.nz/ https://wramp.wand.nz/ [2] https://github.com/wandwramp https://github.com/wandwramp
- readonthegoapp 5y agoI'm sorry, you're lacking the requisite expertise for this entry level developer position. Well, I wrote my own operating system. Sorry.
- joshsyn 5y agoyep, hiring is broken.
- halfer53 5y agoDepends on the audience I did struggle to explain the significance of this project when I was applying for graduate roles, especially to HR people who only look for keywards in CV. But there are people who appreciate my work. I got my internship at AWS because my interviewer who had technical background was very impressed with my work, hence the reason I got hired. Sometimes you just have to find the right audience.
- kyting 5y agoKudos to you! Yes, good hiring managers with sharp vision will always want to hire new grades who is able to write operating systems, compilers or simulators from scratch. Having those essential CS skills are much better than having many buzz words in the resume.
- ericls 5y agoThis looks like one of the most interesting and fulfilling game one can play
- RandyRanderson 5y agoOk, I'll ask - why? You have made it POSIX-ish but wouldn't one goal be to create something new ? I imagine that you just wanted to use a bunch of POSIX tools but won't that limit you in some way?
- fm2606 5y agoNot the OP but, ummm, maybe for no other reason than because he wanted to. Or, he wanted to learn something. I see this question a lot when someone shares their work that they are proud of and I want to scream, "What difference does it make?" In the very first sentence of the README it states "hobbyist, educational" . That pretty much sums up the OP's motivation. Why can't learning and exploration be their own motivation?
- imvetri 5y agoNice job. It's not easy for everyone to be dedicated to get things done.
- ktm8 5y agoI'm mildly annoyed by line 30 in include/posix_include/stdio.h
- halfer53 5y ago:P
- mrlonglong 5y agoIt's only an errant tab ffs.
- failwhaleshark 5y agoOh my. Minix1. That predates me and my gray hairs. I at least got Minix 2 with the printed source code in AST's book. Minix 2 was so annoying on real PC hardware that I had to stick a keyboard repeat rate and delay patch whenever I used it. It was all straightforward and the lab assignments on it were easy because it was built for tinkering rather than efficiency. My uni replaced FreeBSD as the teaching operating system of choice. They also go rid of Oracle DBMS# for Postgres. # Many, many moons ago I nearly got Aaron Swartz'ed. This is because I tried browsing "proprietary" docs using a Java-based internet proxy to bypass IP ACLs from my "off-campus" location. It was clearly located in North Korea. Then, the browsing was so slow (like Tor but even worse), I figured: Why not mirror it locally instead? And, what kind of documentation "toilet-paper" police are going to sit there watching interactively or SNORT monitor what happens to "precious" corporate documentation they forced me to use?
- hoseja 5y agoHow do you feel about the agents of the CIA?
- sytelus 5y ago5 years is a long time. If I recall correctly, Linus took one year to churn out his first usable Unix clone. I wonder why this project took so much more time. PS: This is not a criticism but genuine curiocity why basic OS projects are so expensive given so much already out there. Also, Linus spent majority of hours each day for a year while this might be just a side gig for you.
- halfer53 5y agoLinus is a genius, I don't think I could ever reach his level Yeah it was a side gig for me, I did spend major of my days for the duration of 2 months last year to work on this. In fact, I paused on the development of this project for 1-2 year. But most of the time, it's just 1-2 hour per week side projects on average.
- k4ch0w 5y agoI think you don't have people teaching enough lots of low level coding in college is why. It used to be the first thing you had to learn in Linus's time. Now you use it for one class and use Python/Go/Java the rest of the time. You pretty much have to seek out the knowledge regarding Operating systems in addition to taking on the challenge of learning something like C/C+ or Rust. You have to then break all the ways of thinking about your memory model after being given a GC language. Understanding hardware is needed to debug specific CPU/IO issues that could be caused by assumptions or lack of knowledge. Next comes learning the different algorithms needed by an OS in a new language and then implementing them. This takes a lot of time to get down correctly. It's also a really small and niche community which means you don't have a lot of support from peers that understand the problems you encounter the community isn't an easy one to feel accepted into. It can be hard to get support because you need to be very detailed in your questions or you will be ignored.
- felixthecat543 5y agoHow did you get started with this project?
- ChrisMarshallNY 5y agoThat's great! Thanks for sharing it. I'll probably never have a use for it, myself, but I am always in support of passion projects. Good luck!
- sgt101 5y agoI shared a co-working space with a guy who had written a distributed OS for rendering; it was quite neat - but about 1000 man years from being usable. I felt sad for him - he'd poured his heart into it for more than 10 years (he said) and had no idea of how to turn it into anything other than a hobby project. He hadn't open sourced it, his aspiration was that "Facebook will buy it". I haven't seen him since the pandemic kicked in.
- rualca 5y ago> I felt sad for him - he'd poured his heart into it for more than 10 years (he said) and had no idea of how to turn it into anything other than a hobby project. There is nothing wrong with that. It's fun to do things just for fun. > He hadn't open sourced it, his aspiration was that "Facebook will buy it". It's hard to believe that's anyone's real motivation for such a project.
- sgt101 5y agoYea - agree about fun to do things for fun - but my impression was that he had given up a great deal to work on the system and unfortunately genuinely believed that somehow he would be able to monetize his efforts and that his hard work and insight would "pay off". I hope it does, I don't think it will.
- johnwheeler 5y agoOften when you do things just for fun, you’re driven by the possibilities of what could be. That’s part of the fun. This is especially true of software, and there’s nothing wrong with it. What there is something wrong with is criticizing others’ motivations because they seem outlandish to you. It’s fine and even healthy to be so ambitious. Not all of us want to sit around playing with legos assuming that’s the best a reasonable life has to offer.
- amelius 5y agoDid you ask him why this needed a new OS, as opposed to e.g. some daemons running on top of an existing OS?
- knorker 5y ago"Visual" memory?
- halfer53 5y agoOops, typo :p
- mryalamanchi 5y agoIs there a possibility of integrating GCC compiler and adding support for ELF executables? If yes, then how?
- mryalamanchi 5y agoIt's a tough ask but I would like to understand what will be the steps to go through.
- halfer53 5y agoIt's certainly plausible It's currently using wcc to compile the kernel https://github.com/wandwramp/wcc https://github.com/wandwramp/wcc. We can swap it with gcc for sure, but require bit of work, especially the output format and backend. Winix assumes the output file is in srec format https://en.wikipedia.org/wiki/SREC_(file_format) https://en.wikipedia.org/wiki/SREC_(file_format), so we would need to tweak gcc to support this header format and add backend support for WRAMP architecture. ELF is quite complicated, if I have to do this, I probably just copy some codes from linux.
- bregma 5y agoIf you're not supporting DSOs (shared libraries) then ELF is pretty straightforward. In fact, from the kernel point of view it's still fairly straightforward since all you need to do is either load the (non-DSO) program and go, or else load the (non-DSO) program named in the interpreter entry and go, and leave the hard lifting to the interpreter. The kernel itself never needs to understand about DSOs or GOTs and PLTs or DWARF or any of the other fancy magic that ELF brigs to the table. Just how to find and load the LOAD segments and where the entry address is, or where the PT_INTERP string is. It's really just that easy.
- halfer53 5y agoThanks for the info, good to know that it's easy to implement from kernel's perspective
- andyxor 5y agoAwesome, but how does it even work with so few tests
- peter_d_sherman 5y agoRelated: Waikato RISC Architecture Microprocessor (WRAMP) https://wramp.wand.nz/ https://wramp.wand.nz/ https://github.com/wandwramp/ https://github.com/wandwramp/
- juicexmachina 5y agoHas submitted the 'does it run crisis' issue yet?