13 ms·
Xv6, a simple Unix-like teaching operating system
- bane 13y agoReally cool project. Older, simpler, unixes are surprisingly relevant for undergrad-level learning. It reminds me of my OS class in my undergrad days. We skipped a few bits by using the JVM to get a few things for free, but we still had to write (as groups of 3-5 people) an entire unix-like, complete with user-space programs, messaging systems, etc. Definitely one of the most valuable classes I took for my degree.
- mjg59 13y agoSomeone's going to have the fun job of porting this to UEFI at some point in the not-too distant future.
- swetland 13y agoDoesn't seem like it'd be a big ordeal. Just adapt an existing bootloader to load the kernel. Also it looks like they mostly run it in qemu.
- geofft 13y agoIt's multiboot, so you can just boot it from grub.efi. See sheet 10, xv6/entry.S. The more annoying thing is likely to be that it's 32-bit only. Are there any common machines that do x86-32 UEFI boot? Alternatively, does a 64-bit grub.efi know how to exit long mode and go into protected mode, to multiboot a 32-bit kernel? (I was actually talking the other day with some folks about whether a 6.828 project involving a UEFI port of JOS, the other teaching OS, would make sense, but the 64-bit thing seemed to make it less interesting unless you also plan to do a 64-bit port of JOS. Which has been done before in a final project, admittedly...)
- swetland 13y agoIt's definitely 32-bit only, but structurally it's simple enough that converting to 64-bit might not be a huge effort (and the kernel is only about 6kloc of .c files). I haven't done x86 systems stuff since 64bit became common, so it might be a fun way to learn about what's different.
- mjg59 13y agoAh, true - there's no support for any other BIOS-based services, so that's not going to be a problem. Being 32-bit only isn't really an issue, there's no theoretical problem with a UEFI-based bootloader jumping into 32-bit before executing a multiboot image. You won't be able to make any UEFI runtime calls, but that's probably not a high priority anyway.
- userbinator 13y agoAs long as there are things like SeaBIOS, there is no need for UEFI.
- adamnemecek 13y agoThat reminds me, I stumbled upon the website of the Unix Heritage Society (http://www.tuhs.org/ http://www.tuhs.org/) which has the source code for the early versions of Unix. Here is the first version http://minnie.tuhs.org/cgi-bin/utree.pl?file=V1 http://minnie.tuhs.org/cgi-bin/utree.pl?file=V1 I did not really spend enough time reading the source to understand what's going on but it's interesting regardless.
- derekp7 13y agoThere's also a port of V7 Unix to x86, by http://www.nordier.com/v7x86/ http://www.nordier.com/v7x86/ I only wish that there was an add-on tcp/ip stack for that version, then one could really have fun with it. Yes, I know, the ip stack add on is called BSD.
- rootbear 13y agoThis version hasn't been updated in a long time. I hope the author gets back to it at some point. I've only played with it a bit, but it was very nostalgic. I think adding modern stuff to this is a bit missing the point, but it might be fun to add some sort of GUI to it, something much simpler than X Windows.
- rst 13y agoYou might want to look for 2.9BSD, a PDP-11 Unix variant which was more or less v7 with the TCP stack from 4.xBSD grafted in. Patching this into an x86 port would still be a bit of a project, but it's probably easier than starting with 4.xBSD directly, or starting from scratch.
- __mp 13y agoAlternatively there's also Barrelfish: http://www.barrelfish.org/ http://www.barrelfish.org/ but there the focus lies on multiple cores (48ish+++).
- oscargrouch 13y ago>Russ Cox (rsc@swtch.com) >Frans Kaashoek (kaashoek@mit.edu) >Robert Morris (rtm@mit.edu) sh$%t, this got to be good! Cloning the repo and surfing at the source code, just for fun, right now :)
- kirbyk 13y agoPurdue has a similar OS, Xinu, that was created for teaching. It is actually used in industry around the world. - http://www.xinu.cs.purdue.edu/ http://www.xinu.cs.purdue.edu/
- capkutay 13y agoI wonder how this compares to pintos (http://www.stanford.edu/class/cs140/projects/pintos/pintos_1.html http://www.stanford.edu/class/cs140/projects/pintos/pintos_1...) which is an i386 unix like OS. In my under grad class we had to build thread scheduling, sys calls, virtual memory, and some file system functionality on top of pintos. It was the most challenging and most interesting set of projects I've ever completed.
- merkury7 13y agoAdding to the list of alternatives: MINIX (http://www.minix3.org/ http://www.minix3.org/)
- mwcampbell 13y agoAt the risk of reigniting an old flame war, it seems to me that a microkernel in which core OS functions such as process management and the virtual file system are in separate servers would be more complicated than a monolithic kernel. Having device drivers and specific file systems in user space may be useful, though.
- mcintyre1994 13y agoThe impression I get (university class on OSes) is that the microkernel/modular approach is more about making it simpler to port than making a specific instance simple? Similarly it'd be less complicated to extend since that's just a case of adding a new loadable module. In terms of a learning OS I'm not sure exactly which should be prioritised - extendability/portability or a simple structure? If this is a topic with a lot of depth I'm obviously not getting I'd definitely be interested in a discussion - my exam is soon :)
- mwcampbell 13y agoHave you read the debate between Andy Tanenbaum and Linus Torvalds from about 1992? It's a discussion of the tradeoffs between microkernels (Tanenbaum) and monolithic kernels (Torvalds). You can find it here: http://oreilly.com/catalog/opensources/book/appa.html http://oreilly.com/catalog/opensources/book/appa.html The chapter by Torvalds in the same book is also worth reading: http://oreilly.com/openbook/opensources/book/linus.html http://oreilly.com/openbook/opensources/book/linus.html The claim that microkernels improve portability makes no sense to me, since Linux itself is very portable, as are the BSDs (especially NetBSD and OpenBSD).
- stplsd 13y agoThis is flame between two man with HUGE egos, not a debate.
- sillysaurus2 13y agoYes! I'm so happy to see this featured on HN. You'll be fascinated by the topics presented in xv6. Have you ever: - ... Wondered how a filesystem can survive a power outage? - ... Wondered how to organize C code? - ... Wondered how memory allocation works? - ... Wondered how memory paging works? - ... Wondered about the difference between a kernel function and a userspace function? - ... Wondered how a shell works? (How it parses your commands, or how to write your own, etc) - ... Wondered how a mutex can be implemented? Or how to have multiple threads executing safely? - How multiple processes are scheduled by the OS? Priority, etc? - How permissions are enforced by the OS? Security model? Why Unix won while Multics didn't (simplicity)? - How piping works? Stdin/stdout and how to compose them together to build complicated systems without drowning in complexity? - So much more! I credit studying xv6 as being one of the most important decisions I've made; up there with learning vim or emacs, or touch typing. This is foundational knowledge which will serve you the rest of your life in a thousand ways, both subtle and overt. Spend a weekend or two dissecting xv6 and you'll love yourself for it later. (Be sure to study the book, not just the source code. It's freely available online. The source code is also distributed as a PDF, which seems strange till you start reading the book. Both PDFs are meant to be read simultaneously, rather than each alone.)
- mwcampbell 13y ago> Why Unix won while Multics didn't (simplicity)? Considering how many not-so-simple systems have succeeded in the market, I suspect that the real reasons for the failure of Multics are more nuanced.
- angersock 13y agoEh? How do you figure?
- bodyfour 13y agoI'm not the commenter you're replying to, but I'll take a crack. Saying it was a matter of "simplicity" glosses over too many of the details to be useful. Also consider that many things that early UNIX didn't include from multics got added back later (memory mapped files, dynamic linking, symlinks, ...) so by that measure it was maybe too simple. A few points to consider: * Multics was a joint project, and each participant had different levels of commitment and priorities for the project. This wasn't a recipe for success (see also: Taligent, OS/2) * Hardware. UNIX moved to the PDP-11 soon after that machine became available. That machine went on to become the smash hit of the decade, especially in academia. The GE and Honeywell mainframes that Multics ran on continued their decades-long loss to IBM. I've often wondered how different history would be if Bell Labs had bought any other minicomputer for the project. Probably today UNIX would only be known as a bunch of research papers. By skill or by luck they landed on the platform that would let their creation spread. * Language choice. When the Multics project kicked off, using PL/I seemed like a progressive choice. After all, it had been designed by all the right committees! However, PL/I only had a moment in the sun. To me, the single most impressive thing about the original UNIX team was that they also built C at the same time. This created a virtuous cycle where language-geeks wanted to use C (and thus wanted UNIX) and OS geeks wanted UNIX (and ended up learning C). If PL/I had lived up to its billing of being the true successor to Algol then C might not have had a void to fill and this cycle wouldn't be possible. * Being born to the right (corporate) parent. Building even a small OS at that time cost quite a bit of money: not only to pay bright people for years to work on it, but to buy the minicomputers they needed. Bell labs had the resources and interest to do this but, uniquely, couldn't meaningfully productize it (due to their status as the telephone monopoly at the time) This meant that in the early days it was near-free, source code included. If UNIX had instead been developed inside of a computer company it would have been turned into an exclusive product and probably wouldn't have spread so far in its early days. Bell labs was one of the few organizations outside academia that would have done this. * Just plain talent. The UNIX guys were, of course, flat out brilliant. Take pipes for example. Multics had the ability for processes to be chained, but it wasn't recognized as a first-class feature.. they were still thinking of programs as usually being separate beasts. The flash of insight was "this would be super-useful if all programs behaved this way and if the shell made it easy to construct" It's not a matter of "simplicity", but that they simply saw the problem in different way than everyone who came before. In short, UNIX was just damn good. Certainly UNIX started out as a simple system, and that was part of its success. However, other things have been simple and never went anywhere. The point I'm trying to make is that to understand why UNIX won you need to consider a lot more of the history.
- kps 13y agoI just did a quick grep through the code, and apparently students are expected to understand this.
- bodyfour 13y agoShame this is getting downvoted, probably by people who don't understand the reference. See the second section ("You are not expected to understand this") in: http://cm.bell-labs.com/cm/cs/who/dmr/odd.html http://cm.bell-labs.com/cm/cs/who/dmr/odd.html
- btian 13y agoAlso xv6 book that's meant to accompany the source code. http://pdos.csail.mit.edu/6.828/2012/xv6/book-rev7.pdf http://pdos.csail.mit.edu/6.828/2012/xv6/book-rev7.pdf
- csense 13y agoThe line numbers in the book reference a 92-page printout of the source code here: http://pdos.csail.mit.edu/6.828/2012/xv6/xv6-rev7.pdf http://pdos.csail.mit.edu/6.828/2012/xv6/xv6-rev7.pdf
- Jonanin 13y agoxv6 is a pleasure to hack on and learn from! It's got everything you'd expect from a unix-like OS (multi-processor support, simple scheduler, unix-like FS, many of the standard syscalls, fork/exec based shell, etc). Linux and *BSD may be the modern standard for unix-like OSes, but they're often too complex and opaque for beginners to learn from. It's super easy to read and toy with xv6's code if you want to learn more about OS internals, though. For example, I was able to implement a simple log structured file system: https://github.com/Jonanin/xv6-lfs https://github.com/Jonanin/xv6-lfs Note: if you're looking for companion reading material, check out this OS book: http://pages.cs.wisc.edu/~remzi/OSTEP/ http://pages.cs.wisc.edu/~remzi/OSTEP/ (IMO it's super approachable and understandable for undergrad-level learning)
- Moral_ 13y agoWere using the Xv6 kernel for the OS class here at the university of Utah, for the first time. In my opinion an OS class that makes you hack on low level code is significantly more beneficial tha just a theroy class.
- blt 13y agoAwesome! I really hope MIT releases video lectures for 6.828 so I can do the full course on my own. I'm listening to Berkeley's OS lectures right now but I would love a course built around xv6.
- ash 13y agoThis page has links to 6.828 lecture videos (2011): http://pdos.csail.mit.edu/6.828/2011/schedule.html http://pdos.csail.mit.edu/6.828/2011/schedule.html
- Dewie 13y agoI'm not knowledgeable about different philosophies when it comes to OS's, but it seems that pretty much everyone thinks that being Unix-like is a good thing. Is that true? Or is there some people that think that some different philosophy that is perhaps more or less mutually exclusive with regards to being Unix-like is better? I don't even know if that last question makes sense.
- dsr_ 13y agoUNIX-like is known to work. That is not the same as "good", but it may be close enough. UNIX-like includes philosophies such as "almost everything is a file" and "use human-readable (and writable!) plain text formats by default" and "there will be a stdin, stdout, and stderr; use them all". More than anything else, I think that a UNIX is defined by the idea that any end-user might decide to become a programmer at any moment, and this is OK and perhaps even desirable.
- userbinator 13y ago> UNIX is defined by the idea that any end-user might decide to become a programmer at any moment, and this is OK and perhaps even desirable. Very, very desirable in my view; in contrast to the raging trend of simplification/hiding going on these days, trying to turn computers into sealed-box appliances "you're not supposed to know how it works" that give little in the way of things that can get people interested in learning more about them, I find classic UNIX to be very open in encouraging users to learn more about the system.
- ChuckMcM 13y agoThere are of course a bazillion ideas about OS design, there used to be entire conferences on them :-) One of the features of UNIX was that it was designed by taking a few very simple ideas and more importantly simple constructs, and then building more complex constructs out of them. It made it possible for a handful of researchers to write all the bits of an OS needed to get to a multi-user prompt. Which at the time other examples had giant teams of engineers associated with them. That made it easy to understand at a fundamental level and you could reason about the effects various things would have on it should you make them. One of my favorite books is "Operating Systems Concepts" which has a great survey of the features of any good operating system. So is being Unix-like better? Yes if you want a simple robust operating system. No if you want a hard real-time operating system. Yes if you want a system you can flexibly add devices too. No if you want to build a security model based on capabilities. Yes if you want it to work on a small architectures with stack support. No if your processor doesn't support stacks or the C model of frame pointers. In general though understanding what an Operating system has to do is priceless for a CS person to have in their toolkit.
- contingencies 13y agoAlternatively, historical codebases of various unices at http://www.tuhs.org/archive_sites.html http://www.tuhs.org/archive_sites.html
- clown_penis 13y agoMIT has essentially recreated XINU. BFD. http://en.wikipedia.org/wiki/Xinu http://en.wikipedia.org/wiki/Xinu
- csense 13y agoIf "Xv6's use of the x86 [is intended to make] it more relevant to students' experience," then why use AT&T assembly language syntax rather than Intel?
- rsc 13y agoBecause the GNU toolchain uses AT&T syntax and most students have the GNU toolchain readily available?
- csense 13y ago> GNU toolchain uses AT&T syntax But also accepts Intel syntax with a directive, and even admits "almost all 80386 documents use Intel syntax" [1]. As for user acceptance, a poll on the xkcd forums shows that the vast majority of users prefer Intel syntax [2]. That takes care of the political/network effects arguments. As far as the actual technical arguments, the position against AT&T syntax is obvious and overwhelming -- the unnaturalness of src,dest operand order; the lazyness of the parser's authors at the expense of its users by requiring percents or dollar signs to indicate token types; the usually-redundant size postfix; and the utter incompatibility with basically all non-GNU x86 documentation. [1] https://sourceware.org/binutils/docs/as/i386_002dVariations.html#i386_002dVariations https://sourceware.org/binutils/docs/as/i386_002dVariations.... [2] http://forums.xkcd.com/viewtopic.php?f=40&t=19558 http://forums.xkcd.com/viewtopic.php?f=40&t=19558
- tankenmate 13y agoIf you are running this on your laptop and want to save your battery then use this patch... http://pastebin.com/BxXDDtTh http://pastebin.com/BxXDDtTh
- akkartik 13y agoInteresting. How does the processor restart after hlt?
- tankenmate 13y agoAny interrupt will take the processor out of hlt state; i.e. hit a key on the keyboard, it will fire an interrupt and the processor will start processing again. The clock tick will also fire an interrupt. Once it fires an interrupt it will start searching for runnable processes again; e.g. a process can now return from a read() from it's STDIN tty now that you have typed a key.
- swetland 13y agoThere's some inefficiency here in that wake() doesn't cause the other cpu to wake immediately from hlt so it'll remain halted until the next timer tick or whatnot. It'd be fun to add some scheduler stats and poke around with things a bit. Some really trivial cprintf() tracing in scheduler() implies that single processes can ping-pong between CPUs, but the printing itself is slow and invasive and could possibly be the cause.
- swetland 13y agoNice catch. Actually speeds everything up a bunch. I did it a bit more tersely with: if (p == &ptable.proc[NPROC]) hlt(); after the sti() and initializing p to 0;
- userbinator 13y agoI remember in the pre-Pentium II days people cared rather little about power consumption (TDPs were only a few watts or maybe a dozen at most), and the idle process was basically a very tight infinite loop. Around the time of the PII people (especially overclockers) started noticing that the CPU would stay cooler when halted, and thus started a little industry of "CPU cooling software"... there's an interesting historical page on that here: http://estu.nit.ac.jp/~e982457/other/cpuidle/idle.htm http://estu.nit.ac.jp/~e982457/other/cpuidle/idle.htm A modern processor, on the other hand, takes a few watts halted and a hundred or more when active, and transitions between these states many many times per second. How much things have changed...
- chris_wot 13y agoThe link to the schedule points to: http://pdos.csail.mit.edu/6.828/schedule.html http://pdos.csail.mit.edu/6.828/schedule.html It should be: http://pdos.csail.mit.edu/6.828/2012/schedule.html http://pdos.csail.mit.edu/6.828/2012/schedule.html
- swetland 13y agoBecause splatting object files and intermediates all over the same directories as the source drives me nuts, I shuffled things so that it's a bit easier to find things (splitting source into kernel/, user/, ulib/, tools/, and include/ directories) and generated files are collected into kobj/, uobj/, out/, and fs/ directories. https://github.com/swetland/xv6 https://github.com/swetland/xv6 I also tweaked the scheduler to halt if there's nothing to schedule, per tankenmate's suggestion. A side-effect of this change is the generate-source-code-printout targets were rather severely broken, so they're simply removed for the moment.
- singular 13y agoFor anyone else who is playing with this in OS X Mavericks, I've written a blog post describing how to get things working on it (there are a couple steps that need to be tweaked.) - http://blog.ljs.io/post/71424794630/xv6 http://blog.ljs.io/post/71424794630/xv6 Hopefully this is useful to somebody and isn't too blatantly link whorey :)
- samograd 13y agoI thought this was pretty awesome when I found it a couple of years ago, especially since it's multicore.