42 ms·

Using Rust for an Undergraduate OS Course

126 points by brson 13y ago

rdtsc 13y ago
> Go. Go is a systems programming language Not those kinds of "systems". It is funny, it seems initially it was meant to be those kind of "systems", and then it pivoted as we like to say, to become a "distributed-server-network-backend systems" not "hardware-kernel-OS" kind of systems. And then creators kinds of winged it and remarked how "well, that's what we meant when we said systems". As for Rust, yeah, there are already a few projects trying to build a kernel or drivers in Rust. Here is one for example: https://github.com/charliesome/rustboot https://github.com/charliesome/rustboot
- zhemao 13y ago
  I think that is the definition of "systems" the class is using though. If you look at the programming assignments, they are all about implementing programs in userspace and not hacking on the kernel. Yeah, I was confused at first too. Not sure how you can call it an "OS class" if students never touch kernel code. Although, apparently one project group did manage to write a kernel in rust based off of rustboot (https://github.com/wbthomason/ironkernel https://github.com/wbthomason/ironkernel).
  - rdtsc 13y ago
    Yeah I am a little confused by it. Actually Go or Erlang would be a good choice for this "OS" class and they do seem to have adopted the other "systems" definition.
  - scott_s 13y ago
    It looks to be inspired by CMU's "Introduction to Computer Systems" course (http://www.cs.cmu.edu/~213/ http://www.cs.cmu.edu/~213/). The professors who teach that course wrote a textbook, "Computer Systems: A Programmer's Perspective" (http://csapp.cs.cmu.edu/public/students.html http://csapp.cs.cmu.edu/public/students.html). I was a TA for a year for a version of that course adapted to the school I did my PhD at. We did the same thing: this course became the required one, and OS became an elective. I was skeptical as well, but after TAing it for a year, I think it is absolutely the correct thing to do. This course instills an understanding of the whole stack of a computer system that exists in user-land: assembly; how programs are executed including the stack, data segments, dynamic memory; code generation; process management and concurrency; memory allocation; network and server programming. The point of the course is how different levels of the system interact, not just studying one component in isolation. It is the kind of course I wish I had as an undergrad; instead, I had to build that knowledge piecemeal during grad school. More people will do user-land systems programming than kernel systems programming, so I think it makes sense to make the user-land course required, and the kernel level one an elective.
- mseepgood 13y ago
  > it seems initially it was meant to be those kind of "systems" From the announcement talk: "And it's a systems language in the sense that we intend it to be used to write things like web servers" http://www.youtube.com/watch?v=rKnDgT73v8s http://www.youtube.com/watch?v=rKnDgT73v8s I don't think that Google ever wanted to develop a new OS kernel.
  - rdtsc 13y ago
    > From the announcement talk From the announcement blog (and that was the widely circulated quote in tech news not the video transcript): --- Go combines the development speed of working in a dynamic language like Python with the performance and safety of a compiled language like C or C++. [...] And the compiled code runs close to the speed of C [...] Go is a great language for systems programming with support for multi-processing ... --- http://google-opensource.blogspot.com/2009/11/hey-ho-lets-go.html http://google-opensource.blogspot.com/2009/11/hey-ho-lets-go... > I don't think that Google ever wanted to develop a new OS kernel. As the meme goes, they keep using the word "systems", I don't think it means what they think it means.
    - wglb 13y ago
      Jeez. These dudes were writing "systems" long before most folks here on HN were programming. To me, and to them, "systems programming" has a particular meaning that does not include kernels. Like it means assemblers, compilers, web servers.j I think they have more credibility about what it means than perhaps you do.
      - dbaupp 13y ago
        Before many HNers were even born too.
    - frou_dh 13y ago
      It's a tough crowd when being the authors of both the Unix and Plan 9 kernels isn't sufficient to get the benefit of the doubt on this matter!
- bascule 13y ago
  Rust is a high-level systems programming language. Go is a low-level managed language runtime
  - bjz_ 13y ago
    This is probably the best clarification I've read regarding the difference between Go and Rust. They operate in different domains, and any competition is out of confusion over definitions (the source of many needless disagreements throughout human history).
- tptacek 13y ago
  Other than the word "systems", was there any other indication that the Golang team meant to target OS kernels?
  - zura 13y ago
    Not sure about OS kernels, but their intent was to attract C++ programmers, although, they ended up mostly attracting python programmers.
  - coldtea 13y ago
    You mean, if we assume that the word "systems" doesn't have a well defined meaning, that especially the Go team would know all too well when it used it?
    - scott_s 13y ago
      I define "systems programming" to be programming which must interact directly with the kernel or devices. Doing so requires knowledge of how those systems work. It includes, but is not limited to, writing operating system kernels.
- pjmlp 13y ago
  I am more into D and Rust than Go nowadays, but I think Go can be used for writing an OS if one so desires. The language is quite similar to Oberon in features. You just need to provide a syscall package for bare hardware. If powerful single user graphical OSs were written in Oberon, why not Go?
- nl 13y ago
  Systems programming does not mean operating systems. To me it means "infrastructure" - things like web servers, database servers, or the wide range of distributed systems that are currently written in Java (eg, Hadoop).
johnhamlet 13y ago
"The Rust compiler normally disallows any code with race conditions" Lol what? Unless tasks cannot communicate at all, or they are scheduled entirely deterministically, how can the compiler eliminate race conditions? How can a compiler even determine what is a race conditions and what is intended non-determinstic behaviour.
- pcwalton 13y ago
  Usually we say data races to be clear. Rust has full support for shared memory, but the compiler ensures that you are not accessing it without locking if it's mutable. (If the memory is immutable you can access it without a lock.) You can of course have races in message passing, or the disk, or the network, etc.
  - johnhamlet 13y ago
    ...but you can still have races with locking. Data races don't mean data corruption. It simply means the result is dependent on the scheduler or something else out of your control. Two threads set a shared variable to different values. They both lock it before the set it. But that's still a data race, as which thread write first is dependent on the scheduler.
    - dbaupp 13y ago
      No compiler can reason about whether some high-level piece of code is semantically order-of-execution dependent or not; not every non-deterministic piece of code is a race-condition.
      - johnhamlet 13y ago
        You: "not every non-deterministic piece of code is a race-condition" Wikipedia: "A race condition ... is ... where the output is dependent on the sequence or timing..." Boom! Headshot!
        13y ago
ijl 13y ago
Can anyone elaborate on how "Rust changes the way you think"?
- lifthrasiir 13y ago
  With Rust you need to be explicit about everything's lifetime. You cannot allocate some chunk of memory and freely borrow it; when borrowing you need a certificate (i.e. compiler check) that the borrowed reference has a shorter lifetime than that of the referenced memory. It is tremendously different from the absence of such guarantees (like C) or the GC-based weak guarantees (like Java and Go, which does not solve the logical memory leaks anyway [1]). [1] Not that Rust completely solves the logical memory leaks, but it makes the leaks explicit.
  - oscargrouch 13y ago
    c++ smart pointers under the hood? its actually cool.. to bake some of this directly in the language.. can prevent some memory copying by playing with raw pointers indirections under the hood in the lang runtime, etc.. But you already has to think like this when you use smart pointers in C++.. "who owns this reference?" etc.. But in C++ this is a library.. not a syntax.. point to rust for this
    - sanxiyn 13y ago
      You already have to think like this when you use smart pointers in C++! But C++ compilers don't help you with that! The Rust compiler does. Using Rust, you get surprised how often you get ownership wrong, because the compiler tells you every time. The same analogous code in C++ will still be wrong, but you wouldn't know, since the compiler tells you zilch.
      - carterschonwald 13y ago
        hello sanxiyn! (fyi all, sanxiyn is one of the rust compiler developers, unless i'm mixing up names :) )
2pxl 13y ago
"I'm not aware of any other language that has concurrency constructs as elegant and easy to use as Rust's spawn, and I don't know of any language that comes close to the race-free safety guarantees provided by Rust." I'd recommend having a look at Erlang, it allows you to start lightweight pseudo processes for concurrency and provides a robust data sharing model. Not so sure about low level access, we used C for those parts in the OS course I took.
- thirsteh 13y ago
  > I don't know of any language that comes close to the race-free safety guarantees provided by Rust. STM in Haskell.
- jerf 13y ago
  Erlang's a weird choice for an OS class. An Erlang program is run in a VM that aggressively hides from you the exact things you want exposed in an OS class. As a user, that's great, as a way of teaching OSes, it's not so great. Plus the VM simulates a machine that is quite unlike today's real machines from an OS perspective.
  - rdtsc 13y ago
    Granted for an OS class they just seem to be writting a lot of web and network servers. Given their assignment list Go or Erlang would actually be a fit. I for example remember writing a kernel driver and learning about virtual memory and io schedulers. We wrote file servers in the "networking" class but hey, that was years ago and a different college, so maybe I am just old.
    - tobiasu 13y ago
      Implementing Erlang on bare metal would be a good idea for an OS class, writing code in it, not so much.
zhemao 13y ago
> its (C's) lack of any intrinsic support for concurrency If you are writing the operating system kernel, who is going to give you concurrency support? Does rust running on the bare metal still have "spawn" and the other concurrency primitives?
- pcwalton 13y ago
  They aren't language primitives; they're just part of the library. So presumably you'd implement them yourself using a little bit of inline assembler, and they'd then be on equal footing with the versions in the Rust standard library.
- scott_s 13y ago
  They are not writing the kernel. See the other discussion in the thread regarding the term "systems programming".
z3phyr 13y ago
A similar kind of course for graphics programming is highly desirable. Also, I might like to add, PS4 toolchain fully supports LLVM- Clang. It might be wonderful if rustc starts running on it too...
asveikau 13y ago
I don't really get why this is a problem for a kernel programmer: > lack of any intrinsic support for concurrency An important part of an OS is that it implements support for concurrency - it builds processes and threads out of the raw materials such as page tables and timer interrupts. A kernel also builds its own spinlocks and higher-level waiting primitives. If you are relying on some high level language runtime to do the heavy lifting, is that really learning about how an operating system works? For a kernel I actually see a lack of language support for concurrency as a plus. You need something to sit below the higher level stuff.
- pcwalton 13y ago
  Rust doesn't have any built-in concurrency in the usual sense either (well, except for thread-locals and atomic intrinsics). What it provides is a few simple features (uniqueness and safe references) that allow programs like kernels to create their own safe, race-free abstractions. Instead of providing features directly ("scenario solving"), Rust's concurrency features are more like building blocks to help libraries build the features they need.
kps 13y ago
Is there any support for the article's assertion that C uses ‘=’ for assignment (and ‘==’ for equality) “to save memory”? dmr simply described it as “a matter of taste”¹ and hinted that he was simply used to typing ‘=’ for assignment². [Edit: And the compilers didn't keep source in memory, anyway.] ¹http://cm.bell-labs.com/who/dmr/chist.html http://cm.bell-labs.com/who/dmr/chist.html (also published in HOPL-II) ²History of Programming Languages II, Transcript of C Presentation, p692 (Nice to see Rust trying to bring progress to the C niche, BTW.)
- comex 13y ago
  Not to mention that Rust uses = and == the same way as C, although the expression "a = b" has unit type.
stephen_g 13y ago
This just doesn't make a lot of sense to me, given that almost all systems programming in the real world is done in C and C++, and given the complexity of OS kernels, we're not going to see a general purpose commercial grade one written in Rust either ever (if it falls out of popularity) or for ten to fifteen years (if it becomes very popular). This isn't like application or web programming - in comparison to web technologies it moves at a glacial pace. So I think teaching an OS course in Rust is going to leave them underprepared and be a big shock if the students actually go on to do systems programming on real systems. The other thing is that I did an OS course in C, where we got dirty playing around in the depths of the Linux kernel, replacing the scheduler, writing a filesystem, and so on. It was damn hard, but probably one of the only courses in Uni that really pushed me as a programmer generally (everything else I found easy). I think that it really helps to have experience in unsafe languages like C, just like you really need to do at least a bit of assembly when you teach computer organisation.
- pcwalton 13y ago
  I'm not sure the lack of safety is the pedagogically interesting part though. The interesting parts of writing a kernel for x86 are the low-level interfaces: assembly, I/O ports, the 8259A PIC, the fundamentals of scheduling, writing a slab allocator, and so forth. Rust allows unfettered access to those just as C does. The lack of safety in C just means that the projects take longer to debug because the compiler doesn't catch as many mistakes; I don't think it adds much to the learning experience.
- sixbrx 13y ago
  You may be imagining that it will be complete OS/systems written in non-C language from scratch, but it doesn't have to happen that way. With languages with no GC or optional GC, calling back and forth between the new language and C is easy and direct, and so writing only new (or maybe historically problematic) pieces of a system in the new language becomes feasible.
- pjmlp 13y ago
  Most of the real world uses C and C++, because of the vicious cycle he mentions on the post. You don't need C's lack of safety everywhere for doing an OS. There were OS written in Mesa, Modula-2, Modula-3, Object Pascal, Oberon, Ada and many others. C's strings and vectors are recipe for security exploits without any benefit for kernel programming. Or lack of type safety between compilation units. It is a very good thing students learn C and C++ aren't the only way to write an OS. Sometimes the world seems to forget other languages have been used for the same purpose.
nwg 13y ago
Cool, but i think most people would not call this an OS course.
axaxs 13y ago
It's a good start, but I'd honestly feel cheated if given such a course. I agree that the C tide may need to change, but using a language pre-stabilization AND that has literally no job opportunities is a real drag. So, the same goes for Go as well. And the fact that python or any other such language was even an option scared me just a little. I believe in Rust, and hope it succeeds, but don't feel this was a good avenue for it at this time.
vseloved 13y ago
I'm also teaching an OS class, and IMHO using some new shiny thing is not the right way to approach it, because the students will be put in an ideal imaginary world, and neither will face the real-world practice, nor understand how it all came to be the way it is now. I think, it may be a good follow-up course (like "System Programming in Rust") after the "normal" C-based course. I also don't agree with the notion that other option is "to teach students to write C code riddled with security vulnerabilities, memory leaks, and race conditions". This is exactly what such kind of course should teach them not to do, clearly pointing these problems and explaining how to deal with them. But not having exposure to that in the learning environment will only make them repeat these mistakes in the real world. Btw, I'm a Lisp programmer and not a C fan.
- pjmlp 13y ago
  It is however a very good idea to teach them that C is not the only way to write operating systems and it is currently on this position, because of a few historical accidents. Which we pay every day with extra tooling to cover for the language deficiencies on the area of security.
p0nce 13y ago
For balance, here is an experience report teaching D: http://forum.dlang.org/post/yutwxozxoqrfalzfswht@forum.dlang.org http://forum.dlang.org/post/yutwxozxoqrfalzfswht@forum.dlang...
UbuntuJon 13y ago
Why wasn't PHP on the shortlist?
dom96 13y ago
> I considered five possibilities: Java, Python, D, Go, and Rust. It's a pity that people rarely consider Nimrod (especially when it is far more suitable than the first two possibilities). Here is an example for anyone that is interested: https://github.com/dom96/nimkernel https://github.com/dom96/nimkernel
wbthomason 13y ago
I was one of the students in this course. If you're interested, I would recommend reading the professor's "course wrap-up"(http://rust-class.org/pages/course-wrapup.html http://rust-class.org/pages/course-wrapup.html) - it discusses his philosophy going into the semester. In particular, it addresses the most common comment I've seen in this thread, namely the rationale behind choosing to focus on general systems coding rather than kernel hacking. Personally, I found the course enjoyable and worthwhile. It was not what I was expecting, and I was admittedly disappointed that we didn't go much into actual bare-metal programming. That said, I found it to be a good introduction to the concepts of systems programming, with a few detours. As I am interested in OS coding, I'm performing my own independent study to learn the material which might have been covered in a more "traditional" OS course, but the course did a solid job of covering concurrency, memory allocation, etc. As for the choice of Rust: I really like the language, and plan to continue using it for personal projects. I'm not sure it was quite mature enough for exclusive use this past semester (the 0.7 to 0.8 switch midway through threw the TAs for a bit of a loop), but I've been very impressed so far, and think that this coming semester of the course (which will also use Rust) should have a much better time of it. The one annoyance I had with Rust was the lack of documentation, but the Rust community (including a fair number of those in this thread) were incredibly helpful, and I'm currently working on a project to provide an alternate tutorial, using a series of small "stepping-stone" programs to build from "Hello, World" to a MapReduce implementation. Personally, I think I might have picked either Go or a blend of C and Rust for this past semester, but Rust is becoming a better choice as time proceeds. I was also one of the members on the student team that made the ARM kernel in Rust. Due to the timing of the project (right in the middle of preparation for final exams and the rest of our end-of-semester project due dates) and some "lazily evaluated" work ethic on the part of our team, it was fairly rushed, and is much less complete and polished than I would like. I'm planning on continuing work on it. Though I would disagree with the assertion that separating Rust from its runtime is as "painless" as it has been said to be, it wasn't terribly difficult, and solutions for that particular problem are also constantly being improved.
rha16 13y ago
Seems like a fun course. Just not an OS course.
dave1629 13y ago
I'm the instructor for the class (and author of the post). I'll try and clarify a few things here. If anyone is interested in participating in the next version of the course (which starts Jan 14), please submit the form here: http://rust-class.org/pages/spring2014.html http://rust-class.org/pages/spring2014.html - "Operating systems" vs. "Systems programming" If your definition of an operating systems course is a course where you implement your own OS or hack on the Linux kernel, this wasn't an operating systems course. But, most people have a broader interpretation of operating systems courses today, to include courses where you learn about the layers between high-level programs and physical things, and about how to build robust, scalable, and secure computing systems. This is the second type of course. Deciding do a course that was not entirely focused on building an OS was not related to the choice of using Rust (I already knew I didn't want to do a build-an-OS course before thinking about which language to use), and I don't think its controversial (this is what the majority of top programs already do). I discuss more about this in the general course wrap-up: http://rust-class.org/pages/course-wrapup.html http://rust-class.org/pages/course-wrapup.html - Eliminating "Race Conditions" You are right in pointing out that Rust doesn't eliminate all race conditions (and no language that allows multiple threads and any interaction with the the external world really could do this). My wording here was very careless. What Rust does is use language/compiler mechanisms to eliminate the kinds of pernicious data races (multiple threads reading and writing the same mutable state in uncontrolled ways) that are a very common and hard to find and fix problems in most multi-threaded programs. - C's assignment operator My perhaps somewhat hyperbolic diatribe about C using ‘=‘ for assignment is meant to illustrate how design decisions C's designers made for good reasons given the computing systems they were using in the 1960/1970s, would not be the best decisions if one was designing a language from scratch today. I think the choice of ‘=‘ illustrates this well, but there are dozens of other more serious issues in C's design that were good or necessary choices in 1972, but are undesirable legacies today: not having bounds checking, not specifying the order of evaluation for many constructs, allowing arbitrary and unchecked type casting, unsafe memory management, etc. Other languages (including Rust) that are strongly influenced by C syntax have also adopted the ‘=‘ symbol for assignment, but that doesn’t make it a good thing. For languages like Rust that are targeting experienced programmers it is probably the right choice; for languages like Python that are intended as first languages, it is really unfortunate, and many smart people who might otherwise turn out to be talented programmers are unnecessarily put-off by this. (I don't have concrete data to support this, but from having over 350,000 students in my open intro CS course that uses Python, I have plenty of anecdotal experience with students being confused by this.) Vocational value This was a course at a public university, so its content should not be primarily driven by immediate vocational concerns. There's nothing wrong with vocational courses, where the primary goal of the course is to improve the immediate job prospects of students who take it, but was not and should not be the goal of courses at public universities. (That said, the students entering this class are already very well qualified for the job market, and many of the 4th years in the class already have job offers, so are not in danger of not being able to get an interesting job since they only have C experience from one previous course. Knowing Rust has actually been helpful for some students in the job market, and I don't think having less C experience is a major issue for many positions.) From a purely pragmatic viewpoint, the majority of the costs of the course are not covered from student’s tuition, but are paid by the US and Virginia taxpayers. So, our main goal should be to do what we can to enable and encourage students to do things that make the world a better place (the best ways to do that, of course, are debatable, and maybe it is learning how to hack the Linux kernel, but I think that’s a much tougher argument to make.)

126 points by brson 13y ago

rdtsc 13y ago

> Go. Go is a systems programming language Not those kinds of "systems". It is funny, it seems initially it was meant to be those kind of "systems", and then it pivoted as we like to say, to become a "distributed-server-network-backend systems" not "hardware-kernel-OS" kind of systems. And then creators kinds of winged it and remarked how "well, that's what we meant when we said systems". As for Rust, yeah, there are already a few projects trying to build a kernel or drivers in Rust. Here is one for example: https://github.com/charliesome/rustboot https://github.com/charliesome/rustboot

zhemao 13y ago
I think that is the definition of "systems" the class is using though. If you look at the programming assignments, they are all about implementing programs in userspace and not hacking on the kernel. Yeah, I was confused at first too. Not sure how you can call it an "OS class" if students never touch kernel code. Although, apparently one project group did manage to write a kernel in rust based off of rustboot (https://github.com/wbthomason/ironkernel https://github.com/wbthomason/ironkernel).
- rdtsc 13y ago
  Yeah I am a little confused by it. Actually Go or Erlang would be a good choice for this "OS" class and they do seem to have adopted the other "systems" definition.
- scott_s 13y ago
  It looks to be inspired by CMU's "Introduction to Computer Systems" course (http://www.cs.cmu.edu/~213/ http://www.cs.cmu.edu/~213/). The professors who teach that course wrote a textbook, "Computer Systems: A Programmer's Perspective" (http://csapp.cs.cmu.edu/public/students.html http://csapp.cs.cmu.edu/public/students.html). I was a TA for a year for a version of that course adapted to the school I did my PhD at. We did the same thing: this course became the required one, and OS became an elective. I was skeptical as well, but after TAing it for a year, I think it is absolutely the correct thing to do. This course instills an understanding of the whole stack of a computer system that exists in user-land: assembly; how programs are executed including the stack, data segments, dynamic memory; code generation; process management and concurrency; memory allocation; network and server programming. The point of the course is how different levels of the system interact, not just studying one component in isolation. It is the kind of course I wish I had as an undergrad; instead, I had to build that knowledge piecemeal during grad school. More people will do user-land systems programming than kernel systems programming, so I think it makes sense to make the user-land course required, and the kernel level one an elective.
mseepgood 13y ago
> it seems initially it was meant to be those kind of "systems" From the announcement talk: "And it's a systems language in the sense that we intend it to be used to write things like web servers" http://www.youtube.com/watch?v=rKnDgT73v8s http://www.youtube.com/watch?v=rKnDgT73v8s I don't think that Google ever wanted to develop a new OS kernel.
- rdtsc 13y ago
  > From the announcement talk From the announcement blog (and that was the widely circulated quote in tech news not the video transcript): --- Go combines the development speed of working in a dynamic language like Python with the performance and safety of a compiled language like C or C++. [...] And the compiled code runs close to the speed of C [...] Go is a great language for systems programming with support for multi-processing ... --- http://google-opensource.blogspot.com/2009/11/hey-ho-lets-go.html http://google-opensource.blogspot.com/2009/11/hey-ho-lets-go... > I don't think that Google ever wanted to develop a new OS kernel. As the meme goes, they keep using the word "systems", I don't think it means what they think it means.
  - wglb 13y ago
    Jeez. These dudes were writing "systems" long before most folks here on HN were programming. To me, and to them, "systems programming" has a particular meaning that does not include kernels. Like it means assemblers, compilers, web servers.j I think they have more credibility about what it means than perhaps you do.
    - dbaupp 13y ago
      Before many HNers were even born too.
  - frou_dh 13y ago
    It's a tough crowd when being the authors of both the Unix and Plan 9 kernels isn't sufficient to get the benefit of the doubt on this matter!
bascule 13y ago
Rust is a high-level systems programming language. Go is a low-level managed language runtime
- bjz_ 13y ago
  This is probably the best clarification I've read regarding the difference between Go and Rust. They operate in different domains, and any competition is out of confusion over definitions (the source of many needless disagreements throughout human history).
tptacek 13y ago
Other than the word "systems", was there any other indication that the Golang team meant to target OS kernels?
- zura 13y ago
  Not sure about OS kernels, but their intent was to attract C++ programmers, although, they ended up mostly attracting python programmers.
- coldtea 13y ago
  You mean, if we assume that the word "systems" doesn't have a well defined meaning, that especially the Go team would know all too well when it used it?
  - scott_s 13y ago
    I define "systems programming" to be programming which must interact directly with the kernel or devices. Doing so requires knowledge of how those systems work. It includes, but is not limited to, writing operating system kernels.
pjmlp 13y ago
I am more into D and Rust than Go nowadays, but I think Go can be used for writing an OS if one so desires. The language is quite similar to Oberon in features. You just need to provide a syscall package for bare hardware. If powerful single user graphical OSs were written in Oberon, why not Go?
nl 13y ago
Systems programming does not mean operating systems. To me it means "infrastructure" - things like web servers, database servers, or the wide range of distributed systems that are currently written in Java (eg, Hadoop).

johnhamlet 13y ago

"The Rust compiler normally disallows any code with race conditions" Lol what? Unless tasks cannot communicate at all, or they are scheduled entirely deterministically, how can the compiler eliminate race conditions? How can a compiler even determine what is a race conditions and what is intended non-determinstic behaviour.

pcwalton 13y ago
Usually we say data races to be clear. Rust has full support for shared memory, but the compiler ensures that you are not accessing it without locking if it's mutable. (If the memory is immutable you can access it without a lock.) You can of course have races in message passing, or the disk, or the network, etc.
- johnhamlet 13y ago
  ...but you can still have races with locking. Data races don't mean data corruption. It simply means the result is dependent on the scheduler or something else out of your control. Two threads set a shared variable to different values. They both lock it before the set it. But that's still a data race, as which thread write first is dependent on the scheduler.
  - dbaupp 13y ago
    No compiler can reason about whether some high-level piece of code is semantically order-of-execution dependent or not; not every non-deterministic piece of code is a race-condition.
    - johnhamlet 13y ago
      You: "not every non-deterministic piece of code is a race-condition" Wikipedia: "A race condition ... is ... where the output is dependent on the sequence or timing..." Boom! Headshot!
      - 13y ago

ijl 13y ago

Can anyone elaborate on how "Rust changes the way you think"?

lifthrasiir 13y ago
With Rust you need to be explicit about everything's lifetime. You cannot allocate some chunk of memory and freely borrow it; when borrowing you need a certificate (i.e. compiler check) that the borrowed reference has a shorter lifetime than that of the referenced memory. It is tremendously different from the absence of such guarantees (like C) or the GC-based weak guarantees (like Java and Go, which does not solve the logical memory leaks anyway [1]). [1] Not that Rust completely solves the logical memory leaks, but it makes the leaks explicit.
- oscargrouch 13y ago
  c++ smart pointers under the hood? its actually cool.. to bake some of this directly in the language.. can prevent some memory copying by playing with raw pointers indirections under the hood in the lang runtime, etc.. But you already has to think like this when you use smart pointers in C++.. "who owns this reference?" etc.. But in C++ this is a library.. not a syntax.. point to rust for this
  - sanxiyn 13y ago
    You already have to think like this when you use smart pointers in C++! But C++ compilers don't help you with that! The Rust compiler does. Using Rust, you get surprised how often you get ownership wrong, because the compiler tells you every time. The same analogous code in C++ will still be wrong, but you wouldn't know, since the compiler tells you zilch.
    - carterschonwald 13y ago
      hello sanxiyn! (fyi all, sanxiyn is one of the rust compiler developers, unless i'm mixing up names :) )

2pxl 13y ago

"I'm not aware of any other language that has concurrency constructs as elegant and easy to use as Rust's spawn, and I don't know of any language that comes close to the race-free safety guarantees provided by Rust." I'd recommend having a look at Erlang, it allows you to start lightweight pseudo processes for concurrency and provides a robust data sharing model. Not so sure about low level access, we used C for those parts in the OS course I took.

thirsteh 13y ago
> I don't know of any language that comes close to the race-free safety guarantees provided by Rust. STM in Haskell.
jerf 13y ago
Erlang's a weird choice for an OS class. An Erlang program is run in a VM that aggressively hides from you the exact things you want exposed in an OS class. As a user, that's great, as a way of teaching OSes, it's not so great. Plus the VM simulates a machine that is quite unlike today's real machines from an OS perspective.
- rdtsc 13y ago
  Granted for an OS class they just seem to be writting a lot of web and network servers. Given their assignment list Go or Erlang would actually be a fit. I for example remember writing a kernel driver and learning about virtual memory and io schedulers. We wrote file servers in the "networking" class but hey, that was years ago and a different college, so maybe I am just old.
  - tobiasu 13y ago
    Implementing Erlang on bare metal would be a good idea for an OS class, writing code in it, not so much.

zhemao 13y ago

> its (C's) lack of any intrinsic support for concurrency If you are writing the operating system kernel, who is going to give you concurrency support? Does rust running on the bare metal still have "spawn" and the other concurrency primitives?

pcwalton 13y ago
They aren't language primitives; they're just part of the library. So presumably you'd implement them yourself using a little bit of inline assembler, and they'd then be on equal footing with the versions in the Rust standard library.
scott_s 13y ago
They are not writing the kernel. See the other discussion in the thread regarding the term "systems programming".

z3phyr 13y ago

A similar kind of course for graphics programming is highly desirable. Also, I might like to add, PS4 toolchain fully supports LLVM- Clang. It might be wonderful if rustc starts running on it too...

asveikau 13y ago

I don't really get why this is a problem for a kernel programmer: > lack of any intrinsic support for concurrency An important part of an OS is that it implements support for concurrency - it builds processes and threads out of the raw materials such as page tables and timer interrupts. A kernel also builds its own spinlocks and higher-level waiting primitives. If you are relying on some high level language runtime to do the heavy lifting, is that really learning about how an operating system works? For a kernel I actually see a lack of language support for concurrency as a plus. You need something to sit below the higher level stuff.

pcwalton 13y ago
Rust doesn't have any built-in concurrency in the usual sense either (well, except for thread-locals and atomic intrinsics). What it provides is a few simple features (uniqueness and safe references) that allow programs like kernels to create their own safe, race-free abstractions. Instead of providing features directly ("scenario solving"), Rust's concurrency features are more like building blocks to help libraries build the features they need.

kps 13y ago

Is there any support for the article's assertion that C uses ‘=’ for assignment (and ‘==’ for equality) “to save memory”? dmr simply described it as “a matter of taste”¹ and hinted that he was simply used to typing ‘=’ for assignment². [Edit: And the compilers didn't keep source in memory, anyway.] ¹http://cm.bell-labs.com/who/dmr/chist.html http://cm.bell-labs.com/who/dmr/chist.html (also published in HOPL-II) ²History of Programming Languages II, Transcript of C Presentation, p692 (Nice to see Rust trying to bring progress to the C niche, BTW.)

comex 13y ago
Not to mention that Rust uses = and == the same way as C, although the expression "a = b" has unit type.

stephen_g 13y ago

This just doesn't make a lot of sense to me, given that almost all systems programming in the real world is done in C and C++, and given the complexity of OS kernels, we're not going to see a general purpose commercial grade one written in Rust either ever (if it falls out of popularity) or for ten to fifteen years (if it becomes very popular). This isn't like application or web programming - in comparison to web technologies it moves at a glacial pace. So I think teaching an OS course in Rust is going to leave them underprepared and be a big shock if the students actually go on to do systems programming on real systems. The other thing is that I did an OS course in C, where we got dirty playing around in the depths of the Linux kernel, replacing the scheduler, writing a filesystem, and so on. It was damn hard, but probably one of the only courses in Uni that really pushed me as a programmer generally (everything else I found easy). I think that it really helps to have experience in unsafe languages like C, just like you really need to do at least a bit of assembly when you teach computer organisation.

pcwalton 13y ago
I'm not sure the lack of safety is the pedagogically interesting part though. The interesting parts of writing a kernel for x86 are the low-level interfaces: assembly, I/O ports, the 8259A PIC, the fundamentals of scheduling, writing a slab allocator, and so forth. Rust allows unfettered access to those just as C does. The lack of safety in C just means that the projects take longer to debug because the compiler doesn't catch as many mistakes; I don't think it adds much to the learning experience.
sixbrx 13y ago
You may be imagining that it will be complete OS/systems written in non-C language from scratch, but it doesn't have to happen that way. With languages with no GC or optional GC, calling back and forth between the new language and C is easy and direct, and so writing only new (or maybe historically problematic) pieces of a system in the new language becomes feasible.
pjmlp 13y ago
Most of the real world uses C and C++, because of the vicious cycle he mentions on the post. You don't need C's lack of safety everywhere for doing an OS. There were OS written in Mesa, Modula-2, Modula-3, Object Pascal, Oberon, Ada and many others. C's strings and vectors are recipe for security exploits without any benefit for kernel programming. Or lack of type safety between compilation units. It is a very good thing students learn C and C++ aren't the only way to write an OS. Sometimes the world seems to forget other languages have been used for the same purpose.

nwg 13y ago

Cool, but i think most people would not call this an OS course.

axaxs 13y ago

It's a good start, but I'd honestly feel cheated if given such a course. I agree that the C tide may need to change, but using a language pre-stabilization AND that has literally no job opportunities is a real drag. So, the same goes for Go as well. And the fact that python or any other such language was even an option scared me just a little. I believe in Rust, and hope it succeeds, but don't feel this was a good avenue for it at this time.

vseloved 13y ago

I'm also teaching an OS class, and IMHO using some new shiny thing is not the right way to approach it, because the students will be put in an ideal imaginary world, and neither will face the real-world practice, nor understand how it all came to be the way it is now. I think, it may be a good follow-up course (like "System Programming in Rust") after the "normal" C-based course. I also don't agree with the notion that other option is "to teach students to write C code riddled with security vulnerabilities, memory leaks, and race conditions". This is exactly what such kind of course should teach them not to do, clearly pointing these problems and explaining how to deal with them. But not having exposure to that in the learning environment will only make them repeat these mistakes in the real world. Btw, I'm a Lisp programmer and not a C fan.

pjmlp 13y ago
It is however a very good idea to teach them that C is not the only way to write operating systems and it is currently on this position, because of a few historical accidents. Which we pay every day with extra tooling to cover for the language deficiencies on the area of security.

p0nce 13y ago

For balance, here is an experience report teaching D: http://forum.dlang.org/post/yutwxozxoqrfalzfswht@forum.dlang.org http://forum.dlang.org/post/yutwxozxoqrfalzfswht@forum.dlang...

UbuntuJon 13y ago

Why wasn't PHP on the shortlist?

dom96 13y ago

> I considered five possibilities: Java, Python, D, Go, and Rust. It's a pity that people rarely consider Nimrod (especially when it is far more suitable than the first two possibilities). Here is an example for anyone that is interested: https://github.com/dom96/nimkernel https://github.com/dom96/nimkernel

wbthomason 13y ago

I was one of the students in this course. If you're interested, I would recommend reading the professor's "course wrap-up"(http://rust-class.org/pages/course-wrapup.html http://rust-class.org/pages/course-wrapup.html) - it discusses his philosophy going into the semester. In particular, it addresses the most common comment I've seen in this thread, namely the rationale behind choosing to focus on general systems coding rather than kernel hacking. Personally, I found the course enjoyable and worthwhile. It was not what I was expecting, and I was admittedly disappointed that we didn't go much into actual bare-metal programming. That said, I found it to be a good introduction to the concepts of systems programming, with a few detours. As I am interested in OS coding, I'm performing my own independent study to learn the material which might have been covered in a more "traditional" OS course, but the course did a solid job of covering concurrency, memory allocation, etc. As for the choice of Rust: I really like the language, and plan to continue using it for personal projects. I'm not sure it was quite mature enough for exclusive use this past semester (the 0.7 to 0.8 switch midway through threw the TAs for a bit of a loop), but I've been very impressed so far, and think that this coming semester of the course (which will also use Rust) should have a much better time of it. The one annoyance I had with Rust was the lack of documentation, but the Rust community (including a fair number of those in this thread) were incredibly helpful, and I'm currently working on a project to provide an alternate tutorial, using a series of small "stepping-stone" programs to build from "Hello, World" to a MapReduce implementation. Personally, I think I might have picked either Go or a blend of C and Rust for this past semester, but Rust is becoming a better choice as time proceeds. I was also one of the members on the student team that made the ARM kernel in Rust. Due to the timing of the project (right in the middle of preparation for final exams and the rest of our end-of-semester project due dates) and some "lazily evaluated" work ethic on the part of our team, it was fairly rushed, and is much less complete and polished than I would like. I'm planning on continuing work on it. Though I would disagree with the assertion that separating Rust from its runtime is as "painless" as it has been said to be, it wasn't terribly difficult, and solutions for that particular problem are also constantly being improved.

rha16 13y ago

Seems like a fun course. Just not an OS course.

dave1629 13y ago

I'm the instructor for the class (and author of the post). I'll try and clarify a few things here. If anyone is interested in participating in the next version of the course (which starts Jan 14), please submit the form here: http://rust-class.org/pages/spring2014.html http://rust-class.org/pages/spring2014.html - "Operating systems" vs. "Systems programming" If your definition of an operating systems course is a course where you implement your own OS or hack on the Linux kernel, this wasn't an operating systems course. But, most people have a broader interpretation of operating systems courses today, to include courses where you learn about the layers between high-level programs and physical things, and about how to build robust, scalable, and secure computing systems. This is the second type of course. Deciding do a course that was not entirely focused on building an OS was not related to the choice of using Rust (I already knew I didn't want to do a build-an-OS course before thinking about which language to use), and I don't think its controversial (this is what the majority of top programs already do). I discuss more about this in the general course wrap-up: http://rust-class.org/pages/course-wrapup.html http://rust-class.org/pages/course-wrapup.html - Eliminating "Race Conditions" You are right in pointing out that Rust doesn't eliminate all race conditions (and no language that allows multiple threads and any interaction with the the external world really could do this). My wording here was very careless. What Rust does is use language/compiler mechanisms to eliminate the kinds of pernicious data races (multiple threads reading and writing the same mutable state in uncontrolled ways) that are a very common and hard to find and fix problems in most multi-threaded programs. - C's assignment operator My perhaps somewhat hyperbolic diatribe about C using ‘=‘ for assignment is meant to illustrate how design decisions C's designers made for good reasons given the computing systems they were using in the 1960/1970s, would not be the best decisions if one was designing a language from scratch today. I think the choice of ‘=‘ illustrates this well, but there are dozens of other more serious issues in C's design that were good or necessary choices in 1972, but are undesirable legacies today: not having bounds checking, not specifying the order of evaluation for many constructs, allowing arbitrary and unchecked type casting, unsafe memory management, etc. Other languages (including Rust) that are strongly influenced by C syntax have also adopted the ‘=‘ symbol for assignment, but that doesn’t make it a good thing. For languages like Rust that are targeting experienced programmers it is probably the right choice; for languages like Python that are intended as first languages, it is really unfortunate, and many smart people who might otherwise turn out to be talented programmers are unnecessarily put-off by this. (I don't have concrete data to support this, but from having over 350,000 students in my open intro CS course that uses Python, I have plenty of anecdotal experience with students being confused by this.) Vocational value This was a course at a public university, so its content should not be primarily driven by immediate vocational concerns. There's nothing wrong with vocational courses, where the primary goal of the course is to improve the immediate job prospects of students who take it, but was not and should not be the goal of courses at public universities. (That said, the students entering this class are already very well qualified for the job market, and many of the 4th years in the class already have job offers, so are not in danger of not being able to get an interesting job since they only have C experience from one previous course. Knowing Rust has actually been helpful for some students in the job market, and I don't think having less C experience is a major issue for many positions.) From a purely pragmatic viewpoint, the majority of the costs of the course are not covered from student’s tuition, but are paid by the US and Virginia taxpayers. So, our main goal should be to do what we can to enable and encourage students to do things that make the world a better place (the best ways to do that, of course, are debatable, and maybe it is learning how to hack the Linux kernel, but I think that’s a much tougher argument to make.)