6 ms·

Why Rust? Doesn't Microsoft have their own similar language, C#? And then isn't there some lore about the reason Windows Vista took so long to release was due t

by esaym 3y ago

Why Rust? Doesn't Microsoft have their own similar language, C#? And then isn't there some lore about the reason Windows Vista took so long to release was due to them trying to write the entire OS in C# or something and then having to redo everything?

chungy 3y ago
C# isn't even remotely similar. A managed language running in a heavy VM runtime versus a direct-to-machine-code compiled language...
- 29athrowaway 3y ago
  Have you heard of native AOT compilation?
  - saagarjha 3y ago
    Still requires a substantial runtime.
    - pjmlp 3y ago
      Which in many cases has already been proven by making it part of the kernel, even if Xerox, DEC/Olivetti, ETHZ, MSR projects weren't made available to wider audiences. One can do a Google, and push it no matter what, or give up and take the adoption path that is easier to bring the safety luddites along for the ride.
      - saagarjha 3y ago
        I don’t fundamentally oppose such a thing, but I assume doing so would require very careful effort. It’s much harder than integrating Rust.
        pjmlp 3y ago
        For me, Rust is a nicer Ada like effort, a kind of compromise. We can wait the usual progress rate of one funeral at a time until a new generation buys into fully managed OS, or another Google/Apple big spender forces them into developers no matter what, which most likely I will not live through. So we're left with the compromise of allowing only userspace for managed languages, while adopting something like Rust for the "only over my dead body folks" anti any form of automatic memory management languages. Given that even the success stories of bare metal deployments of managed runtimes doesn't change their minds anyway.
        saagarjha 3y ago
        Yeah idk why people are so against this, I guess old habits die hard.
  - deleted 3y ago
    [deleted]
  - stevefan1999 3y ago
    1. That still needs a GC, and thus heap space. Unless you make everything stackalloc which is impossible in kernel programming 2. It is heavily pinned by unnecessary metadata. Well, actually some maybe necesdary to facilitate GC but it is bloated anyway 3. RyuJIT is not as optimized as LLVM, and RyuJIT produced the AOT code. There in an abandoned project called LILAC attemping to use LLVM as a second AOT generator but it was never heard from again after 2019. 4. Native interoperability in .net requires JIT as well, to generate things like call sites and runtime thunks (function pointers to restore CLR context). Not only that wastes more resources but also that not all P/Invoke calls can be AOT compiled.
    - stevefan1999 3y ago
      oops here's some typo: 1. Well technically speaking, you need a unified heap space. Just like in Linux kernel you have vmalloc and kmalloc which does virtual memory and real, page-aligned physical memory. So this is technically means I will have to pre-allocate everything. But it is not gonna work, how do you save space? 2. "necessary". I should have enabled autocorrect on my phone, it does more good than harm this time if I had it on. 3. It's actually LLILC (https://github.com/dotnet/llilc https://github.com/dotnet/llilc). I just remembered how to pronounce it. But if the name is wrong, then you have to correct it.
    - pjmlp 3y ago
      - Singularity - Midori - Oberon - Mesa/Cedar - Topaz It is a matter of actually wanting to make it happen, or giving up to the voices that think otherwise.
      - omnicognate 3y ago
        That list just shows people have been experimenting with such things for decades, including in a major push with Midori: over 100 devs across Microsoft and Microsoft Research, and while things have been learned and fed into the development of runtimes and operating systems it hasn't changed the fact that all the major OS kernels are written in non-memory-safe languages. Rust, on the other hand, is starting to directly impact these kernels. Joe Duffy himself said with Midori they "started with C# and .NET [but] were forced to radically depart in the name of security, reliability, and performance" and that at one point they had 11 different garbage collectors. The jury is still out (and has been deliberating for an awfully long time) on whether you can build or even contribute to a successful, mainstream, general-purpose OS kernel in a garbage-collected language but a distinctive selling point of Rust is that you can start adding memory safety without having to. And the fact that this is actually starting to happen to more than one such kernel while Rust is still a fairly new language could be interpreted as evidence that maybe, just maybe, the GC really was the problem all along.
- lostmsu 3y ago
  I don't think runtime is that big. The classlib is.
kortilla 3y ago
C# is akin to Java, not Rust.
- pjmlp 3y ago
  Good then, given that PTC and Aicas have been seling bare metal JVMs for the last 20 years, with soft real time deployments.
nitinreddy88 3y ago
You should read about: https://en.wikipedia.org/wiki/Midori_(operating_system) https://en.wikipedia.org/wiki/Midori_(operating_system) Tested and Failed. There are lot of interesting learnings and blogposts from this project and lot of them are actually landed in different products of Microsoft
- pjmlp 3y ago
  Tested, and successfully run the Asian Bing network nodes during its lifetime. If anything, it failed at the Windows business unit politics. Unfortunely it lacked the kind of management that has pushed Java no matter what on Android, or either Swift or the high road on iOS.
CSMastermind 3y ago
> And then isn't there some lore about the reason Windows Vista took so long to release was due to them trying to write the entire OS in C# or something and then having to redo everything? It seems like you're confusing a few different things there. The 'redo' step that you're talking about happened because in the early 2000s Microsoft was getting blasted by security breaches left and right. Pretty much every senior developer in the Windows org was pulled to work on security patches. An in between update to the consumer version of Windows (codename Longhorn) was planned to hold customers over until the team had the time to do a proper iteration on the next OS. As the senior devs were wrapping up their security fixes and coming back to see about shipping Longhorn they decided that it wasn't up to their standards and more importantly they didn't think it was moving in the right direction because it didn't include all the security fixes they had been doing. Longhorn had essentially 'forked' XP pre-security fix and it was decided that it would be easier to throw out all of the Longhorn code and start fresh then to try and port all the security fixes over. So they just that: they threw out everything that had been worked on, 'reset' the project to be based on the latest code with all the security fixes present, and then started the development of Vista fresh.
- GeekyBear 3y ago
  > Longhorn had essentially 'forked' XP pre-security fix and it was decided that it would be easier to throw out all of the Longhorn code and start fresh then to try and port all the security fixes over. Microsoft internal emails from Jim Allchin (who ran Windows at the time) to Bill Gates reveal that the issue was performance. > LH is a pig and I don’t see any solution to this problem. https://web.archive.org/web/20210427171552/http://blog.seattlepi.com/microsoft/2007/01/10/jim-allchins-mac-message-the-full-text/ https://web.archive.org/web/20210427171552/http://blog.seatt... Eventually, the solution was to drop the use of a garbage collected runtime (desired for memory safety) and go back to C/C++ (necessary for performance). Now they are moving to a language that gives you the memory safety without taking the performance hit.
  - StressedDev 3y ago
    Longhorn was slow but I doubt garbage collection was the main cause for its poor performance. First, it was mostly written in C and C++ (remember, Microsoft does not rewrite Windows, it evolves it). Second, the reason it was slow was some teams checked in code which was not ready. What I mean is the code was buggy, had exceptionally poor performance, etc. The code should never have been shared with the entire Windows organization. Note teams could have private source code repositories and Windows builds. Changes did not have to be shared with the entire Windows team. The better teams shared their changes when they worked and were ready. On Vista, a lot of teams shared things which were far from ready. Note that this was unusual and did not happen on Windows XP.
xedrac 3y ago
Maybe because it solves about 80% of the PITA security bugs that have beset Windows and other systems software since the dawn of software development? C# is garbage collected with a big runtime. It's not even an option.
- pjmlp 3y ago
  Xerox PARC would think otherwise, pity it lost to a PDP-11 OS.
otabdeveloper4 3y ago
Any large company has bored people who will sneak in random languages and frameworks. The Windows network stack used to have an embedded Prolog interpreter, just because. I'm willing to bet there's a small Brainfuck codebase somewhere in the depths of Google.
StressedDev 3y ago
Microsoft never tried to rewrite Windows in C# during the Vista project. I know this because I worked on Vista and my team wrote code in C++. Vista did include some new C# APIs and Frameworks. The best ones were Windows Presentation Foundation and PowerShell (it uses a lot of .NET technologies). Vista took a long time for a lot of reasons. I do not know all of them but my observation was there were a few basic problems: 1. Teams were allowed to check in buggy code. This led Vista to be unstable while it was being developed. Note the Kernel code was usually very good. The shell was frequently difficult to use because of the bugs and because of poor performance. Note that when it shipped, it worked much better than it did during development. I remember being shocked at how well it worked. I also remember going back to Windows XP and realizing I actually liked Vista better (it had a better user interface, and I liked the improved Windows Update app). 2. Very poor project management. Basically, no one knew what needed to be done or how long it would take. 3. A lot of overly ambitious projects and features. Some teams really tried to do revolutionary things. Sometimes they succeeded (The Windows Display Driver Model is an example of this). Often, they failed (Media Foundation is a great example of a mediocre API which came out of Vista). 4. Some teams took dependencies on immature APIs or frameworks. The problem was the framework and the applications using it were being developed at the same time. This led to a lot of reworked because applications kept on having to be updated because their dependencies changed or were cancelled. 5. Poor leadership - Will Poole led the Window Client team (AKA the desktop version of Windows). Wille Poole previously led the Windows Media player team. My impression was he valued political skill, politics and empire building over competence and technical excellence. After Windows Vista shipped, he was "promoted" to working on Windows for Emerging Markets. After that, he "managed" his administrative assistant. He then "retired". I am sure I missed a lot of things and I certainly do not know everything because I worked on a small part of a huge project. Windows had thousands of software engineers, program managers, testers and managers and they worked on a lot of different things.
- fifteen1506 3y ago
  I preferred Vista interface to Windows 7. Vista SP1 fixed most problems -- but you needed 2GB RAM.
- cbozeman 3y ago
neonsunset 3y ago
C# is much more capable and fitting this purpose today than 10 years ago (perf, codegen quality and infrastructure have improved by orders of magnitude). However, it cannot compete with Rust when it comes to systems programming. Mainly because it has different tradeoffs and its high level abstractions and other features like LINQ, interfaces, non-struct generics and async/await are very much not zero-cost, unlike in Rust. In addition, it does not offer crucial features one would want in kernel development: deterministic memory usage, compile-time safety guarantees for writing concurrent data structures and general systems-programming-first language design. While C# is a perfectly viable choice for writing userspace OS components, applications, UI and etc., it is a bad choice for kernelspace when C++ or Rust exist.
fulafel 3y ago
Rust is gaining in mindshare and C++ background curmudgeons still believe OS code can't be in GC supporting languages. But maybe it's better not to look a gift horse in the mouth, replacing unsafe code with something better still works toward slowly establishing mem safety as table stakes in OS code.

by esaym 3y ago

chungy 3y ago

C# isn't even remotely similar. A managed language running in a heavy VM runtime versus a direct-to-machine-code compiled language...

29athrowaway 3y ago
Have you heard of native AOT compilation?
- saagarjha 3y ago
  Still requires a substantial runtime.
  - pjmlp 3y ago
    Which in many cases has already been proven by making it part of the kernel, even if Xerox, DEC/Olivetti, ETHZ, MSR projects weren't made available to wider audiences. One can do a Google, and push it no matter what, or give up and take the adoption path that is easier to bring the safety luddites along for the ride.
    - saagarjha 3y ago
      I don’t fundamentally oppose such a thing, but I assume doing so would require very careful effort. It’s much harder than integrating Rust.
      - pjmlp 3y ago
        For me, Rust is a nicer Ada like effort, a kind of compromise. We can wait the usual progress rate of one funeral at a time until a new generation buys into fully managed OS, or another Google/Apple big spender forces them into developers no matter what, which most likely I will not live through. So we're left with the compromise of allowing only userspace for managed languages, while adopting something like Rust for the "only over my dead body folks" anti any form of automatic memory management languages. Given that even the success stories of bare metal deployments of managed runtimes doesn't change their minds anyway.
        saagarjha 3y ago
        Yeah idk why people are so against this, I guess old habits die hard.
- deleted 3y ago
  [deleted]
- stevefan1999 3y ago
  1. That still needs a GC, and thus heap space. Unless you make everything stackalloc which is impossible in kernel programming 2. It is heavily pinned by unnecessary metadata. Well, actually some maybe necesdary to facilitate GC but it is bloated anyway 3. RyuJIT is not as optimized as LLVM, and RyuJIT produced the AOT code. There in an abandoned project called LILAC attemping to use LLVM as a second AOT generator but it was never heard from again after 2019. 4. Native interoperability in .net requires JIT as well, to generate things like call sites and runtime thunks (function pointers to restore CLR context). Not only that wastes more resources but also that not all P/Invoke calls can be AOT compiled.
  - stevefan1999 3y ago
    oops here's some typo: 1. Well technically speaking, you need a unified heap space. Just like in Linux kernel you have vmalloc and kmalloc which does virtual memory and real, page-aligned physical memory. So this is technically means I will have to pre-allocate everything. But it is not gonna work, how do you save space? 2. "necessary". I should have enabled autocorrect on my phone, it does more good than harm this time if I had it on. 3. It's actually LLILC (https://github.com/dotnet/llilc https://github.com/dotnet/llilc). I just remembered how to pronounce it. But if the name is wrong, then you have to correct it.
  - pjmlp 3y ago
    - Singularity - Midori - Oberon - Mesa/Cedar - Topaz It is a matter of actually wanting to make it happen, or giving up to the voices that think otherwise.
    - omnicognate 3y ago
      That list just shows people have been experimenting with such things for decades, including in a major push with Midori: over 100 devs across Microsoft and Microsoft Research, and while things have been learned and fed into the development of runtimes and operating systems it hasn't changed the fact that all the major OS kernels are written in non-memory-safe languages. Rust, on the other hand, is starting to directly impact these kernels. Joe Duffy himself said with Midori they "started with C# and .NET [but] were forced to radically depart in the name of security, reliability, and performance" and that at one point they had 11 different garbage collectors. The jury is still out (and has been deliberating for an awfully long time) on whether you can build or even contribute to a successful, mainstream, general-purpose OS kernel in a garbage-collected language but a distinctive selling point of Rust is that you can start adding memory safety without having to. And the fact that this is actually starting to happen to more than one such kernel while Rust is still a fairly new language could be interpreted as evidence that maybe, just maybe, the GC really was the problem all along.
lostmsu 3y ago
I don't think runtime is that big. The classlib is.

kortilla 3y ago

C# is akin to Java, not Rust.

pjmlp 3y ago
Good then, given that PTC and Aicas have been seling bare metal JVMs for the last 20 years, with soft real time deployments.

nitinreddy88 3y ago

You should read about: https://en.wikipedia.org/wiki/Midori_(operating_system) https://en.wikipedia.org/wiki/Midori_(operating_system) Tested and Failed. There are lot of interesting learnings and blogposts from this project and lot of them are actually landed in different products of Microsoft

pjmlp 3y ago
Tested, and successfully run the Asian Bing network nodes during its lifetime. If anything, it failed at the Windows business unit politics. Unfortunely it lacked the kind of management that has pushed Java no matter what on Android, or either Swift or the high road on iOS.

CSMastermind 3y ago

> And then isn't there some lore about the reason Windows Vista took so long to release was due to them trying to write the entire OS in C# or something and then having to redo everything? It seems like you're confusing a few different things there. The 'redo' step that you're talking about happened because in the early 2000s Microsoft was getting blasted by security breaches left and right. Pretty much every senior developer in the Windows org was pulled to work on security patches. An in between update to the consumer version of Windows (codename Longhorn) was planned to hold customers over until the team had the time to do a proper iteration on the next OS. As the senior devs were wrapping up their security fixes and coming back to see about shipping Longhorn they decided that it wasn't up to their standards and more importantly they didn't think it was moving in the right direction because it didn't include all the security fixes they had been doing. Longhorn had essentially 'forked' XP pre-security fix and it was decided that it would be easier to throw out all of the Longhorn code and start fresh then to try and port all the security fixes over. So they just that: they threw out everything that had been worked on, 'reset' the project to be based on the latest code with all the security fixes present, and then started the development of Vista fresh.

GeekyBear 3y ago
> Longhorn had essentially 'forked' XP pre-security fix and it was decided that it would be easier to throw out all of the Longhorn code and start fresh then to try and port all the security fixes over. Microsoft internal emails from Jim Allchin (who ran Windows at the time) to Bill Gates reveal that the issue was performance. > LH is a pig and I don’t see any solution to this problem. https://web.archive.org/web/20210427171552/http://blog.seattlepi.com/microsoft/2007/01/10/jim-allchins-mac-message-the-full-text/ https://web.archive.org/web/20210427171552/http://blog.seatt... Eventually, the solution was to drop the use of a garbage collected runtime (desired for memory safety) and go back to C/C++ (necessary for performance). Now they are moving to a language that gives you the memory safety without taking the performance hit.
- StressedDev 3y ago
  Longhorn was slow but I doubt garbage collection was the main cause for its poor performance. First, it was mostly written in C and C++ (remember, Microsoft does not rewrite Windows, it evolves it). Second, the reason it was slow was some teams checked in code which was not ready. What I mean is the code was buggy, had exceptionally poor performance, etc. The code should never have been shared with the entire Windows organization. Note teams could have private source code repositories and Windows builds. Changes did not have to be shared with the entire Windows team. The better teams shared their changes when they worked and were ready. On Vista, a lot of teams shared things which were far from ready. Note that this was unusual and did not happen on Windows XP.

xedrac 3y ago

Maybe because it solves about 80% of the PITA security bugs that have beset Windows and other systems software since the dawn of software development? C# is garbage collected with a big runtime. It's not even an option.

pjmlp 3y ago
Xerox PARC would think otherwise, pity it lost to a PDP-11 OS.

otabdeveloper4 3y ago

Any large company has bored people who will sneak in random languages and frameworks. The Windows network stack used to have an embedded Prolog interpreter, just because. I'm willing to bet there's a small Brainfuck codebase somewhere in the depths of Google.

StressedDev 3y ago

Microsoft never tried to rewrite Windows in C# during the Vista project. I know this because I worked on Vista and my team wrote code in C++. Vista did include some new C# APIs and Frameworks. The best ones were Windows Presentation Foundation and PowerShell (it uses a lot of .NET technologies). Vista took a long time for a lot of reasons. I do not know all of them but my observation was there were a few basic problems: 1. Teams were allowed to check in buggy code. This led Vista to be unstable while it was being developed. Note the Kernel code was usually very good. The shell was frequently difficult to use because of the bugs and because of poor performance. Note that when it shipped, it worked much better than it did during development. I remember being shocked at how well it worked. I also remember going back to Windows XP and realizing I actually liked Vista better (it had a better user interface, and I liked the improved Windows Update app). 2. Very poor project management. Basically, no one knew what needed to be done or how long it would take. 3. A lot of overly ambitious projects and features. Some teams really tried to do revolutionary things. Sometimes they succeeded (The Windows Display Driver Model is an example of this). Often, they failed (Media Foundation is a great example of a mediocre API which came out of Vista). 4. Some teams took dependencies on immature APIs or frameworks. The problem was the framework and the applications using it were being developed at the same time. This led to a lot of reworked because applications kept on having to be updated because their dependencies changed or were cancelled. 5. Poor leadership - Will Poole led the Window Client team (AKA the desktop version of Windows). Wille Poole previously led the Windows Media player team. My impression was he valued political skill, politics and empire building over competence and technical excellence. After Windows Vista shipped, he was "promoted" to working on Windows for Emerging Markets. After that, he "managed" his administrative assistant. He then "retired". I am sure I missed a lot of things and I certainly do not know everything because I worked on a small part of a huge project. Windows had thousands of software engineers, program managers, testers and managers and they worked on a lot of different things.

fifteen1506 3y ago
I preferred Vista interface to Windows 7. Vista SP1 fixed most problems -- but you needed 2GB RAM.
cbozeman 3y ago

neonsunset 3y ago

C# is much more capable and fitting this purpose today than 10 years ago (perf, codegen quality and infrastructure have improved by orders of magnitude). However, it cannot compete with Rust when it comes to systems programming. Mainly because it has different tradeoffs and its high level abstractions and other features like LINQ, interfaces, non-struct generics and async/await are very much not zero-cost, unlike in Rust. In addition, it does not offer crucial features one would want in kernel development: deterministic memory usage, compile-time safety guarantees for writing concurrent data structures and general systems-programming-first language design. While C# is a perfectly viable choice for writing userspace OS components, applications, UI and etc., it is a bad choice for kernelspace when C++ or Rust exist.

fulafel 3y ago

Rust is gaining in mindshare and C++ background curmudgeons still believe OS code can't be in GC supporting languages. But maybe it's better not to look a gift horse in the mouth, replacing unsafe code with something better still works toward slowly establishing mem safety as table stakes in OS code.