13 ms·

Writing Network Drivers in Rust [pdf]

364 points by MrBuddyCasino 8y ago

hardwaresofton 8y ago
Related (Writing network drivers in Go): https://news.ycombinator.com/item?id=18399389 https://news.ycombinator.com/item?id=18399389 I'm excited to read both of these.
- aquadrop 8y ago
  Btw, they also have C# in the same series: https://www.net.in.tum.de/fileadmin/bibtex/publications/theses/2018-ixy-c-sharp.pdf https://www.net.in.tum.de/fileadmin/bibtex/publications/thes...
- arendtio 8y ago
  Same university and department :D
  - hardwaresofton 8y ago
    Yeah I noticed! Seeing that this is an undergrad thesis makes me feel like I did nothing in undergrad. I'm impressed
    - blub 8y ago
      Maybe I missed something, but all of these network drivers in Go, Rust, C# etc are just re-implementations of the original C driver, where it would seem to me that the real work happened. That would make the ports still be an interesting experiment, but not as impressive IMO.
      - tracker1 8y ago
        I think it depends. Writing a non-trivial language port is an interesting way to show off a language. Though it also tends to be an ugly way to show off a language. It depends on the ugliness of implementation for a given language. Early in C# I saw some really ugly implementations of crypto from other languages, for example. Often pretty obvious the person porting didn't understand the new language well. Some things hard in one language are easy in another, and it usually helps to have a good understanding of both. Just my own POV. However, low level code often requires one to do things "the hard way" for performance reasons. I particularly appreciate what Go, Rust, C# and others bring to the table. Most of the code I write isn't particularly performance intensive, so I tend to reach for JS in early implementations. Though happy to rewrite if/when performance becomes a concern. I really enjoy articles like this.
jononor 8y ago
The Rust version of ixy seems to have as many or a bit more lines of code than the C version? That is a bit suprising to me. 1306 for Rust, and '1000' for C. The Rust additionally uses 4 dependencies from crates.io Unsafe usage is quite small, at 10%, which seems good for a hardware device driver.
- carlmr 8y ago
  Does Rust claim to use fewer lines?
  - swiftcoder 8y ago
    No. Although that's mostly because the standard rustfmt is very generous about using vertical space.
  - Filligree 8y ago
    For high-level code it usually does. Idiomatic Rust can look surprisingly similar to Python. That doesn't apply to driver development, but it might be surprising to someone only moderately familiar with the language.
- klohto 8y ago
  I’m not sure if this is a critique or a question. If former, then I don’t understand why would that matter. Even at 4 times the length of the code, Rust is safer, more robust and better optimized. So why would the length matter?
  - Riverheart 8y ago
    One of Rust's selling points is zero cost abstractions which would normally imply doing more work with less code (as you would expect in a high level language like Python). So it is surprising that it takes that many more lines of code to achieve what C does without those abstractions.
    - rantanplan 8y ago
delta1 8y ago
FTA "Based on our findings we can conclude that Rust is a very well-suited programming language for writing network drivers. It is not only a fast and safe systems programming language but was also voted most beloved programming language in 2016, 2017 and 2018 [15, 16, 17]. Writing more drivers in Rust would certainly lead to safer and more reliable computer systems." Working link to their code: https://github.com/ixy-languages/ixy.rs https://github.com/ixy-languages/ixy.rs
- Xophmeister 8y ago
  How is “most beloved” a prerequisite to writing network drivers? Don’t get me wrong, if Rust is a good technical fit, then that’s great, but so convince me on those objective measures.
  - specializeded 8y ago
    As a human who’ll be writing X, I sometimes enjoy the perspective of other humans who write X.
  - nothrabannosir 8y ago
    Popularity influences ecosystem which influences programs written in the language. A language is more than just its spec. Larger mind share improves availability and quality of 3rd party libs, features, performance and quality of the compiler, brain power to guide the language forward, etc. It's the second order effects that matter.
  - simias 8y ago
    "Most beloved" is a weird way to put it but if it's a way to gauge the state of the language's ecosystem I'd say it's a very practical concern. For instance these past few days I've been writing a small bootloader for some embedded ARM chip. I pondered going with Rust but ended up using C for two main reasons: - I know that if it's doable in Rust it's doable in C but the opposite is not always true (or straightforward). In particular bare-metal/very low level Rust is still somewhat experimental, you can't use stable Rust if you want to be able to use inline ASM or other advanced features. I have very tight constraints regarding code size and memory use and I was concerned that I might not control that as effectively with Rust code than with C. - There's already a bunch of C code written for this chip that I can reuse without any friction (no binding generation, no ffi etc...). The vendor even provides a huge header file with the full memory map and some code to init the PLLs. If you're coding in a vacuum then how "beloved" a language is is mostly irrelevant but if you want to benefit from the ecosystem then having a language that people want to use and support is extremely important. The tooling will mature and stabilize as more and more people use Rust to write bare-metal code and device drivers, which will eventually make my first point above irrelevant. Then if the language becomes popular enough it will be easier to find Rust code or pre-made crates to do what you want which will hopefully eventually solve my 2nd issue.
dswalter 8y ago
Per a talk from Joshua Liebow-Feeser at the Rust Belt Rust 2018 conference, Google is heavily using Rust for networking in their new Fuchsia operating system. This is a rough approximation of a quote he gave during the talk: "What makes Rust different is not that you can write high-performance, bare-metal code. People use C and C++ to do that all the time. What makes Rust different is that when you write that code, it is safe, clean, and easy to use, and you are confident in its correctness." If even Google is using Rust for performance-critical development, that seems pretty promising to me.
- daveoflynn 8y ago
  Here are Joshua’s slides from the talk mentioned above: https://joshlf.com/files/talks/Move%20Fast%20and%20Don't%20Break%20Things.pdf https://joshlf.com/files/talks/Move%20Fast%20and%20Don't%20B...
- Numberwang 8y ago
  Is this on YouTube? I can't find the talk.
  - shepmaster 8y ago
    Not yet. I'm slowly putting them together and really hope to have them uploaded by the end of the year. You can follow our channel[1], which should (?) notify you when this year's videos are published. [1]: https://www.youtube.com/channel/UCptxtVyJkQAJZcFwBbIDZcg https://www.youtube.com/channel/UCptxtVyJkQAJZcFwBbIDZcg
- mav3rick 8y ago
  The Chrome OS team is writing Rust for it's Crostini project as well.
- erwan 8y ago
  Fuschia seem more like an experiment in object-capabilities design rather than "performance-critical development" to me.
Jhsto 8y ago
I recently wrote my bachelors thesis and I wish it could have been as practical as this. Props for German universities for allowing a software engineering project to be accepted. There is no way I could have gotten away with so little amount of academic references.
- ambentzen 8y ago
  When I took my bachelors, each semester we split the 27-ish people in our year into groups of about 6 people that had to write a piece of software related to a theme, e.g. compilers or robots, where the courses that semester supported the theme. At the end of the semester we handed in a report of approximately 100 pages detailing the decisions and design and theory behind the practical work we did, which we had to defend orally. The bachelor project was more of the same, but each group should do a part of a bigger piece of software. I don't recall having any academic references in our final report. It was a very nice way to learn, as you had your group members to study with and learn from each other. This was in Denmark, Aalborg University. Which have just been named the best engineering university in Europe, and the fourth best in the world [0]. [0] https://www.usnews.com/education/best-global-universities/engineering https://www.usnews.com/education/best-global-universities/en...
  - leesec 8y ago
    That sounds so awesome compared to the style of learning I experienced in college. I wish more colleges took a practical approach.
  - Fredej 8y ago
    Hah, I was reading this thinking "That sounds so much like Aalborg". I'm so incredibly happy I went there :) Here's another report from MIT from earlier this year, reporting largely the same, placing Aalborg at 3rd or 4th depending on the method used [0]. [0] http://neet.mit.edu/wp-content/uploads/2018/03/MIT_NEET_GlobalStateEngineeringEducation2018.pdf http://neet.mit.edu/wp-content/uploads/2018/03/MIT_NEET_Glob...
- klmr 8y ago
  Conversely, I got a point deduction on my master thesis for including too many (almost exclusively academic) references. Mental. (They were all relevant.)
bvelica 8y ago
I was hopping to see something like that. Thank you!
_pmf_ 8y ago
What really would be nice is an extension of what Zinc does, namely encode the IO registers as strongly types entities. Maybe not even encode their usage, but also their usage (r/w/rw).
godelmachine 8y ago
Contrast this with https://news.ycombinator.com/item?id=18399389 https://news.ycombinator.com/item?id=18399389, both of which are trending.
noodlesUK 8y ago
Does anyone happen to have a copy of the LaTeX template that they are using? I really like the typography of the two that are currently on the frontpage.
- todd8 8y ago
  I like the formatting too. It looks like Computer Modern type face with a nice clean layout. This is the template I found at Tum University’s Department of Informatica for a undergrad thesis: https://www.overleaf.com/latex/templates/tum-thesis-for-informatics-template/ttjnzgmgvvxj https://www.overleaf.com/latex/templates/tum-thesis-for-info...
- tlamponi 8y ago
  It may be this: https://github.com/fwalch/tum-thesis-latex https://github.com/fwalch/tum-thesis-latex At least it's linked here: https://www.in.tum.de/fuer-studierende/bachelor-studiengaenge/informatik-games-engineering/abschlussarbeit/ https://www.in.tum.de/fuer-studierende/bachelor-studiengaeng...
cesarb 8y ago
Having written device drivers before, the first thing I wondered was how he allocated a fixed region of physical memory in user space... "In kernel space there is an API to allocate DMA memory, in user space we have to use other mechanisms since this API is not available here. To disable swapping, we can use mlock(2). [...] However, mlock(2) only ensures that the page is kept in memory. The kernel is still able to move pages to different physical addresses. To circumvent this problem, we use huge pages with a size of 2 MiB. These pages cannot be migrated yet by the Linux kernel, thus stay resident in physical memory." He didn't, he's depending on an implementation quirk which might change in the future (and also depending on the physical memory backing huge pages being in a region which can be reached by the network device through DMA, though that's probably always true on x86 when memory encryption is not in use).
- sk_hazratali 8y ago
  https://www.sherekhudahazratali.com/2018/11/noor-wala-aya-hai-lyrics.html https://www.sherekhudahazratali.com/2018/11/noor-wala-aya-ha...
- codingbeer 8y ago
  DPDK also relies on this quirk if there is no IOMMU available. He does mention the IOMMU as future work, but in the context of security.
- deepsun 8y ago
  Well, Linus is obsessed with "don't break userspace", so as long as there is N > 0 users who use that implementation detail, it will stay forever and become documented limitation of Linux kernel :(
xyproto 8y ago
Reading the conclusion in the PDF, the Rust version is not faster than the C version and not necessarily safer either, since there is plenty of "unsafe" Rust code involved. While Rust is nice in some areas, I'm not convinced this is anything more than yet another "let's rewrite it in Rust, because I happen to like Rust" article. Angry comments defending Rust incoming in 3..2..
- dswalter 8y ago
  Since it's part of what looks like an overall project (ixy) to show how it's possible to do this sort of low-level programming in a variety of languages (the roadmap includes js and python), the developer experience is an important component of what they're trying to shed light on.
- kungtotte 8y ago
  How is less than 10% "plenty"? Especially considering that 100% of C-code involving memory access is inherently unsafe using the Rust definition of the term.
- red75prime 8y ago
  > not necessarily safer either Rust version will not be safer if the authors did 10 times more errors per line than average C programmer.
  - red75prime 8y ago
    Rust version has 10% of lines marked as `unsafe`. Programmers need to make 10 times more errors per line to introduce the same number of errors that are considered unsafe in Rust (memory corruption, data races, double frees, deallocated memory usage). I'm not sure what's wrong with this statement.
lukego 8y ago
Since we are looking at PCI Ethernet drivers written in languages other than C (fun!) here is one written in Forth too: https://github.com/openbios/openfirmware/blob/master/dev/ne2000/ne2kpci.fth https://github.com/openbios/openfirmware/blob/master/dev/ne2... Good to remember that C and Unix kernels haven't always had a monopoly on device drivers.
discoball 8y ago
Nim produces smaller binaries, has zero-pause, realtime gc (and you can even define gc behavior) and compiles to optimized C. It's a win over Rust in every way including ease of use. Big issue with Nim is tribalism, pure and simple. It's author failed to get any major corporations behind it so it doesn't get hyped up in constant Medium articles and conference talks and therefore does not get a lot of attention, which keeps it a niche language, sadly.
- the_duke 8y ago
  Nim is a lovely language but I disagree with most of your points. * The Rust borrow checker prevents a lot of issues in C(++) code. Including thread safety. Nim does not provide anything new in this regard. * The lack of a GC is a feature, not a detriment. One of the reasons why Rust applications are often very fast is because the language naturally guides you to avoid allocations when possible and allows low level control where necessary. Combined with the safety guarantees of the borrow checker, this is a powerful combination. This is possible in Nim but using GC is easier and will be the preferred solution when in doubt. * While Mozilla support does help a lot for sure, a lot of the hype around Rust (imo) comes from people trying it out, just falling in love with and then promoting it
  - discoball 8y ago
    << While Mozilla support does help a lot for sure, a lot of the hype around Rust (imo) comes from people trying it out, just falling in love with and then promoting it >> If there was no rampant tribalism, my comment would not be buried with negative votes. It would receive a response like the one you gave, which is subjective, and that would be the end of it. Some may even up-vote it because diversity is always good. But feels like Rust users are zealous and it's a bad sign, tbh. Wouldn't want to be part of a community that acted with hostility towards others for presenting alternative view points.
    - the_duke 8y ago
      You were downvoted because your claims were not substantiated by factual analysis. Rather, your comment seemed rather "tribal" while criticizing Rust fans of the same.
ilovecaching 8y ago
This is very cool. I’ve been working on using Rust to manage an XDP based network driver that is much, much, much faster than the socket API in C, and slightly faster than DPDK. There’s no driver in the traditional sense, Rust works as a control plane and eBPF injector, while the XDP programs communicate back to Rust through eBPF maps. I’m extremely bullish on Rust, it’s such a breath of fresh air to be able to write maintainable, modern code to do low level tasks. I was a C/C++ dev for a little under a decade before switching to Rust and it really delivers on its promises. Also Rust using llvm is a huge boon to writing eBPF programs.
- Klasiaster 8y ago
  This sounds very interesting, do you have some code (parts) to share?
  - jvtbatman 8y ago
    I'm also very interested in any more information regarding this since I'll be doing my thesis work on XDP soon and would like to use Rust!
OpenBSD-reich 8y ago
Why rust? Didn't Ada and SPARK already pretty much solve the problems rust is tackling? rust feels like the systemd of languages: came out of nowhere and very heavily evangelized.
- tathougies 8y ago
  Ada doesn't have the memory guarantees of rust. Ada's type system is closer to that of C. Rust's type system is based on ML, which is based on a proven sound type theory. Ada has had a lot of work put into it, but the trouble is that the foundation was never verified to begin with.
- imtringued 8y ago
  Do those languages have anything even remotely similar to linear types [0]? If not then they aren't even in the same league as Rust. [0] https://en.wikipedia.org/wiki/Substructural_type_system https://en.wikipedia.org/wiki/Substructural_type_system These types of comments really irritate me. New thing X comes out that solves an unsolved problem. Then someone comments "why not obsolete thing Y that doesn't solve the problem sufficiently"? I constantly see comparisons like JVM vs WASM [1], or in this case Ada vs Rust. And very often the newer thing is superior and is not popular just because of cargo culting. [1] i.e. To obtain similar performance while still using java bytecode you will have to stop using the Java object model completely and exclusively use sun.misc.Unsafe and calculate field offsets of structs manually and still have the risk of vulnerabilities like buffer overflows similar to writing programs in C.
deleted 8y ago
[deleted]

364 points by MrBuddyCasino 8y ago

Related (Writing network drivers in Go): https://news.ycombinator.com/item?id=18399389 https://news.ycombinator.com/item?id=18399389 I'm excited to read both of these.

aquadrop 8y ago
Btw, they also have C# in the same series: https://www.net.in.tum.de/fileadmin/bibtex/publications/theses/2018-ixy-c-sharp.pdf https://www.net.in.tum.de/fileadmin/bibtex/publications/thes...
arendtio 8y ago
Same university and department :D
- hardwaresofton 8y ago
  Yeah I noticed! Seeing that this is an undergrad thesis makes me feel like I did nothing in undergrad. I'm impressed
  - blub 8y ago
    Maybe I missed something, but all of these network drivers in Go, Rust, C# etc are just re-implementations of the original C driver, where it would seem to me that the real work happened. That would make the ports still be an interesting experiment, but not as impressive IMO.
    - tracker1 8y ago
      I think it depends. Writing a non-trivial language port is an interesting way to show off a language. Though it also tends to be an ugly way to show off a language. It depends on the ugliness of implementation for a given language. Early in C# I saw some really ugly implementations of crypto from other languages, for example. Often pretty obvious the person porting didn't understand the new language well. Some things hard in one language are easy in another, and it usually helps to have a good understanding of both. Just my own POV. However, low level code often requires one to do things "the hard way" for performance reasons. I particularly appreciate what Go, Rust, C# and others bring to the table. Most of the code I write isn't particularly performance intensive, so I tend to reach for JS in early implementations. Though happy to rewrite if/when performance becomes a concern. I really enjoy articles like this.

jononor 8y ago

The Rust version of ixy seems to have as many or a bit more lines of code than the C version? That is a bit suprising to me. 1306 for Rust, and '1000' for C. The Rust additionally uses 4 dependencies from crates.io Unsafe usage is quite small, at 10%, which seems good for a hardware device driver.

carlmr 8y ago
Does Rust claim to use fewer lines?
- swiftcoder 8y ago
  No. Although that's mostly because the standard rustfmt is very generous about using vertical space.
- Filligree 8y ago
  For high-level code it usually does. Idiomatic Rust can look surprisingly similar to Python. That doesn't apply to driver development, but it might be surprising to someone only moderately familiar with the language.
klohto 8y ago
I’m not sure if this is a critique or a question. If former, then I don’t understand why would that matter. Even at 4 times the length of the code, Rust is safer, more robust and better optimized. So why would the length matter?
- Riverheart 8y ago
  One of Rust's selling points is zero cost abstractions which would normally imply doing more work with less code (as you would expect in a high level language like Python). So it is surprising that it takes that many more lines of code to achieve what C does without those abstractions.
  - rantanplan 8y ago

delta1 8y ago

FTA "Based on our findings we can conclude that Rust is a very well-suited programming language for writing network drivers. It is not only a fast and safe systems programming language but was also voted most beloved programming language in 2016, 2017 and 2018 [15, 16, 17]. Writing more drivers in Rust would certainly lead to safer and more reliable computer systems." Working link to their code: https://github.com/ixy-languages/ixy.rs https://github.com/ixy-languages/ixy.rs

Xophmeister 8y ago
How is “most beloved” a prerequisite to writing network drivers? Don’t get me wrong, if Rust is a good technical fit, then that’s great, but so convince me on those objective measures.
- specializeded 8y ago
  As a human who’ll be writing X, I sometimes enjoy the perspective of other humans who write X.
- nothrabannosir 8y ago
  Popularity influences ecosystem which influences programs written in the language. A language is more than just its spec. Larger mind share improves availability and quality of 3rd party libs, features, performance and quality of the compiler, brain power to guide the language forward, etc. It's the second order effects that matter.
- simias 8y ago
  "Most beloved" is a weird way to put it but if it's a way to gauge the state of the language's ecosystem I'd say it's a very practical concern. For instance these past few days I've been writing a small bootloader for some embedded ARM chip. I pondered going with Rust but ended up using C for two main reasons: - I know that if it's doable in Rust it's doable in C but the opposite is not always true (or straightforward). In particular bare-metal/very low level Rust is still somewhat experimental, you can't use stable Rust if you want to be able to use inline ASM or other advanced features. I have very tight constraints regarding code size and memory use and I was concerned that I might not control that as effectively with Rust code than with C. - There's already a bunch of C code written for this chip that I can reuse without any friction (no binding generation, no ffi etc...). The vendor even provides a huge header file with the full memory map and some code to init the PLLs. If you're coding in a vacuum then how "beloved" a language is is mostly irrelevant but if you want to benefit from the ecosystem then having a language that people want to use and support is extremely important. The tooling will mature and stabilize as more and more people use Rust to write bare-metal code and device drivers, which will eventually make my first point above irrelevant. Then if the language becomes popular enough it will be easier to find Rust code or pre-made crates to do what you want which will hopefully eventually solve my 2nd issue.

dswalter 8y ago

Per a talk from Joshua Liebow-Feeser at the Rust Belt Rust 2018 conference, Google is heavily using Rust for networking in their new Fuchsia operating system. This is a rough approximation of a quote he gave during the talk: "What makes Rust different is not that you can write high-performance, bare-metal code. People use C and C++ to do that all the time. What makes Rust different is that when you write that code, it is safe, clean, and easy to use, and you are confident in its correctness." If even Google is using Rust for performance-critical development, that seems pretty promising to me.

daveoflynn 8y ago
Here are Joshua’s slides from the talk mentioned above: https://joshlf.com/files/talks/Move%20Fast%20and%20Don't%20Break%20Things.pdf https://joshlf.com/files/talks/Move%20Fast%20and%20Don't%20B...
Numberwang 8y ago
Is this on YouTube? I can't find the talk.
- shepmaster 8y ago
  Not yet. I'm slowly putting them together and really hope to have them uploaded by the end of the year. You can follow our channel[1], which should (?) notify you when this year's videos are published. [1]: https://www.youtube.com/channel/UCptxtVyJkQAJZcFwBbIDZcg https://www.youtube.com/channel/UCptxtVyJkQAJZcFwBbIDZcg
mav3rick 8y ago
The Chrome OS team is writing Rust for it's Crostini project as well.
erwan 8y ago
Fuschia seem more like an experiment in object-capabilities design rather than "performance-critical development" to me.

Jhsto 8y ago

I recently wrote my bachelors thesis and I wish it could have been as practical as this. Props for German universities for allowing a software engineering project to be accepted. There is no way I could have gotten away with so little amount of academic references.

ambentzen 8y ago
When I took my bachelors, each semester we split the 27-ish people in our year into groups of about 6 people that had to write a piece of software related to a theme, e.g. compilers or robots, where the courses that semester supported the theme. At the end of the semester we handed in a report of approximately 100 pages detailing the decisions and design and theory behind the practical work we did, which we had to defend orally. The bachelor project was more of the same, but each group should do a part of a bigger piece of software. I don't recall having any academic references in our final report. It was a very nice way to learn, as you had your group members to study with and learn from each other. This was in Denmark, Aalborg University. Which have just been named the best engineering university in Europe, and the fourth best in the world [0]. [0] https://www.usnews.com/education/best-global-universities/engineering https://www.usnews.com/education/best-global-universities/en...
- leesec 8y ago
  That sounds so awesome compared to the style of learning I experienced in college. I wish more colleges took a practical approach.
- Fredej 8y ago
  Hah, I was reading this thinking "That sounds so much like Aalborg". I'm so incredibly happy I went there :) Here's another report from MIT from earlier this year, reporting largely the same, placing Aalborg at 3rd or 4th depending on the method used [0]. [0] http://neet.mit.edu/wp-content/uploads/2018/03/MIT_NEET_GlobalStateEngineeringEducation2018.pdf http://neet.mit.edu/wp-content/uploads/2018/03/MIT_NEET_Glob...
klmr 8y ago
Conversely, I got a point deduction on my master thesis for including too many (almost exclusively academic) references. Mental. (They were all relevant.)

bvelica 8y ago

I was hopping to see something like that. Thank you!

_pmf_ 8y ago

What really would be nice is an extension of what Zinc does, namely encode the IO registers as strongly types entities. Maybe not even encode their usage, but also their usage (r/w/rw).

godelmachine 8y ago

Contrast this with https://news.ycombinator.com/item?id=18399389 https://news.ycombinator.com/item?id=18399389, both of which are trending.

noodlesUK 8y ago

Does anyone happen to have a copy of the LaTeX template that they are using? I really like the typography of the two that are currently on the frontpage.

todd8 8y ago
I like the formatting too. It looks like Computer Modern type face with a nice clean layout. This is the template I found at Tum University’s Department of Informatica for a undergrad thesis: https://www.overleaf.com/latex/templates/tum-thesis-for-informatics-template/ttjnzgmgvvxj https://www.overleaf.com/latex/templates/tum-thesis-for-info...
tlamponi 8y ago
It may be this: https://github.com/fwalch/tum-thesis-latex https://github.com/fwalch/tum-thesis-latex At least it's linked here: https://www.in.tum.de/fuer-studierende/bachelor-studiengaenge/informatik-games-engineering/abschlussarbeit/ https://www.in.tum.de/fuer-studierende/bachelor-studiengaeng...

cesarb 8y ago

Having written device drivers before, the first thing I wondered was how he allocated a fixed region of physical memory in user space... "In kernel space there is an API to allocate DMA memory, in user space we have to use other mechanisms since this API is not available here. To disable swapping, we can use mlock(2). [...] However, mlock(2) only ensures that the page is kept in memory. The kernel is still able to move pages to different physical addresses. To circumvent this problem, we use huge pages with a size of 2 MiB. These pages cannot be migrated yet by the Linux kernel, thus stay resident in physical memory." He didn't, he's depending on an implementation quirk which might change in the future (and also depending on the physical memory backing huge pages being in a region which can be reached by the network device through DMA, though that's probably always true on x86 when memory encryption is not in use).

sk_hazratali 8y ago
https://www.sherekhudahazratali.com/2018/11/noor-wala-aya-hai-lyrics.html https://www.sherekhudahazratali.com/2018/11/noor-wala-aya-ha...
codingbeer 8y ago
DPDK also relies on this quirk if there is no IOMMU available. He does mention the IOMMU as future work, but in the context of security.
deepsun 8y ago
Well, Linus is obsessed with "don't break userspace", so as long as there is N > 0 users who use that implementation detail, it will stay forever and become documented limitation of Linux kernel :(

xyproto 8y ago

Reading the conclusion in the PDF, the Rust version is not faster than the C version and not necessarily safer either, since there is plenty of "unsafe" Rust code involved. While Rust is nice in some areas, I'm not convinced this is anything more than yet another "let's rewrite it in Rust, because I happen to like Rust" article. Angry comments defending Rust incoming in 3..2..

dswalter 8y ago
Since it's part of what looks like an overall project (ixy) to show how it's possible to do this sort of low-level programming in a variety of languages (the roadmap includes js and python), the developer experience is an important component of what they're trying to shed light on.
kungtotte 8y ago
How is less than 10% "plenty"? Especially considering that 100% of C-code involving memory access is inherently unsafe using the Rust definition of the term.
red75prime 8y ago
> not necessarily safer either Rust version will not be safer if the authors did 10 times more errors per line than average C programmer.
- red75prime 8y ago
  Rust version has 10% of lines marked as `unsafe`. Programmers need to make 10 times more errors per line to introduce the same number of errors that are considered unsafe in Rust (memory corruption, data races, double frees, deallocated memory usage). I'm not sure what's wrong with this statement.

lukego 8y ago

Since we are looking at PCI Ethernet drivers written in languages other than C (fun!) here is one written in Forth too: https://github.com/openbios/openfirmware/blob/master/dev/ne2000/ne2kpci.fth https://github.com/openbios/openfirmware/blob/master/dev/ne2... Good to remember that C and Unix kernels haven't always had a monopoly on device drivers.

discoball 8y ago

Nim produces smaller binaries, has zero-pause, realtime gc (and you can even define gc behavior) and compiles to optimized C. It's a win over Rust in every way including ease of use. Big issue with Nim is tribalism, pure and simple. It's author failed to get any major corporations behind it so it doesn't get hyped up in constant Medium articles and conference talks and therefore does not get a lot of attention, which keeps it a niche language, sadly.

the_duke 8y ago
Nim is a lovely language but I disagree with most of your points. * The Rust borrow checker prevents a lot of issues in C(++) code. Including thread safety. Nim does not provide anything new in this regard. * The lack of a GC is a feature, not a detriment. One of the reasons why Rust applications are often very fast is because the language naturally guides you to avoid allocations when possible and allows low level control where necessary. Combined with the safety guarantees of the borrow checker, this is a powerful combination. This is possible in Nim but using GC is easier and will be the preferred solution when in doubt. * While Mozilla support does help a lot for sure, a lot of the hype around Rust (imo) comes from people trying it out, just falling in love with and then promoting it
- discoball 8y ago
  << While Mozilla support does help a lot for sure, a lot of the hype around Rust (imo) comes from people trying it out, just falling in love with and then promoting it >> If there was no rampant tribalism, my comment would not be buried with negative votes. It would receive a response like the one you gave, which is subjective, and that would be the end of it. Some may even up-vote it because diversity is always good. But feels like Rust users are zealous and it's a bad sign, tbh. Wouldn't want to be part of a community that acted with hostility towards others for presenting alternative view points.
  - the_duke 8y ago
    You were downvoted because your claims were not substantiated by factual analysis. Rather, your comment seemed rather "tribal" while criticizing Rust fans of the same.

ilovecaching 8y ago

This is very cool. I’ve been working on using Rust to manage an XDP based network driver that is much, much, much faster than the socket API in C, and slightly faster than DPDK. There’s no driver in the traditional sense, Rust works as a control plane and eBPF injector, while the XDP programs communicate back to Rust through eBPF maps. I’m extremely bullish on Rust, it’s such a breath of fresh air to be able to write maintainable, modern code to do low level tasks. I was a C/C++ dev for a little under a decade before switching to Rust and it really delivers on its promises. Also Rust using llvm is a huge boon to writing eBPF programs.

Klasiaster 8y ago
This sounds very interesting, do you have some code (parts) to share?
- jvtbatman 8y ago
  I'm also very interested in any more information regarding this since I'll be doing my thesis work on XDP soon and would like to use Rust!

OpenBSD-reich 8y ago

Why rust? Didn't Ada and SPARK already pretty much solve the problems rust is tackling? rust feels like the systemd of languages: came out of nowhere and very heavily evangelized.

tathougies 8y ago
Ada doesn't have the memory guarantees of rust. Ada's type system is closer to that of C. Rust's type system is based on ML, which is based on a proven sound type theory. Ada has had a lot of work put into it, but the trouble is that the foundation was never verified to begin with.
imtringued 8y ago
Do those languages have anything even remotely similar to linear types [0]? If not then they aren't even in the same league as Rust. [0] https://en.wikipedia.org/wiki/Substructural_type_system https://en.wikipedia.org/wiki/Substructural_type_system These types of comments really irritate me. New thing X comes out that solves an unsolved problem. Then someone comments "why not obsolete thing Y that doesn't solve the problem sufficiently"? I constantly see comparisons like JVM vs WASM [1], or in this case Ada vs Rust. And very often the newer thing is superior and is not popular just because of cargo culting. [1] i.e. To obtain similar performance while still using java bytecode you will have to stop using the Java object model completely and exclusively use sun.misc.Unsafe and calculate field offsets of structs manually and still have the risk of vulnerabilities like buffer overflows similar to writing programs in C.

deleted 8y ago

[deleted]