7 ms·

Rewriting m4vgalib in Rust

372 points by zimmerfrei 7y ago

brohee 7y ago
https://www.youtube.com/watch?v=HkUUJ-rOUPg https://www.youtube.com/watch?v=HkUUJ-rOUPg has an output of the demo, and this is indeed very impressive for hardware that isn't supposed to output video at all...
- makapuf 7y ago
  May I also chime in and present my old bitbox console, a VGA on the same chips, with a dozen games and emulators for it ? http://bitboxconsole.blogspot.com/?m=1 http://bitboxconsole.blogspot.com/?m=1
  - StavrosK 7y ago
    You need a lot more photos, in my opinion. I wanted to look at some to help me engage with the project and clicked around to the blog, then Github, then the wiki, then Hardware but left when I didn't see any photos anywhere.
    - makapuf 7y ago
      Thanks a lot for the feedback !
- vijaybritto 7y ago
  They are unbelievably performant! I think he needs to join the Android auto team and make their car software run smooth like this!
  - pjmlp 7y ago
    Even if Android was pure C++ instead of Java, given how those Frameworks got designed, I doubt it would help much.
  - sswezey 7y ago
    He's actually already part of the Fuchsia team, so he very well could be working on the successor to Android.
    - vijaybritto 7y ago
      Thats great news! I really hope they improve the android auto responsiveness
- ncmncm 7y ago
  That was seven years ago. I liked this better, from four years ago: https://m.youtube.com/watch?v=7yXxhvKmVb0 https://m.youtube.com/watch?v=7yXxhvKmVb0
dry_soup 7y ago
This was more interesting than the average "rewrite in Rust" blog post. Specifically I thought the part about how having compiler-enforced memory safety allows for the use of patterns that would otherwise be difficult to use safely interesting.
- epage 7y ago
  > compiler-enforced memory safety allows for the use of patterns that would otherwise be difficult to use safely I have to say, this has been amazing. For context, I'm a C++ developer who has had minimal run-ins with the borrow checker because I was already used to thinking about ownership. I refactored a large-ish Rust crate I maintain to use references for a fundamental type. I would have considered it irresponsible to do that with a C++ codebase unless absolutely required. Even if I did the upfront analysis to make sure it was safe, I'd need to re-do that exercise for every architectural change, at least. Instead with Rust, I just tried it on a whim, knowing the borrow checker had my back.
  - pjmlp 7y ago
    In C++ you can have some help via static analysis. Unfortunately very few care to actually use them. https://www.bfilipek.com/2019/12/cpp-status-2019.html?m=1#what-additional-tools-do-you-use https://www.bfilipek.com/2019/12/cpp-status-2019.html?m=1#wh... As I learned since Turbo Pascal days, if it isn't part of the language, it seldom gets used.
    - roca 7y ago
      That's quite right. But also there is a big difference between "the static checker will probably catch errors of this kind" and "the compiler will certainly catch errors of this kind". The latter is far more liberating.
- Avshalom 7y ago
  worth pointing out, i think, that that was the original justification for rust: that by building a safe language that still gave you the level of control you'd expect from a C(++) they could experiment with all sorts of otherwise difficult and subtle concurrent code to figure out what would actually work/help. Much like Erlang's concurrency fell out as a consequence of needing to be confidently write high availability code, Rust's general safety falls out of needing to write concurrent code fearlessly.
PudgePacket 7y ago
Author also wrote "Learn Rust the dangerous way". > In this series, I'm trying something different. Let's take a grungy, optimized, pointer-type-punning, SSE-using, heap-eschewing, warning-disabling C program and look at how we could create the same program in Rust. http://cliffle.com/p/dangerust/0/ http://cliffle.com/p/dangerust/0/
- rvr_ 7y ago
  I dislike nitpicking, but the following is wrong: "In C, bodies[i] is exactly the same as *(bodies + i) — it performs pointer arithmetic and a dereference and nothing else. In particular, it assumes that you have some reason to know that i is a valid index for the array. This is the common case in C and gets a shorthand using square brackets." In C, bodies[i] is not (bodies + i), with the exception being char arrays. The general rule is: bodies[i] = *(bodies + i * sizeof(bodies[0]))
  - heinrich5991 7y ago
    No, it's actually defined to be the same. The pointer arithmetic in C follows your intuitive understanding of array indexing. #include <stdio.h> int main() { int array[] = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16}; printf("%d %d\n", array[4], *(array + 4)); return 0; } prints "4 4".
  - barrkel 7y ago
    + in C advances pointers in sizeof-pointed-to thing increments. It's why ++ works too. Similarly - on pointers returns a difference in units of sizeof-pointed-to thing.
  - jcranmer 7y ago
    The semantics of a[i] is exactly that of *(a + i). Which is why i[bodies] is actually legal C code.
deleted 7y ago
[deleted]
0xdead 7y ago
It is hard to take someone seriously when they write "C/C++" in their articles like C and C++ are the same thing.
blargmaster33 7y ago
Why did rust not go with standard c syntax
- steveklabnik 7y ago
  There are features Rust has that aren't in C, and so have no precedent. So it's not really possible. We did take a lot of influence from C, though.
- pornel 7y ago
  Rust did copy C syntax to a large degree. It didn't copy some of the mistakes such as type declarations, which are hard to parse (typedef makes AST depend on itself) and hard to understand ("array of function pointers" is a head-scratcher in C, easy in Rust).
  - masklinn 7y ago
    Aside from the feedback loop, C-style decls also make the language less regular in the presence of type inference as you need an "inference pseudo-type" to take the place of the type you're omitting. Also prefix casts have similar issues as prefix type + typedef.
- therein 7y ago
  I appreciate the ability to avoid having to enclose my conditions in parenthesis, and the main operand of statements like match, switch etc. to not be wrapped in parenthesis.
- heavenlyblue 7y ago
  I am sorry, but if you think language syntax “beauty” is something to be considered while picking the language to develop in, then you’re probably bot senior enough to make decisions which one of them to use for the project. There’s only one true answer to this: “it doesn’t matter at all - I’ll learn it in a week@.
  - orf 7y ago
bluejekyll 7y ago
If the author is reading this, this might be the greatest single article I’ve read that expresses the benefits of Rust and how minimal the trade-offs are. Wonderfully written piece, thank you! BTW, in your bounds checking examples, I think it can be further simplified (would need to check the output to see if it's any better), this: fn fill_1024(array: &mut [u8], color: u8) { // Perform an explicit, checked, slice of the array before // entering the loop. let array = &mut array[..1024]; for i in 0..1024 { array[i] = color; } } Could be: fn fill_1024(array: &mut [u8], color: u8) { for i in &mut array[..1024] { *i = color; } } Here's the playground: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=832233ec0b3319fcec86c2330be93523 https://play.rust-lang.org/?version=stable&mode=debug&editio... Thanks again, this is a wonderful piece.
- masklinn 7y ago
  I'm quite terrible at assembly, but it looks like both got inlined and compiled to a memset: leaq 8(%rsp), %rdi movl $1024, %edx movl $255, %esi callq *memset@GOTPCREL(%rip) for the 0xFF fill and leaq 8(%rsp), %rdi movl $1024, %edx movl $238, %esi callq *memset@GOTPCREL(%rip) for the 0xEE fill. However it turns out the original also compiles to that, because the inlining means the fixed size array is visible to the optimiser.
  - jchw 7y ago
    With lower opt-level, it does not get optimized to memset, and the succinct version actually compiles better, eliding the bounds checking in the loop as well as, for some reason, needing a smaller stack. https://godbolt.org/z/Wvuen5 https://godbolt.org/z/Wvuen5 This leads me to believe the shorter approach is actually better.
    - masklinn 7y ago
      > With lower opt-level Yeah but who'd do that? OTOH I found an "issue" with the playground: the compiler realises that since only a single array is ever created the functions can only ever be called with the proper bounds, so even with inline(never) everything gets compiled down to a memset without bounds check. And trying to defeat that by modifying buffer to be 1023, the compiler realises the assertions are always going to crash so it creates a weird-ass "bad" version (which it calls), only implements the slicing (and its error) in the base and doesn't even bother outputting code for the simpler version. Tricksty compiler. Using rand() and slicing so it can't know whether it's getting slices of 1023 or 1024 elements and generating a random fillchar (otherwise the fillchar gets moved into the functions) finally gets it to generate the expected code… and remove one of the two functions, calling the other one instead. So yeah, at this point I will conclude your version works just as well as the article's at getting optimised.
pdimitar 7y ago
I lack the knowledge of that particular piece of software but may I see that this has been extremely interesting to read! Also, I quite love the trend of "we rewrote our software in Rust" lately. I loved the speed of C/C++ about 15 years ago but the mounting complexity and the increasingly huge game of whack-a-mole pushed me away from them. Now I am gradually and casually learning Rust and I absolutely love what I am seeing and experiencing.
ncmncm 7y ago
This is a persuasive article, but it is still not clear whether Rust will achieve a self-sustaining mass of developers. That is, precisely: enough that each new person who masters Rust will be able to find work writing Rust. (To be clear, it would be good if this were to happen.) The deepest observation in the article is the remark about the cognitive load imposed by need to avoid pitfalls inherent in the programming model. I coded K&R C in the 80s, and while I never shipped an argument-type bug, avoiding them was a continual distraction. I coded C90 from 2006 to 2012, with a similar cognitive load I was happy to leave behind, although again I never shipped a bug caused by its failings. (We had bugs, but they were specification bugs: building the wrong thing.) Writing C++ post-14 has been an increasing pleasure. The cognitive load noted is present at times, but decreasingly so as the Standard Library and other libraries get more powerful. I am disappointed that the author found it too hard to implement a correct program with C++, but more disappointed that he chose to blame his tools for his failure. It has become fashionable to insist that correct programs cannot be written in this or that unfashionable language, but in fact a great many correct programs have been written in all of them, not excepting octal machine code. What varies is how much work is needed, and how many people can do it: it is certain that, for any chosen language past or future, most people cannot or will not write a correct program in it. Rust is not different in this. Many people coding C++, and perhaps most coding C, are not writing correct code, and have over the years written a very great deal of bad code. Will those same people succeed in learning Rust, and learning to write good code in it? It appears that the people currently adopting Rust are more skilled than is typical, so it seems hard to generalize their results to more typical programmers. C++ and Rust embody different choices in emphasis: Rust, safety; C++, library power. Stewards of C++ have decided the best way to get substantial correct programs is to enable encapsulating semantics in well-tested, well-optimized libraries that can be used without compromising performance or system architecture. Rust has chosen to emphasize making wrong low-level code hard to express, although its ability to express libraries will only increase. It will not be clear for a long time which will have better results. Will enough people learn Rust to write the correct low-level code needed? Will enough C++ programmers learn to use libraries that make (re-)writing dangerous low-level code unnecessary? What is clear is that the number of C++ programmers is growing faster than ever, and interest in coding to high level C++ libraries, as measured by attendance at an ever increasing number of conferences, is exploding. Even attendance at each ISO Standard meeting is quite a lot larger than at each previous one, now reliably in hundreds. The number of new C++ programmers in any unit time is still larger than the number of new Rust programmers, by a large factor. So, the experiment is interesting and valuable. What we can be confident about, either way, is that, for every C coder, or C++ coder writing low-level, often bad code, switching to Rust or to C++ using reliable libraries will result in more good code. It matters much less which they choose.

372 points by zimmerfrei 7y ago

https://www.youtube.com/watch?v=HkUUJ-rOUPg https://www.youtube.com/watch?v=HkUUJ-rOUPg has an output of the demo, and this is indeed very impressive for hardware that isn't supposed to output video at all...

makapuf 7y ago
May I also chime in and present my old bitbox console, a VGA on the same chips, with a dozen games and emulators for it ? http://bitboxconsole.blogspot.com/?m=1 http://bitboxconsole.blogspot.com/?m=1
- StavrosK 7y ago
  You need a lot more photos, in my opinion. I wanted to look at some to help me engage with the project and clicked around to the blog, then Github, then the wiki, then Hardware but left when I didn't see any photos anywhere.
  - makapuf 7y ago
    Thanks a lot for the feedback !
vijaybritto 7y ago
They are unbelievably performant! I think he needs to join the Android auto team and make their car software run smooth like this!
- pjmlp 7y ago
  Even if Android was pure C++ instead of Java, given how those Frameworks got designed, I doubt it would help much.
- sswezey 7y ago
  He's actually already part of the Fuchsia team, so he very well could be working on the successor to Android.
  - vijaybritto 7y ago
    Thats great news! I really hope they improve the android auto responsiveness
ncmncm 7y ago
That was seven years ago. I liked this better, from four years ago: https://m.youtube.com/watch?v=7yXxhvKmVb0 https://m.youtube.com/watch?v=7yXxhvKmVb0

dry_soup 7y ago

This was more interesting than the average "rewrite in Rust" blog post. Specifically I thought the part about how having compiler-enforced memory safety allows for the use of patterns that would otherwise be difficult to use safely interesting.

epage 7y ago
> compiler-enforced memory safety allows for the use of patterns that would otherwise be difficult to use safely I have to say, this has been amazing. For context, I'm a C++ developer who has had minimal run-ins with the borrow checker because I was already used to thinking about ownership. I refactored a large-ish Rust crate I maintain to use references for a fundamental type. I would have considered it irresponsible to do that with a C++ codebase unless absolutely required. Even if I did the upfront analysis to make sure it was safe, I'd need to re-do that exercise for every architectural change, at least. Instead with Rust, I just tried it on a whim, knowing the borrow checker had my back.
- pjmlp 7y ago
  In C++ you can have some help via static analysis. Unfortunately very few care to actually use them. https://www.bfilipek.com/2019/12/cpp-status-2019.html?m=1#what-additional-tools-do-you-use https://www.bfilipek.com/2019/12/cpp-status-2019.html?m=1#wh... As I learned since Turbo Pascal days, if it isn't part of the language, it seldom gets used.
  - roca 7y ago
    That's quite right. But also there is a big difference between "the static checker will probably catch errors of this kind" and "the compiler will certainly catch errors of this kind". The latter is far more liberating.
Avshalom 7y ago
worth pointing out, i think, that that was the original justification for rust: that by building a safe language that still gave you the level of control you'd expect from a C(++) they could experiment with all sorts of otherwise difficult and subtle concurrent code to figure out what would actually work/help. Much like Erlang's concurrency fell out as a consequence of needing to be confidently write high availability code, Rust's general safety falls out of needing to write concurrent code fearlessly.

PudgePacket 7y ago

Author also wrote "Learn Rust the dangerous way". > In this series, I'm trying something different. Let's take a grungy, optimized, pointer-type-punning, SSE-using, heap-eschewing, warning-disabling C program and look at how we could create the same program in Rust. http://cliffle.com/p/dangerust/0/ http://cliffle.com/p/dangerust/0/

rvr_ 7y ago
I dislike nitpicking, but the following is wrong: "In C, bodies[i] is exactly the same as *(bodies + i) — it performs pointer arithmetic and a dereference and nothing else. In particular, it assumes that you have some reason to know that i is a valid index for the array. This is the common case in C and gets a shorthand using square brackets." In C, bodies[i] is not (bodies + i), with the exception being char arrays. The general rule is: bodies[i] = *(bodies + i * sizeof(bodies[0]))
- heinrich5991 7y ago
  No, it's actually defined to be the same. The pointer arithmetic in C follows your intuitive understanding of array indexing. #include <stdio.h> int main() { int array[] = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16}; printf("%d %d\n", array[4], *(array + 4)); return 0; } prints "4 4".
- barrkel 7y ago
  + in C advances pointers in sizeof-pointed-to thing increments. It's why ++ works too. Similarly - on pointers returns a difference in units of sizeof-pointed-to thing.
- jcranmer 7y ago
  The semantics of a[i] is exactly that of *(a + i). Which is why i[bodies] is actually legal C code.

deleted 7y ago

[deleted]

0xdead 7y ago

It is hard to take someone seriously when they write "C/C++" in their articles like C and C++ are the same thing.

blargmaster33 7y ago

Why did rust not go with standard c syntax

steveklabnik 7y ago
There are features Rust has that aren't in C, and so have no precedent. So it's not really possible. We did take a lot of influence from C, though.
pornel 7y ago
Rust did copy C syntax to a large degree. It didn't copy some of the mistakes such as type declarations, which are hard to parse (typedef makes AST depend on itself) and hard to understand ("array of function pointers" is a head-scratcher in C, easy in Rust).
- masklinn 7y ago
  Aside from the feedback loop, C-style decls also make the language less regular in the presence of type inference as you need an "inference pseudo-type" to take the place of the type you're omitting. Also prefix casts have similar issues as prefix type + typedef.
therein 7y ago
I appreciate the ability to avoid having to enclose my conditions in parenthesis, and the main operand of statements like match, switch etc. to not be wrapped in parenthesis.
heavenlyblue 7y ago
I am sorry, but if you think language syntax “beauty” is something to be considered while picking the language to develop in, then you’re probably bot senior enough to make decisions which one of them to use for the project. There’s only one true answer to this: “it doesn’t matter at all - I’ll learn it in a week@.
- orf 7y ago

bluejekyll 7y ago

If the author is reading this, this might be the greatest single article I’ve read that expresses the benefits of Rust and how minimal the trade-offs are. Wonderfully written piece, thank you! BTW, in your bounds checking examples, I think it can be further simplified (would need to check the output to see if it's any better), this: fn fill_1024(array: &mut [u8], color: u8) { // Perform an explicit, checked, slice of the array before // entering the loop. let array = &mut array[..1024]; for i in 0..1024 { array[i] = color; } } Could be: fn fill_1024(array: &mut [u8], color: u8) { for i in &mut array[..1024] { *i = color; } } Here's the playground: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=832233ec0b3319fcec86c2330be93523 https://play.rust-lang.org/?version=stable&mode=debug&editio... Thanks again, this is a wonderful piece.

masklinn 7y ago
I'm quite terrible at assembly, but it looks like both got inlined and compiled to a memset: leaq 8(%rsp), %rdi movl $1024, %edx movl $255, %esi callq *memset@GOTPCREL(%rip) for the 0xFF fill and leaq 8(%rsp), %rdi movl $1024, %edx movl $238, %esi callq *memset@GOTPCREL(%rip) for the 0xEE fill. However it turns out the original also compiles to that, because the inlining means the fixed size array is visible to the optimiser.
- jchw 7y ago
  With lower opt-level, it does not get optimized to memset, and the succinct version actually compiles better, eliding the bounds checking in the loop as well as, for some reason, needing a smaller stack. https://godbolt.org/z/Wvuen5 https://godbolt.org/z/Wvuen5 This leads me to believe the shorter approach is actually better.
  - masklinn 7y ago
    > With lower opt-level Yeah but who'd do that? OTOH I found an "issue" with the playground: the compiler realises that since only a single array is ever created the functions can only ever be called with the proper bounds, so even with inline(never) everything gets compiled down to a memset without bounds check. And trying to defeat that by modifying buffer to be 1023, the compiler realises the assertions are always going to crash so it creates a weird-ass "bad" version (which it calls), only implements the slicing (and its error) in the base and doesn't even bother outputting code for the simpler version. Tricksty compiler. Using rand() and slicing so it can't know whether it's getting slices of 1023 or 1024 elements and generating a random fillchar (otherwise the fillchar gets moved into the functions) finally gets it to generate the expected code… and remove one of the two functions, calling the other one instead. So yeah, at this point I will conclude your version works just as well as the article's at getting optimised.

pdimitar 7y ago

I lack the knowledge of that particular piece of software but may I see that this has been extremely interesting to read! Also, I quite love the trend of "we rewrote our software in Rust" lately. I loved the speed of C/C++ about 15 years ago but the mounting complexity and the increasingly huge game of whack-a-mole pushed me away from them. Now I am gradually and casually learning Rust and I absolutely love what I am seeing and experiencing.

ncmncm 7y ago

This is a persuasive article, but it is still not clear whether Rust will achieve a self-sustaining mass of developers. That is, precisely: enough that each new person who masters Rust will be able to find work writing Rust. (To be clear, it would be good if this were to happen.) The deepest observation in the article is the remark about the cognitive load imposed by need to avoid pitfalls inherent in the programming model. I coded K&R C in the 80s, and while I never shipped an argument-type bug, avoiding them was a continual distraction. I coded C90 from 2006 to 2012, with a similar cognitive load I was happy to leave behind, although again I never shipped a bug caused by its failings. (We had bugs, but they were specification bugs: building the wrong thing.) Writing C++ post-14 has been an increasing pleasure. The cognitive load noted is present at times, but decreasingly so as the Standard Library and other libraries get more powerful. I am disappointed that the author found it too hard to implement a correct program with C++, but more disappointed that he chose to blame his tools for his failure. It has become fashionable to insist that correct programs cannot be written in this or that unfashionable language, but in fact a great many correct programs have been written in all of them, not excepting octal machine code. What varies is how much work is needed, and how many people can do it: it is certain that, for any chosen language past or future, most people cannot or will not write a correct program in it. Rust is not different in this. Many people coding C++, and perhaps most coding C, are not writing correct code, and have over the years written a very great deal of bad code. Will those same people succeed in learning Rust, and learning to write good code in it? It appears that the people currently adopting Rust are more skilled than is typical, so it seems hard to generalize their results to more typical programmers. C++ and Rust embody different choices in emphasis: Rust, safety; C++, library power. Stewards of C++ have decided the best way to get substantial correct programs is to enable encapsulating semantics in well-tested, well-optimized libraries that can be used without compromising performance or system architecture. Rust has chosen to emphasize making wrong low-level code hard to express, although its ability to express libraries will only increase. It will not be clear for a long time which will have better results. Will enough people learn Rust to write the correct low-level code needed? Will enough C++ programmers learn to use libraries that make (re-)writing dangerous low-level code unnecessary? What is clear is that the number of C++ programmers is growing faster than ever, and interest in coding to high level C++ libraries, as measured by attendance at an ever increasing number of conferences, is exploding. Even attendance at each ISO Standard meeting is quite a lot larger than at each previous one, now reliably in hundreds. The number of new C++ programmers in any unit time is still larger than the number of new Rust programmers, by a large factor. So, the experiment is interesting and valuable. What we can be confident about, either way, is that, for every C coder, or C++ coder writing low-level, often bad code, switching to Rust or to C++ using reliable libraries will result in more good code. It matters much less which they choose.