19 ms·
Integrating “safe” languages into OpenBSD?
- Annatar 9y agoBravo Theo! Someone finally stood up to hype!
- deleted 9y ago[deleted]
- xwvvvvwx 9y agoMaybe I'm misunderstanding but there is a pretty serious effort to rewrite all the gnu coreutils in Rust: https://github.com/uutils/coreutils https://github.com/uutils/coreutils
- mrweasel 9y agoThat is sort of addressed by Theo: >Such ecosystems come with incredible costs. For instance, rust cannot even compile itself on i386 at present time because it exhausts the address space.
- jrs95 9y agoPeople are still using i386? I'd assume even if they are, it's such a tiny minority that it shouldn't be an excuse to hold everyone else back.
- zaarn 9y agoLinux supported i386 until 2012, I think in this context it is more likely to refer to pre-Pentium 86 CPUs though (I'm not certain on that) I've seen i386 multiple times to also refer to any x86 Arch.
- dmm 9y agoi386 in this context refers to 32bit intel x86 cpu architecture in general and generic pc compatibles specifically. OpenBSD currently runs on 486s and better: > All CPUs compatible with the Intel 80486 or better,[0] [0]https://www.openbsd.org/i386.html https://www.openbsd.org/i386.html
- tsmarsh 9y agoOne of OpenBSDs sweet spots is turning old hardware into useful, secure, reliable, network infrastructure. i386 might not be as popular as it was on 'normal' OS, but I wouldn't be surprised if OpenBSD had a lot of people still using it.
- raimue 9y agoOpenBSD supports a wide variety of hardware platforms, including machines with Alpha, PA-RISC, and SPARC64 processors. On each of them, the base system is able to compile itself. If rustc cannot even build itself on i386, what kind of support can we expect for other platforms with an even smaller user base? On a project such as OpenBSD they cannot suddenly drop platforms and only support amd64 as portability is one of their main "selling" points. Furthermore, that would also mean to lose the developers that are interested in these alternative platforms and probably chose OpenBSD because of the platform support. Furthermore, such developers are usually not only contributing to platform-specific parts, but also to system utilities and ports. For the full list of platforms, see https://www.openbsd.org/plat.html https://www.openbsd.org/plat.html
- masklinn 9y ago> On a project such as OpenBSD they cannot suddenly drop platforms and only support amd64 as portability They… don't need to? That rustc can't compile itself on i386 doesn't mean you can't ship a rustc for i386, it just means you have to cross-compile it.
- _jal 9y agoAddressed in the article. > In OpenBSD there is a strict requirement that base builds base.
- raimue 9y agoI see it as part of portability that you do not need to use any external system for bootstrapping. Imagine as a developer who compiles base from source you had to find another system only to compile rustc and then transfer it to your machine. And you would not have to do this only once, but for every compiler bug fix coupled with the overall rapid evolution of Rust. I think many in the OpenBSD community would oppose such an approach, even without further considering other aspects such as security implications.
- clort 9y agoThis is the case IIRC with Android - even though I want to build it for a 32-bit ARM architecture, I can't build it on anything but 64-bit. I guess the vast majority of Android systems cannot even compile Android! https://source.android.com/setup/requirements https://source.android.com/setup/requirements
- DCKing 9y agoOpenBSD is a perfectly servicable operating system on an old Atom board I have (nice router!), and on i386 only Core Duo iMacs/Macbooks long abandoned by Apple. These will be perfectly good machines to use for a long time, and it's really great you can get up to date support on these systems from this great project. Obsolete means different things to different people. I'm a big fan of what Rust promises, but the solution is not that OpenBSD changes its policies or that OpenBSD drops i386. Rust should become self hosting on i386.
- mfukar 9y agoThere's significant infrastructure in place that uses i386. It used to be fairly popular, I'm sure you can google it.
- rubyfan 9y agoTheo would say, if you think so then get to writing code instead of commenting about it. These aren’t just philosophical comments, the implications for him and OBSD devs is to spend massive time developing these things. Dropping a supported platform and taking on a huge investment of effort needs serious justification.
- jlgaddis 9y agoYep: $ uname -a OpenBSD hostname 5.9 GENERIC.MP#6 i386
- sillysaurus3 9y agoRust fell into the same trap that killed many, many gamedev companies: Performance matters. Even more than features.
- oblio 9y agoIt depends what kind of performance. The only performance they are "lacking" seems to be compilation speed (an annoyance, but they're working on it) and compilation memory usage (rarely a problem, considering we also have C++, Java and C# around :) ).
- ionforce 9y agoAre you implying that Rust isn't performant?
- yellowapple 9y agoIf compilation exhausts the IA32 address space, then I'd say it's not adequately performant as a whole, regardless of how "efficient" the resulting binaries might be.
- mikepurvis 9y agoHard disagree. Compilation is like encoding a video— it's a price you pay once, and if you know the resulting binary will be run millions of times, it's totally worthwhile spending a lot of compute and memory upfront to get that binary as fast as possible.
- madez 9y agoSo, you say Python is totally missing the point and is wrong? Even when encoding videos, performance matters. Also, that optimizing compilation is important is no reason to not work on i386. This is a point Rust needs to fix. And not only i386 support, but also other architecture families, as host.
- 9y ago
- rtpg 9y agoThis seems like a bit of a trite point unless many rust developers are actually working inside i386. Though the compiler itself might not work very well in i386. Not many people are whining about our C compiler toolchains not fitting into our microcontrollers.
- 2trill2spill 9y ago> This seems like a bit of a trite point unless many rust developers are actually working inside i386. Though the compiler itself might not work very well in i386. Trite point? OpenBSD supports 1386[1] if they pull a rust compiler into base and start rewriting things in rust, then they can't support 1386. Dropping a supported platform is not a "trite point". [1]: https://www.openbsd.org/plat.html https://www.openbsd.org/plat.html
- JetSetWilly 9y agoAt this point i386 is legacy for most of the world. OpenBSD is an ultra-conservative, orthodox project, therefore they will probably support i386 for years into the future - I mean, they supported VAX until 2016. That is a choice they are entitled to make, the trade-off being it would appear to make most modern technologies a poor fit for adoption in OpenBSD - that's the price they have to pay. It is a problem of their own making.
- Paianni 9y agoIf OBSD feels it can make use of older arches then so be it, many users will find less intensive jobs for the respective hardware and it saves it from going to waste/recycling.
- yellowapple 9y agoNot to mention plenty of people in the "developed" world (let alone "developing") can't afford to buy a new computer, and thus are going to use 32-bit "legacy" desktops and laptops for a very long time.
- accurrent 9y agoHe has a point though... As we get more CPU/ram we as programmers don't even bother to check how many resources we are using. Personally, I don't know whether its a good thing or a bad thing. Also in terms of systems languages I believe only rust has some potential to truly replace C. Although, a large part of C usage still takes place in the embedded world where rust has yet to be ported to many embedded processors. Go's GC makes it a show stopper for use in many places as for haskell I would be very interested to see more low level embedded programming going on in it.
- masklinn 9y ago> He has a point though… Does he? He "stat[es as] fact" that > There has been no attempt to move the smallest parts of the ecosystem, to provide replacements for base POSIX utilities. which as xwvvvvwx notes is categorically wrong, then points out that rustc can't compile itself on i386, which is relevant… how?
- _jal 9y agoHe's talking specifically about OpenBSD base. Unless you can point to a rust binary in OpenBSD that Theo forgot about, he's not wrong. i386 is relevant because OpenBSD supports i386.
- HelloNurse 9y agoAnd if the end user is unable to compile everything, i386 would be only half-supported (or maybe, for the austere OpenBSD maintainers, not supported at all).
- masklinn 9y ago> He's talking specifically about OpenBSD base. No, he is very explicitly saying that > There has been no attempt […] provide replacements for base POSIX utilities. Which once again is categorically false, a github repository purporting to do exactly that has been provided. > i386 is relevant because OpenBSD supports i386. i386 is supported, the issue is compiling the compiler on i386.
- IshKebab 9y agoAnd Go: https://github.com/ericlagergren/go-coreutils https://github.com/ericlagergren/go-coreutils Does he really not know this or is he ignoring them to make a point?
- tankenmate 9y agoSome would consider Go to not be as safe as Rust for example. But his argument still holds for POSIX compliance, it is a loveless task so it it gets done at snail's pace.
- deleted 9y ago[deleted]
- akerro 9y agoIt uses remote dependencies. First compilation in a single-run environment will be very slow, there won't be second compilation. Go compiler is fast when you add the `-i` flag, without it it takes a couple of seconds to compile a few hundred lines, a few more minutes when you have to `go get` packages. Now, github goes down, your build is broken for that time.
- gjem97 9y agoNot sure what the "it" is in your first sentence, but allow me to address the rest of your points. "First compilation ... will be very slow": the go compiler is in fact quite fast, orders of magnitude faster than C++ or rust compilers. The fact that there's a noticeable pause when you want to compile thousands of files does not mean that its slow. "There won't be a second compilation": not for each top level tool, but there certainly could be shared packages that don't need to be recompiled. "Go compiler is fast when you add the `-i` flag", ok, do that then. "Now, github goes down, your build is broken": you only need to depend on external github references if you want always to build against the latest version of your referenced code. I can't imagine anyone interested in stability wants this. There are lots of options for vendoring your dependencies in tree.
- akerro 9y ago
- akerro 9y ago>Such ecosystems come with incredible costs. For instance, rust cannot >even compile itself on i386 at present time because it exhausts the >address space. Is cargo supported on i386 platforms? Also Rust complies itself, afaik there is no way to compile Rust/Cargo but to use previous version of it. If one of the past builds of Rust is backdoored, any version between then and now is backdoored, language is safe, environment... as safe as it was never compromised. OpenBSD compiles everywhere where C code works, Rust/Cargo works where it's supported and it will takes decades to catch-up on some architectures.
- hood_syntax 9y agoI'm pretty sure that attack you describe is mentioned in literature as essentially undefeatable. I really wish I could remember exactly what the name of it was; the gist is, there has to be a first compiler somewhere. If at any point in the chain, the compiler is infected with a self-propagating virus that hides itself in the byte code of the binary, it can ensure that the exploit is in every future version of the compiler. I may be remembering the details a bit wrong, but it was a good read.
- traverseda 9y agoYou want the "trusting trust" keyword.
- skj 9y agoA talk by Ken Thompson, "trusting trust".
- mcguire 9y agoTechnically, his Turing award lecture.
- edwincheese 9y agoI think you are referring to “Reflections on Trusting Trust” by Ken Thompson. https://www.ece.cmu.edu/~ganger/712.fall02/papers/p761-thompson.pdf https://www.ece.cmu.edu/~ganger/712.fall02/papers/p761-thomp...
- jackpot51 9y agoTheo is completely ignorant here. I also saw him spout similar inaccuracies in this video: https://youtu.be/fYgG0ds2_UQ?t=2112 https://youtu.be/fYgG0ds2_UQ?t=2112 Uutils and Redox are setting out to provide POSIX compatible coreutils, and Redox builds from scratch in less than 30 minutes.
- cat199 9y ago> Theo is completely ignorant here. perhaps, in one sense - or, alternatively is busy not spending all his time chasing the latest 5000 fads and 100000 not-yet-implemented projects which may or may not ever end up being completed...
- creatonez 9y agoUnless I'm mistaken, POSIX compatibility is only a goal of uutils, not Redox OS.
- smhenderson 9y agoWell, jackpot should know, he's the author of Redox.
- octo_t 9y agoSomething that is 'setting out' to provide something, doesn't seem like it is production ready enough to run a production operating system on top of.
- blub 9y agoYou might want to add a disclaimer or at least a note in your profile that you're the same jackpot51 which is the creator of Redox.
- jackpot51 9y agoSorry, I did not mean to be secretive.
- JetSetWilly 9y ago> So rather than bothering to begin, you wrote an email. This is where Theo appears to me to be quite different to eg Linus. He plays the man, not the ball, with personal attacks.
- jackpot51 9y agoI believe that Linus's attacks are usually good intentioned, to get people in the Linux community to act better. I am not sure what Theo's intent is.
- ams6110 9y agoIf you want to effect change, do some work, don't write emails.
- cuckcuckspruce 9y agoAs can be expected from a project with a slogan of "Shut up and hack"[1]. [1] https://www.openbsd.org/lyrics.html#51b https://www.openbsd.org/lyrics.html#51b As the notes indicate, this saying goes back much further than this release song.
- ams6110 9y agoWhere's the attack there? It's a statement of fact.
- JetSetWilly 9y agoIf I say "You have a big nose, it makes you ugly" as the first line in my email, it may well be a "statement of fact" and yet still be a personal attack. Your implication that a statement of fact cannot be a personal attack is untrue. Whether the OP has or has not written Posix utilities in rust is of no relevance to the actual argument whatsoever, it is just a way to have a dig at the OP.
- SomeStupidPoint 9y ago
- SCHiM 9y agoThe points made against switching to newer languages are valid. It will cost time, it will cost effort, it will take many years of work to convince stubborn maintainers to switch. But it will not stop _everyone_ from adopting better languages, and their efforts will eventually surpass the older, less secure systems. When there's feature parity, each and every new exploit will be called out: "this wouldn't have happened in our system'. And that is a good thing. There _are_ better alternatives to C. As for the point about how nobody is working on replacements, that's wrong. There's (partial) replacements for most coreutils written in rust, and a whole kernel has been under active development for years now.
- jackpot51 9y agoYep, I look forward to the day OpenBSD is obsolete. How Theo can be so anal about features like ASLR but then completely disregard the potential of compiler enforced memory safety is beyond my cognitive power.
- busterarm 9y agoConsidering the complete lack of viable competition in the space, that day is a long way off.
- mjl- 9y agoi also saw the same eurobsdcon youtube video. i agree it is somewhat hard to imagine replacing long-time unix tools with newer versions that only differ in being implemented in a new language. making sure nothing breaks takes so much time, with not enough reward. but writing new tools in safer languages makes more sense to me. yes, you would need the toolchain in base. not easy, but should be possible. i only know go. perhaps it's still too much of a moving target for openbsd. you could write programs in new safe language and put them in openbsd ports. problem is that it then isn't a "part of openbsd". do openbsd developers really want to write their next daemon in c? for how long will they stick to c?
- fsloth 9y agoA bit incendiary: I fail to see what advantage rewriting existing and proven tools with a new language would bring. Shouldn't the main value new tools bring to be enable writing of new things? Isn't focusing on existing utils more like a lack of imagination and OCD on optimizing a thing beyond any further value?
- geggam 9y agoMade me chuckle to think of writing systemd in rust
- cesarb 9y ago> I fail to see what advantage rewriting existing and proven tools with a new language would bring. This is not the first rewrite, and won't be the last one. Earlier environments were very memory-constrained, which led to optimizing for memory usage; a rewrite with less memory constraints can focus on speed (as mentioned in the GNU coding standards: "For example, Unix utilities were generally optimized to minimize memory use; if you go for speed instead, your program will be very different. [...]"). The current challenge is the "end of Moore's law" leading to an increasing use of multiple cores, instead of faster cores. Developers will have to focus on parallelism instead of raw speed, and new languages can help. > Shouldn't the main value new tools bring to be enable writing of new things? Or writing old things in a new way. > Isn't focusing on existing utils more like a lack of imagination and OCD on optimizing a thing beyond any further value? These tools are the base over which your system is built (their GNU version isn't named "coreutils" for nothing). Focusing more effort on them makes sense.
- bad_user 9y ago> The current challenge is the "end of Moore's law" leading to an increasing use of multiple cores, instead of faster cores. Developers will have to focus on parallelism instead of raw speed, and new languages can help. Moore's law is indeed nearing its end, but this also ends the multiple core trend. If you're betting on multiple CPU cores taking off more than they did already, don't.
- vog 9y ago
- liveoneggs 9y agoThe bootstrap problem is real and getting worse: go needs go, haskell needs haskell, rust needs rust; it's nasty. There are already "better" and "safer" languages that could easily replace the entire unix userland. Heck, awk can do half of it, but all of that C code is already written...
- cesarb 9y ago> go needs go, haskell needs haskell, rust needs rust And C needs C, or for more recent compilers, C++ needs C++. You just don't notice it because these two languages are already part of the base system.
- progman 9y ago> go needs go, haskell needs haskell, rust needs rust That's painful. It's one big point which Nim [1] does better. It compiles to C and bootstraps from C. That makes porting to other platforms much easier. I don't understand why Haskell and Rust don't provide bootstrapping from C. Did they write the first compilers with Assembler? > You just don't notice it because these two languages are already part of the base system. C (not C++) has become the essential foundation on any platform. You will likely not be able to sell your embedded system unless it supports C. [1] https://nim-lang.org https://nim-lang.org
- cesarb 9y ago> I don't understand why Haskell and Rust don't provide bootstrapping from C. Did they write the first compilers with Assembler? I don't know about Haskell, but the first Rust compiler was written in OCaml. See for instance the comments at https://www.reddit.com/r/rust/comments/6nt2j1/is_there_any_existing_literature_on_the_old_ocaml/ https://www.reddit.com/r/rust/comments/6nt2j1/is_there_any_e...
- progman 9y ago> See for instance the comments ... Interesting! So, it should basically be possible to bootstrap from source using OCaml and the old source from the Git repository. It's interesting that OCaml was used to develop Rust since OCaml is also used for formal verification of SPARK (a dialect of Ada) which is another language for safety critical applications. OCaml seems to be a good choice to invent new big languages.
- Nokinside 9y agoIf "safe" language would be real magic pill for safe programming, Ada would have replaced C and C++ in systems programming and safety critical applications long time ago.
- steveklabnik 9y agoThere are no magic pills. There's a lot more to languages than just the languages themselves.
- sitkack 9y agoAnd it should have. The use of C and C++ in safety critical applications should be classified as gross negligence.
- agumonkey 9y agoI believe Ada suffered from its DoD origins, from the era of giant monolithic project management and its verbosity. Rust is a different kind. Also, I've seen people doing generic low level network stacks (or at least articles about it) in Ada, seems lovely.
- Nokinside 9y agoAda is extremely well designed language. Verbosity designed for better safety trough readability and there is science backing it up.
- shmerl 9y ago> I wasn't implying. I was stating a fact. There has been no attempt to move the smallest parts of the ecosystem, to provide replacements for base POSIX utilities. Apparently Redox developers disagree :) https://github.com/redox-os/coreutils https://github.com/redox-os/coreutils These are even based on BSD coreutils.
- jstewartmobile 9y agoThis moment should be in CS textbooks. Want something? Wrong way: write an e-mail Right way: Here is my feature-per-feature, bug-per-bug rewrite of grep in Rust
- reggieband 9y agoI remember I stopped frequenting /r/programming when it became this weird Haskell echo chamber. It was a bubble where a small but vocal faction inside the community seemed to over react to any criticism. They methodically and tirelessly responded to every comment with an endless litany of "facts" showing how Haskell could do anything from os kernels to game programming. They touted every industry mention of Haskell use and demanded evidential proof for any claim Haskell might not be suited for every situation. It reminds me of the Sea Lion comic[1]. I'm not sure if that faction still holds as much sway today. I get the same feeling on Hacker News about Rust. There is a point where well-meaning evangelism does more harm than good. I sometimes get the feeling people spend more time writing defences of these languages than they spend time writing programs in them. 1. http://wondermark.com/c/2014-09-19-1062sea.png http://wondermark.com/c/2014-09-19-1062sea.png
- rapsey 9y ago> I sometimes get the feeling people spend more time writing defences of these languages than they spend time writing programs in them. Programmers love language wars.
- srcmap 9y ago> Programmers love language wars Programming Language Advocates love language wars Most programmers just like to be aware of ALL the PROs and CONs of the languages and choose the right one for the task at hand.
- Nomentatus 9y agoIn WWII it was famously true that fighter pilots would fiercely argue for the absolute superiority of whatever plane they had been flying for a few months. For example, Thunderbolt pilots hated being moving to Mustangs (etc) - even though to our eyes the Mustang is obviously a superior plane. Once your instincts adapt to something intimately (and programming languages require that); that's what feels right. You'll argue vociferously against a change and use what are objective reasons. (The Thunderbolt could dive faster, it was a brick.) Of course, in the United States Army Air Force, you still had to switch. The old hands usually remain convinced of the superiority of what they know, and irritated by Young Turks telling them something else, nicely or not. Because the Young Turks can't be right. But they are, often.
- AtticusTheGreat 9y agoIt would be interesting to try to cobble together a Linux distribution with a rust userland, using coreutils (https://github.com/uutils/coreutils https://github.com/uutils/coreutils) and over time building more and more of the userland/interface in rust.
- cat199 9y agowhyy couldn't they pick a different name for this project? rustutils? coreutils-rust? name collision like this is how we have millions of people running around thinking that vim is vi while they bash emacs for having bloat as compared to the 'minimalism' of 'vi'..
- nanolith 9y agoI have spent a long time using alternative languages. I'm a compiler geek. However, most of these languages ultimately lead to disappointment. Not because they necessarily fail, but rather because the problem of writing software is not a problem of language or platform. It is a problem of thought and abstraction. So, the big promises that these new languages and platforms make will ultimately only be fulfilled for a subset of applications. The rest of the time, we must still trudge through the process of good design, good testing, and good real-world feedback. People love to beat up on C, because it does not have feature X or feature Y, and because there is plenty of bad C software to point at. However, it isn't the language that leads to bad software. Any entrenched popular language has plenty of bad software examples, and plenty of security vulnerabilities. The problem isn't the language, but rather, the developer. A good developer who understands and appreciates the problem can write good software in machine code, C, C++, Rust, Java, Haskell, C#, Go, D, or any other language or platform. A good developer who understands secure programming processes can write secure software in any of these languages, given enough time. What these higher-level languages offer is better abstraction, which can make time-to-market faster. However, in a deeply entrenched system that is already in market, maintaining the current languages and platforms is typically the better play. This is especially true when the source code is as meticulously maintained as it is with OpenBSD. Someone who understands the basics of C can easily read the OpenBSD kernel or userland, and understand everything about how a given program works. Some knowledge of BSD Make, Bourne shell, Korn shell, and Perl may be required to understand the init scripts and build process, but in all, it is a system that is easy to understand, easy to maintain, and with an excellent development process in place to deliver two builds a year. OpenBSD isn't broke, and a new language isn't going to fix it. That being said, there are tools that the OpenBSD team could use that neither impact the build time nor require adopting fad language of the week. There are excellent model checkers out there for C, and quite a few proof assistants are gaining the ability to check proofs on C code directly, using Separation Logic and Hoare triplets. These tools can complement an entrenched C system by providing an external mechanism of verification that allows designers to formally verify that implementation meets specification. While formally verifying all of OpenBSD would be a tall order, taking hundreds of man years, formally verifying critical pieces of the kernel and creating simplified contract boundaries for the remaining code would be an excellent complement to the code review process that the OpenBSD team already does. Better still, it is something that could pay off without reinventing OpenBSD to match the current fad, as such verification can be run independently of the standard build process. If I were to make a suggestion to the OpenBSD team, it would be to explore such tools. But, to de Raadt's point, such a suggestion would only be valid if someone did the heavy lifting to put such a tool in place, and if it were of utility to the team. They won't adopt something for the sake of that thing, but they are keen for anything that can practically improve the security of their system.
- Polyisoprene 9y agoSeems most os developers don’t care about the users and only about their maintenance and adoption cost. I WANT my computer and files protected by software written in a safe way using safe languages. I don’t care about the compile time of grep, I stopped caring about that kind of dick measurement competitions a long time ago. Give me a safe, open operating system and I’m willing to trade Posix compatibility and compile speed.
- chc4 9y agoI'm looking forward to the world of running Servo on Redox and never having to worry about a buffer overflow again.
- dbolgheroni 9y agoCurious, but this kind of shitty attitude won't work with OpenBSD nor with any other open source that I know. It is not about what you care, but about what they care. Like if they own you something. That being said, I wonder in what safe language the OS you used to write this is written.
- Polyisoprene 9y agoI wouldn't call protecting my computer and data being shitty attitude. OS developers need to realize that secure is not something they can compromise on going forward.
- w8rbt 9y agoSomeone should start by writing all the setuid apps in Go. OpenBSD would probably be more open to Go than Rust since Go is a lot like C with a Garbage Collector. They love C.
- anthk 9y agoThey already have pledge(4) with C.
- andrewmcwatters 9y agoThe problem "safe" languages have in this context is that they really truly are no better than C, for the simple sake of them throwing out performance and resource considerations from the get-go. Additionally, if you take even just a day to read up on UNIX and C history, you'd quickly find the two are essentially intertwined, and that to usurp C, you'd have to nix UNIX, too. I've yet to see anyone with serious industry experience come out and say sane things about replacing C. It's just not going to happen. Not in my lifetime, unless someone decides to write a new operating system, and Linux subsequently loses footing.