18 ms·
The Path to Rust
- joobus 10y agoI'd like to know what the author considers "systems work"; I don't consider garbage-collected languages (Go, Python) "systems" languages.
- adamnemecek 10y agoYou are probably referring to the sentence "Rust is quickly becoming my favorite language for all systems work (which is most of what I do anyway), and has largely replaced both Go, Python, and C/C++ in my day-to-day." One could also interpret is as "Rust is quickly becoming my favorite language for all systems work [...], and has __also__ largely replaced both Go, Python, and C/C++ in my day-to-day."
- Jonhoo 10y agoThat is indeed how I meant it. That said, Go is certainly being used as a systems language, and especially in the domain of networked services, it is rapidly replacing C/C++.
- Scarbutt 10y agoreplacing python is a bit of stretch no? I'm sure you can, but is it practical? like for quickly hacking stuff up.
- IndianAstronaut 10y agoI find getting stuff up and running with Go is quite easy. Just set up a small package with some quick hacks and then a separate file to run the package functions.
- IndianAstronaut 10y agoI find getting stuff up and running with Go is quite easy. Just set up a small package with some quick hacks and then a separate file to run the package functions.
- Jonhoo 10y agoActually, no. I've found that I'm maybe even more productive in Rust than I am in Python, even for one-off tasks. The one exception is probably plotting, but I'm sure a Rust library for that isn't far off.
- pjmlp 10y agoOnce upon a time I used Python for systems administration tasks and that was a long time ago, around 2003. Python doesn't offer much against languages that compile to native code, have type inference (if strong typed) and REPL. The only reason for me, would be if I am required to make use of a framework or library that requires me to use Python.
- lmm 10y agoI use Scala for that kind of use case sometimes. If you have a REPL and you have good enough type inference you can pretty much just do what you would do in Python, and as a bonus you can reuse library functions from your real system rather than having to duplicate or shell out to them from your scripts
- jguegant 10y ago"and especially in the domain of networked services, it is rapidly replacing C/C++." Rapidly? Where?
- Jonhoo 10y agoI don't have any numbers to support this claim, but it's certainly true that we are seeing a lot of software being developed in Go, and network software in particular.
- ngrilly 10y agoExamples of services that typically would have been written in C/C++ but are written in Go: Docker, YouTube Vitess (a MySQL proxy), CockroachDB (a distributed and strongly consistent SQL database). Other examples of companies using Go for network services: Dropbox (most backend), CloudFlare (Railgun and other things), Uber (geofencing), Bitly (messaging with NSQ), Disqus (realtime commenting), DigitalOcean, Heroku, SoundCloud, Dailymotion.
- pjmlp 10y agoThanks to the adoption of Docker, Dropbox and other well known SV companies. Even Microsoft is now contributing to the eco-system due to their collaboration with Docker. Also Joe Duffy (from MSR Midori) tweets about his Go experiences. Personally I prefer other languages, but I am surely happy to see more Go and less unsafe languages being used.
- IndianAstronaut 10y agoGo also has the advantage of a simple syntax like Python. Moving some of our ETL processes to Go from Python has been quite easy.
- ngrilly 10y agoThere is also Netflix using it for a proxy: > The decision to use Go was deliberate, because we needed something that had lower latency than Java (where garbage collection pauses are an issue) and is more productive for developers than C, while also handling tens of thousands of client connections. Go fits this space well. Source: http://techblog.netflix.com/2016/05/application-data-caching-using-ssds.html http://techblog.netflix.com/2016/05/application-data-caching... Discussion: https://www.reddit.com/r/golang/comments/4l0fv2/the_netflix_tech_blog_application_data_caching/ https://www.reddit.com/r/golang/comments/4l0fv2/the_netflix_...
- sanderjd 10y ago"Systems" has come to include things like networked services (eg. HTTP and DNS servers), which you can certainly do successfully in GC'd languages in many circumstances. Personally, I'd just like to see a widely agreed upon definition, so that we can all stop being confused when this comes up.
- LionessLover 10y agoNaming confusion is completely normal. Even in fields like learning human anatomy you'll learn different words for some things depending on who your teacher is, even though one would think that there should have been sufficient time for such issues to settle and the professionals agree on one word. But naming always is an accident of space and time - where did it happen, when did it happen, and then the path from there. Language is dynamic and a little fuzzy - in the sciences as well. You can define all you want, the problem always is other people, who choose their own definitions. It's fun - it keeps you on your toes :-) In this context, for example, I don't see a reason for a world government to impose one definition of "systems programming" worldwide by military force (this is what it would take!). Just like in almost everything in human language, you get it from the context. You should try to teach a computer to recognize human speech including the meaning, then you'll realize that almost all of it relies on context and pre-existing knowledge. "Precision" comes from the interpretation - human language communication is the nightmare of people who love functional programming, it's full of hidden state and context and active interpretation. Which is why it's so easy for this to happen: http://dilbert.com/strip/2015-06-07 http://dilbert.com/strip/2015-06-07 If someone wants to argue, there is no way to provide a water-tight human-language text that "Dick from the Internet" can't attack. There always is a way to mis-interpret human language.
- sanderjd 10y agoThanks, this is a great response. I shall henceforth embrace the fun and swear to never again be lured by the superficial temptations of naming hegemony!
- pjmlp 10y agoHere is the Project Oberon source code, implemented in a GC enabled systems programming language. http://people.inf.ethz.ch/wirth/ProjectOberon/index.html http://people.inf.ethz.ch/wirth/ProjectOberon/index.html All the way from the hardware Verilog to the GUI. This is the 2013 re-edition of the initial project. Later versions of the operating system with its 3D Gadgets framework were quite nice to use. It doesn't get more systems programming than that.
- ustolemyname 10y agoOthers don't consider languages which do not handle OS signals a "systems" language: https://github.com/rust-lang/rfcs/issues/1368 https://github.com/rust-lang/rfcs/issues/1368
- Jonhoo 10y agoNot entirely sure what you mean here? Even Bash can handle (some) OS signals, but I don't think anyone considers that a systems language?
- ustolemyname 10y agoWell, maybe not Bash, but it seems a dozen years ago referring to Perl as a systems language would not have caused anybody to blink ;) I apologize for how glib my remark above was, it didn't add much to the conversation, and was more a knee jerk response to the notion that "Systems Language" is now, or ever has been, well defined. I brought up the handling of signals, because process management is, in practice, a pretty big deal in what I would consider systems programming. But I don't think it's important to all systems programming. Much like I don't think avoiding garbage collection is necessary for all systems programming, though it's clearly an issue for some applications. What I do find odd is the notion that it is necessary to not have garbage collection, but not having a solid story behind handling signals from the OS receives a pass. Rust can handle signals, but its implementation is very platform specific (even the currently recommended crate lacks Windows support), and ultimately calls out the FFI. So it seems that saying Rust has support for signals is like saying Go supports manual memory management because you can call C.malloc and C.free...
- kibwen 10y agoI'm unsure what this is implying. Rust is fully capable of handling signals, it just doesn't provide any nice APIs for such in the standard library.
- SiVal 10y agoIs there a roadmap for what is currently planned for the standard library? A lot of language choice these days comes down to what high-quality batteries are included and what will require either trusting some unknown 3rd party's github project or your own homemade version.
- otoburb 10y agoThe author wrote further down in the article under the heading "Performance Without Sacrifice": >>Unfortunately, higher-level languages are often not a great fit for systems code. Systems code is often performance critical (e.g., kernels, databases), so the developer wants predictable performance, and tight control over memory allocation/de-allocation and data layout. This can be hard to achieve in higher-level languages or when using a garbage collector.
- adiabatty 10y agoOne popular, reasonable definition for "systems language" is "used for writing programs that don't have a GUI" (other than web-based ones). Another popular, reasonable definition for "systems language" is "something one could write an OS in".
- unimpressive 10y agoI think the latter is a more reasonable definition than the former, since I've written plenty of command line applications in python. And as much as I like python, if it qualifies as a 'system language' than just about anything does. Perhaps it could be amended "Primarily used for writing programs without a GUI."
- Jonhoo 10y agoI'm slightly hesitant to agree with both of these. The first seems wrong to me, because Bash qualifies, whereas Tcl doesn't. The second is better, though I think it excludes some interesting languages, like Scala, which you probably wouldn't write an operating system in, but that you could certainly build "systems" in (e.g., Spark).
- samuellb 10y agoI'd call a systems programming language "something which can use the low level ABI of the underlying platform". So C and e.g. FreePascal would fit the description (for most hardware platforms). And Java, for instance, would not unless the platform runs Java bytecode.
- pjmlp 10y agoThen you are ignoring all the great work done at Xerox PARC, DEC/Olivetti, Royal Signals and Radar Establishment, ETHZ, AT&T, Microsoft Research which used garbage-collected system programming languages. If the whole OS stack can be implemented in the language, ignoring the Assembly help that even C requires, it is a systems language regardless if it has a GC for heap management or not.
- adrusi 10y agoBy that criteria neither Go nor Python can be considered systems languages. Maybe you could implement some basic code to bootstrap CPython in Cython or RPython, I'm not entirely sure, but neither of those are really Python. The only garbage collected languages that I'm aware of that can implement a full OS stack are LISP, Red and maybe Smalltalk.
- pjmlp 10y agoI wonder why you are disqualifying Go, given that it is fully boostraped as of version 1.6, yes even the GC is written in Go. It is also a descendant of Oberon, which was used very much successfully at ETHZ during the 90's. Every time I have this argument I wish some PhD student bothers to rewrite Native Oberon in Go. Also you should educate yourself, a small list of GC enabled systems programming language is: Lisp, Algol-68RS, Mesa/Cedar, Modula-2+, Modula-3, Oberon, Oberon-2, Active Oberon, Component Pascal, Sing#, SystemC#, D. There are plenty of ACM, SIGPLAN, digital archive papers from the OS written in those languages.
- nickpsecurity 10y agoAs pjmlp says, Oberon systems were implemented in a GC language. The latest, A2 Bluebottle, runs fast and light despite being written in a safe, GC, system language. Easily more responsive than my Linux box. There's also GC's that can support latencies good enough for soft, resl-time applications. Far as Go, it's designed to recreate the Oberon-2 experience of Pike. It's basically a modified Oberon. If Oberon can do OS's, then Go can do OS's with some modifications to runtime or something.
- accnumnplusone 10y agoThis discussion keeps coming up in the Go world. As with many words "systems" has multiple uses eg operating systems and enterprise systems. I don't understand the confusion.
- Animats 10y agoWell, the functional crowd won. An example expression from the parent article: let idx = args // iterate over our arguments .iter() // open each file .map(|fname| (fname.as_str(), fs::File::open(fname.as_str()))) // check for errors .map(|(fname, f)| { f.and_then(|f| Ok((fname, f))) .expect(&format!("input file {} could not be opened", fname)) }) // make a buffered reader .map(|(fname, f)| (fname, io::BufReader::new(f))) // for each file .flat_map(|(f, file)| { file // read the lines .lines() // split into words .flat_map(|line| { line.unwrap().split_whitespace() .map(|w| w.to_string()).collect::<Vec<_>>().into_iter() }) // prune duplicates .collect::<HashSet<_>>() .into_iter() // and emit inverted index entry .map(move |word| (word, f)) }) .fold(HashMap::new(), |mut idx, (word, f)| { // absorb all entries into a vector of file names per word idx.entry(word) .or_insert(Vec::new()) .push(f); Is there editor support for indenting this stuff?
- Chris2048 10y ago> // iterate over our arguments > .iter() hmmm. Not far off the old > i++ // add one to i
- tatterdemalion 10y agoThese are quite different. `.iter()` is equivalent to creating the counter and declaring both the increment and conclusion condition, only its guaranteed to correctly access each member of the collection once and to never be broken, and for many collections (e.g. arrays) it is written in a way which allows for powerful optimizations.
- Chris2048 10y agoI mean in terms of comment useful-ness.
- dikaiosune 10y agoRustfmt can clean that up, but I'm on mobile and can't run the examples through to see what it looks like.
- Jonhoo 10y agoAuthor of the post here. Curious that this got posted again. Was originally posted as https://news.ycombinator.com/item?id=11773332 https://news.ycombinator.com/item?id=11773332. Can the posts be merged by a mod somehow?
- 0xmohit 10y agoGood to see such articles that provide an insight into various aspects of a programming language. A couple of other beginner-friendly resources would include: - An alternative introduction to Rust [1] - 24 days of Rust [2] - CIS 198: Rust Programming [3] [1] http://words.steveklabnik.com/a-new-introduction-to-rust http://words.steveklabnik.com/a-new-introduction-to-rust [2] http://zsiciarz.github.io/24daysofrust/ http://zsiciarz.github.io/24daysofrust/ [3] http://cis198-2016s.github.io/ http://cis198-2016s.github.io/
- Mihies 10y agoOne thing I am missing is dependency injection/ioc. How does one effectively unit test without it?
- Jonhoo 10y agoYou can still do dependency injection in Rust..? There even exists a create for it: https://github.com/Nercury/di-rs https://github.com/Nercury/di-rs. There's also some further discussion here: https://users.rust-lang.org/t/how-do-you-implement-dependency-injection-in-rust/213 https://users.rust-lang.org/t/how-do-you-implement-dependenc...
- Mihies 10y agoI saw that but it seems really complicated for something that should be straightforward. Or it just looks so? (coming from c#)
- SideburnsOfDoom 10y agoIf I understand it right, Rust has "zero-cost abstractions" meaning that they don't exist at run-time, just when compiling. e.g. http://blog.rust-lang.org/2015/05/11/traits.html http://blog.rust-lang.org/2015/05/11/traits.html This is the right choice if you want the language to be "a better C" but it means that there is none of the Run-Time Type Information that we are used to in C# (or java). The RTTI plays a small but key role in e.g. how a DI container works: The container works out what parameters a constructor needs at runtime and instantiates those recursively; and in how a mocking framework works: Generating a class on the fly with the right interface declaration. Even XUnit/NUnit work by using RTTI: The testing framework looks for all classes/methods with the right attribute metadata, and runs them. It's easily fast enough for how we use C# and Java, but it's not zero-cost. There are unit testing tools in Rust, but they cannot work the same way - IIRC, there is compiler support for them. So, While Rust as a good chance at being "A better C", don't expect it to do all the things that C# does involving types at runtime. A different goal meant a different language design. There are other interesting things in Rust like the Macro system that might help do similar things, but it's really not going to work like in C#
- namelezz 10y ago> Once your code compiles, you’ll find (at least I have) that it is much more likely to be correct (i.e., do the right thing) than if you tried to write similar C, C++, or Go code. What issues in Go do this sentence refer to?
- seren 10y agoI don't think that Go can detect race condition errors at compile time. So it could compile and sometimes fails mysteriously. In Rust, as long as you don't use unsafe block, the compiler should detect it. This is not really a Go issue, rather that Rust is much more stringent on what it "correct" code compared to C, C++ and Go.
- gizzlon 10y agoI think you're right. Go has a race detector, but it work on run time: https://golang.org/doc/articles/race_detector.html https://golang.org/doc/articles/race_detector.html Guess you would normally run your tests with it, and maybe compile a dev / beta / RC version with race detection enabled? Never used it in prod, but it did find a problem for me.
- seren 10y agoI believe this is one of the main promise of Rust, even if it is not stated like that, if it compiles it works and it is safe. But obviously for any other language (C,Go,etc) you can use a compiler + static analysis + dynamic analysis to have the same confidence. What I like with Rust is that implicitly you know that every one has to run the equivalent of analysis tool to release/deploy the code.
- vertex-four 10y ago> I believe this is one of the main promise of Rust, even if it is not stated like that, if it compiles it works and it is safe. Not quite. There's certain types of issue which its type system won't detect - you can, in fact, easily get into a race condition, just not what's known as a data race - and there's no promise that what you've written actually does what you expect it to do, just that it'll do something without use-after-free and other memory-related bugs, and if you've written your program well it's more likely than e.g. Python that it does what you intended (in my personal experience). Getting closer to "if it compiles, it works", but still not entirely there, there'd be the likes of Idris and other systems with proofs embedded into their type systems - but even then you still have to verify that the proof matches what you're actually trying to do in the real world.
- pimeys 10y agoIn my current job, I was given a task to write a service which should handle billions of events eventually in the fastest possible way. The language choice was given for me to decide, and I was thinking that maybe I'll do it with C++14, Scala, Go or Rust. C++ has it's quirks and I'm not really enjoying it's build tool of choice, cmake. Scala I can write fast, but scaling the app in Mesos would consume lots of memory; every task would take a huge slice of RAM because of the additional JVM. Go I just don't like as a language (personal taste) and I think the GC adds a bit too much of pausing for this app, so I gave Rust a shot. The first week was madness. I'm fairly experienced developer and the Rust compiler hit me to my fingers constantly. Usually the only way out was to write the whole part again with a different architecture. On the second week I was done with my tasks, very comfortable with the language and just enjoying the tooling. Also I was relying on tests way less because of the compiler, even less than with Scala. If it compiles, it has a big chance of working. Cargo is awesome. Let me repeat: Cargo is awesome. Also I like how I can write code with Vim again and even though for some things I need to read the Rust source, it is pretty easy to get answers if you're in trouble. In the end I wrote some integration tests with Python and I'm quite happy with them. Now I want to write more Rust.
- karterk 10y agoOn the other hand, as your system gets more complex, you will start missing tried and test libraries from some of the more mature languages you have mentioned. Maybe the extra time you have to spend writing that code will compensate for the extra memory you would have spent because of the JVM. Not taking anything away from Rust - I'm just giving an alternative perspective.
- pimeys 10y agoThat is a risk we've analyzed and are willing to take. Scala can stay on the data side of our platform, but the real time part will be either Rust or C++14 (and Python where we don't need raw speed and can handle everything with one thread). For the libraries the situation is not that bad. Something I was missing from Scala side was Kamon. It is a really awesome library I've been using (and contributing) for some time. For Rust I needed to do my own solution with threads and a bit of macro love. It works, it is using a constant amount of ram and when we get our next Rust project running, I might open source that part for everybody to use...
- zimbatm 10y agoIs rust ever going to re-introduce the N:M model again ? For services which need to handle 1M connections the system threads are too expensive and mio brings back the callback hell.
- lifthrasiir 10y agoHave you tried mioco [1]? It looks fine enough to avoid the callback hell. [1] https://github.com/dpc/mioco https://github.com/dpc/mioco
- zimbatm 10y agoThanks. So no clear winner yet. Does libraries have to be re-written to support the coroutines ?
- GolDDranks 10y agoMIO is a very low-level library (as it tries to be as zero-overhead as possible), and there's multiple nicer abstractions written on top of it. Check this: https://github.com/carllerche/mio#libraries https://github.com/carllerche/mio#libraries
- eddyb 10y agomio has nothing to do with callbacks in its design, unless you build such an abstraction yourself on top of it. However, without an ergonomic way to create state machines (i.e. generators), it's hard to use at all in the intended fashion (small per-connection state instead of large boxed closures or coroutine/thread stacks).
- Jweb_Guru 10y agoRust's M:N design was not going to scale to 1M connections, and it's hugely unlikely that you need 1M simultaneous connections.
- Munksgaard 10y ago> This latter point is particularly interesting; the Rust compiler will not compile a program that has a potential race condition in it. I feel obliged to point out that this is false. Rust prevents _data races_, but not _race conditions_. You can read more in the Rustonomicon here: https://doc.rust-lang.org/nomicon/races.html https://doc.rust-lang.org/nomicon/races.html
- Jonhoo 10y agoIn my defense, the next sentence is "Unless you explicitly mark your code as `unsafe`, your code simply cannot have data races." That said, I've updated the article text to now say data races in both places.
- loup-vaillant 10y ago> Rust is not the most beginner-friendly language out there — the compiler is not as lenient and forgiving as that of most other languages […], and will regularly reject your code […]. This creates a relatively high barrier to entry […]. In particular, Rust’s “catch bugs at compile time” mentality means that you often do not see partial progress — either your program doesn’t compile, or it runs and does the right thing. […] it can make it harder to learn by doing than in other, less strict languages. I don't see how making the type system stricter makes the language harder to learn. Maybe that's because I know another relatively paranoid type system (Ocaml), but still. A type system that rejects your code is like a teacher looking at a proof you just wrote, and tells you "this doesn't even make sense, and here's why". It may be frustrating, but this kind of feedback loop is tighter than what you would get from a REPL. And you do see partial progress: the type errors change and occur further in the source code as you correct your program. Each error is an opportunity to fix a typo or a misconception. The distinction between a broken prototype that doesn't even compile and a working program isn't binary: when you correct a type error, your program is less broken, even though it doesn't compile yet.
- bjz_ 10y agoThis is a perspective that I think we could do better at advocating for when it comes to static typing. It's not really just about safety, it's also about being able to rapidly figure out a good model for your problem through a conversation with the type system. It will not hesitate to let you know when you don't make sense, or you haven't explained things correctly!
- IshKebab 10y agoBecause a lot of the time it does make sense and the borrow checker just isn't clever enough to know. For example: https://github.com/rust-lang/rfcs/issues/811 https://github.com/rust-lang/rfcs/issues/811
- loup-vaillant 10y agoThen the borrow checker is just flawed. I know, I know, no non-trivial checker can be both sound and complete. But if too many useful programs end up being rejected because of that gap, the type system must be changed.
- deleted 10y ago[deleted]
- shmerl 10y agoThis is also a pending issue: https://github.com/rust-lang/rfcs/issues/349 https://github.com/rust-lang/rfcs/issues/349
- jeffdavis 10y agoI really like my experience with rust so far also, but a few caveats: * try!() Is pretty annoying * Working effectively with C in non-lexical ways seems to involve some unstable libraries and still requires nightly rust * Macros are safer, but can't do some things that C macros can. For instance, they are hygienic, which means you can't conjure up new identifiers. For that, you need a syntax plugin, which is very powerful but the APIs aren't stable yet. This goes to the previous point. * A few annoyances, like warning when you don't use a struct field as "dead code". If I'm interfacing with C I probably need that struct field whether the rust compiler sees it or not, but I don't want to disable all dead code warnings for that.
- tatterdemalion 10y agoYou can attach the allow(dead_code) attribute to the struct - or even the individual field - which will scope it to that declaration only. Still annoying when you have a lot of them, but better in my opinion than not having dead code warnings on fields, since the use case you describe is the minority of Rust structs that are declared.
- jeffdavis 10y agoMaybe there should be an annotation saying that the layout of a struct is important for ABI compatibility, and it would silence warnings related to the layout.
- tatterdemalion 10y agoC FFI structs have to be marked `#[repr(C)]` to guarantee they'll be represented how C structs would be. It might be a reasonable change to implicitly allow dead fields on any struct with a C repr (since using the struct in C effectively makes all of its fields public).
- kjaleshire 10y agoThe `?` operand appended to function calls makes them behave like `try!()`. It's only in nightly currently, on its way to being stabilized.
- georgewsinger 10y agoIf I'm not hacking on something super low-level, like hardware or an OS, then should I still try Rust? Why not stay within super high-level/expressive programming languages like Haskell/clojure? I ask because a lot of extremely smart people I know like Rust.
- steveklabnik 10y agoRust still tries to be expressive, even with its low-level nature. I have heard of people saying that they have started writing Rust instead of Python, because they feel like it's close enough in expressiveness most of the time, yet much faster.
- nercury 10y agoRust is good. Great tooling: Libraries can be written, published and used quickly. Code can be easily rebuilt on upcoming Rust versions without uninstalling anything. Integrated testing and documentation generation. Cross-compilation. Truly cross-platform. As an example, Rust even supports both GNU and MSVC targets on Windows, that means Rust libraries can be linked into the C++ projects compiled with MINGW or MSVC. All the standard library features are cross-platform, unless namespaced otherwise. Rust linear type system requires variables be either uniquely mutable or immutable, but not both. This solves whole class of resource management problems (memory being the most important) in the simplest possible way. This compile-time guarantee also works for all references to data, which become thin pointers at runtime. This, together with Send-able and Sync-able types, also ensures no data races. Language is up-to-date. No-nulls, pattern matching, closures, generics, attributes will feel familiar to programmers coming from various languages. Language had many iterations and changes, and every bit of it was designed with care. Right now it may be a bit explicit, but sugar can always be added later if it appears worth it. Stable release cycle, stability guarantee. We know when new beta will be renamed to stable and we know what's in it! That means there is time to check all ecosystem for possible breakage, even though historically Rust introduced hardly any breaking changes since 1.0 a year ago.