6 ms·
That is very exciting. I like the direction the project is going. But, one thing worries me -- performance. Anyone know a list or tracker of benchmarks to see
by rdtsc 12y ago
That is very exciting. I like the direction the project is going.
But, one thing worries me -- performance. Anyone know a list or tracker of benchmarks to see how it roughly compares.
Or is it too ealry too talk about it? Am I pointing to the elephant in the room? I am ok with saying "we are not there to talk about it, stop stirring the pot". I understand that.
But looking at the future, if Rust is to compete with C++, it needs to perform close to it. Not necessarily in the "safer"/default mode, but it needs to have an unsafe, turn-off-all-checks-run as fast as you can mode.
C++11/C++14 improved C++ many enjoy and like using it. Many that don't are probably using it for performance reasons. That is a niche Rust will have to compete in.
- cmrx64 12y agoRust's performance is competitive with clang and improving, even in the "safer" mode. We have benchmarks in many of the libraries (search for #[bench]) and some in src/test/bench. We're on the benchmarking game, but that is really just a game, not indicative of anything besides how much time someone has spent microoptimizing a specific program. Someone is working on a custom LLVM pass that will remove essentially every null check that Rust adds. On top of that, there are still a few more changes on the way that aren't easy, but should give a nice perf improvement.
- rdtsc 12y ago> but that is really just a game, Very much so, agreed. It is a PR game most of all. It is unfortunate. But it is probably a game that needs to be played. I know microbenchmarks are bogus, but I still click and look at them.
- coldtea 12y ago>But it is probably a game that needs to be played. Not really. The language is supposed to catter to professional programmers, not script kiddies.
- Argorak 12y agoHate to break it to you, but most professional programmers have no deep knowledge of: * runtime systems * detailed memory management * algorithmic complexity Yet, quite a lot of those build parts of huge systems that work and meet business requirements all the time, which makes them very desirable professionals. Being able to competently benchmark is a fringe skill.
- coldtea 12y agoSure -- but I'd call them "employed programmers" (as opposed to "professional"). In any case, I don't think Rust's target group doesn't know about runtime systems, detailed memory management and algorithmic complexity. After all it's mostly if you care about all three that you'll consider Rust.
- Argorak 12y ago"professionals" are, by definition, people that are employed in a profession.
- bad_user 12y agoSorry, downvoted you by mistake. Wanted to upvote, because yes, that's the definition of "professional", contrary to an "amateur" which is somebody engaged in a certain activity without monetary compensation (for the love of it only).
- coldtea 12y agoSorry, but "by definition" there are several meanings in the dictionary. Merely employed is the lower level. And I think I made the way I mean it clear. E.g From dictionary.reference.com: noun: - a person who is expert at his or her work: You can tell by her comments that this editor is a real professional. - a person who engages in an activity with great competence adj: - extremely competent in a job, etc - (of a piece of work or anything performed) produced with competence or skill
- sanxiyn 12y agoHere is a link to the source code of the initial version of the custom LLVM pass to eliminate null checks. Hopefully this will be upstreamed to LLVM in the future. https://github.com/rust-lang/llvm/blob/rust-llvm-2014-06-28/lib/Transforms/Scalar/NullCheckElimination.cpp https://github.com/rust-lang/llvm/blob/rust-llvm-2014-06-28/...
- legulere 12y agoThere's a small typo in there: argumetns -> arguments
- semisight 12y agoThere technically is an "unsafe" mode, but not in the way you're thinking. You can drop down to C level by writing code in unsafe blocks and using methods prefixed with "unsafe." This will stop the compiler from doing things like inserting bounds checking and should perform fairly close to Clang. Generally though, rust strives for the same ideal as C++: you only get what you pay for. You can carve out the subsection of the language you need through things like feature gating without too much trouble right now (and the standard library is still improving in this regard).
- diaz 12y agoJust a quick reply to link to this answer on reddit with the question: Does 'unsafe' disable array bound checking? http://www.reddit.com/r/rust/comments/2aqqxa/does_unsafe_disable_array_bound_checking/ http://www.reddit.com/r/rust/comments/2aqqxa/does_unsafe_dis...
- pjmlp 12y agoOh, the open door to security exploits that C developers love so much. When Hoare designed Algol(1960) he was explicitly requested not to allow it, due to security concerns[0]. [0] "The 1980 ACM Turing Award Lecture",
- djur 12y agoAs I understand it, the goal is for Rust to perform as well or better than C++ without the need for an unsafe mode. From the long discussions I've seen on the Rust mailing list, I think the likelihood of a "disable all checks" mode is nil.
- BlackMonday 12y agoYes, idiomatic C++ which uses the safety features that C++ has to offer, as far as I understand. Not the raw performance that C++ can give you. Just wanted to point that out. Source: http://pcwalton.blogspot.de/2010/12/c-design-goals-in-context-of-rust.html http://pcwalton.blogspot.de/2010/12/c-design-goals-in-contex...
- dbaupp 12y ago> Not necessarily in the "safer"/default mode, but it needs to have an unsafe, turn-off-all-checks-run as fast as you can mode. By design, Rust allows you to opt-in to this mode locally via `unsafe { ... }`; this allows you to be 100% safe through the vast majority of the code base, but then eek out the very last drop of performance in core routines where there's no safe alternative. Then only those regions need to be checked for memory safety vulnerabilities rather than the whole codebase. Although, many things can be written in safe code and still be performant: iterators provide safe and highly efficient sequential access to vectors and arrays (right now they optimise to the same code as the equivalent C/C++, other a bug in LLVM where it's not recognising that some pointers are guaranteed to be nonnull).
- masklinn 12y ago> By design, Rust allows you to opt-in to this mode locally via `unsafe { ... }`; It should be noted that contrary to e.g. Swift's -Ofast, Rust's unsafe {} does not change language semantics. It allows operations marked as unsafe (non-bounds-checked array accesses, raw pointer and pointer munging, etc…) but operations usable outside unsafe {} have the exact same semantics inside.
- valarauca1 12y agoCurrently Rust's perfmance is about that of Java's. Which is and isn't in contention with C++. Something's yes, some no. Rust beats C++ in a few areas on average (tree look ups for example) but that's about it. It's not to early for it to be a concern. It should be a concern in system languages aiming to compete with lanagues like C/C++/Fortran. But I feel rust isn't in a terrible competition for being at least 15 years late to the party.
- cmrx64 12y agoDo you have specific benchmarks to back up this claim? I think it's completely unfounded. Every test we've done against C and C++, we're competitive. Exception is the performance of the IO/task runtime, which still needs work, but is unrelated to the language.
- dbaupp 12y agoThere's also the factor of immaturity: the language is young and changing, so it hasn't really been possible/sane to spend the effort writing high-performance micro-optimised libraries for various tasks.
- sanderjd 12y agoAre the IO/task runtime tests you've seen available to look at somewhere?
- steveklabnik 12y agoI'm thinking of https://github.com/rust-lang/rust/issues/15363 https://github.com/rust-lang/rust/issues/15363 as one example.
- sanderjd 12y agoGreat, thanks.
- steveklabnik 12y ago> it needs to have an unsafe, turn-off-all-checks-run as fast as you can mode This will never happen, as it is contrary to the goals of the project. As mentioned by others, locally, you can ignore some checks, but there won't be a compiler flag to turn them off. In some ways, it's too early to talk about it, and in some ways, it's not. It's not too early to talk about decisions which would place a certain upper bound on performance, but it's kinda too early to do some kinds of direct comparisons. The team has not really been working on performance generally, as the focus is to get the language semantics correctly. There's lots of places, for example, where rustc generates subpar LLVM IR, and it's a matter of someone writing a patch.
- andrewchambers 12y agoWell, Rust uses llvm, so unless rustc is generating really poor intermediate code, its performance will be the same as clang C/C++.
- Jweb_Guru 12y agoThat isn't quite true. Swift will be using LLVM as a code generator, but because it will use ARC (a form of garbage collection), it will still end up slower than C++ in many benchmarks. That's not the only example, of course--while LLVM helps immensely, it's not a miracle worker and the original language semantics do make a difference.
- pjmlp 12y agoThis is not true, as Swift ARC is supported by the compiler itself, whereas C++ ARC are library based without any compiler supporter. So a Swift compiler will be able to optimize away ARC calls deemed unnecessary via dataflow optimizations, which a C++ cannot apply.
- dbaupp 12y agoI think C++ give you many more tools to avoid using "ARCs", e.g other smart pointers, not needing smart pointers at all etc.
- pjmlp 12y agoNot using smart pointers is "memory leaks and security exploits" land. RAII is great, but not when dealing with shared data structures. So I lack to see how it can be better. Note that I am a big C++ fan since the C++ARM days, but I am also aware of many of its warts.
- dbaupp 12y agoI just meant you don't need necessarily need pointers at all, since you put things directly onto the stack (or into vectors/your favourite data structure), no need for additional indirection. (Maybe Swift offers this too, but it's certainly something C++ excels at.)
- coldtea 12y ago>But, one thing worries me -- performance. Anyone know a list or tracker of benchmarks to see how it roughly compares. Or is it too ealry too talk about it? Am I pointing to the elephant in the room? I'd say it's too early. First, Rust is a compiled, statically typed languaged using LLVM, including stuff such as non aliasing guarantess over C, and designed to be secure AND fast. It's not like it's some scripting language. Second, you should benchmark your program first. Worrying about a language's "performance" in abstract is more premature than even Knuth anticipated.
- nirvdrum 12y agoA fast language runtime helps all programs. If it's slow, there's precious little you can do about it. And the design of the language may end up such that certain aspects are just always going to be slow. So, I very much disagree with the notion that this a premature optimization (and I'm starting to think that phrase needs to just go away). You seem to allude to as much by delineating between compiled and scripting languages.
- Jweb_Guru 12y agoYou can run Rust completely without a runtime. Part of the reason for that is that there are much bigger performance hawks than you or me monitoring Rust who have significant input on its implementation. That is one of the reasons that Rust moved GC into a library, for example. The last thing I am worried about with Rust is its performance.
- nirvdrum 12y agoAwesome. Rust's performance is something that impresses me, so I'm certainly not suggesting that it's slow. The point I was making is it's a valid question to ask and a valid thing to measure. Having fought with slow language features (e.g., blocks in Ruby), slow stdlib (e.g., older versions of Scala's collections), and slow techniques adopted by library authors (e.g., ask for forgiveness with exceptions instead of permission) there's just a lot that impacts many if not all apps and if it's outside your own application code, it can be very hard to fix. One of the things I really like about go releases is they indicate exactly what's changed on the performance front, so I have a good idea of what to expect. And since they measure it, they can improve it.
- DCKing 12y agoIt's the job of the compiler to optimize out things it knows are secure. If an array lookup is known to be secure at compile time, then LLVM already optimizes out bounds checking without the need of a fast mode [1]. If there is not enough evidence available to the compiler that a particular array lookup is not safe, then that evidence is also not available to the programmer. Therefore, one would have to write a manual bounds check anyway. Double bounds checking, one by the compiler and one by the compiler, can also be optimized out. Maybe Rust is not capable of completely covering all optimization cases right now, but I don't think there's a need for a turn-off-all-checks mode. Does anybody have any material about how a turn-off-all-checks-mode improved performance for another language? I'm curious about what the difference is. [1]: https://github.com/rust-lang/rust/issues/9024 https://github.com/rust-lang/rust/issues/9024
- gsg 12y ago> If there is not enough evidence available to the compiler that a particular array lookup is not safe, then that evidence is also not available to the programmer. That's quite wrong. Humans are capable of deductions that compilers, which must necessarily use conservative, constructive methods, can't dream of. They can also have information that isn't available to a compiler even in principle, although this is less likely to apply to eliminating bounds checking.
- DCKing 12y agoIt's worded quite strongly, but the programmer can only be certain that a certain bounds check is guaranteed to succeed if the compiler agrees. In other situations something about the current context may allow a bounds check to pass, but the "current context" is always bound to change in any sufficiently complicated program.
- pjmlp 12y agoYes, and programmers on a development team change with time, taking their knowledge with them. So the next guy touching the code just makes the code blow up, due to wrong assumptions.