17 ms·
What Rust Can Do That Other Languages Can't
- Yan_Coutinho 10y agoRust seems to be a good language for game development. I watched this guy (https://www.liveedu.tv/gexon/videos/evAbX-dotakiller-gamedev-indiedev-onedev-rust-42 https://www.liveedu.tv/gexon/videos/evAbX-dotakiller-gamedev...) and it made me want to try it.
- akie 10y agoI have no idea what this means, and I have a masters in Computer Science and 20 years of industry experience. Enlighten me?
- tatterdemalion 10y agoThis is not a great way to ask a question. Your degrees and years of experience are not relevant to understanding the subject matter of this post. In order to enlighten you, we'd need to know what specifically was confusing about this article to you. What did you not understand?
- HerraBRE 10y agoI think it was actually a nice nod to all the folks who haven't got that much experience, and maybe felt silly for not understanding the code snippet. :)
- rubber_duck 10y agoIn C++ it's quite easy to get burned by keeping a reference to something where the owner went out of scope eg. class Foo { int bar; int& getBar() { return bar; } } if you take ref of bar and Foo goes out of scope well fuck. It gets even worse, eg. : vector<int> foo; int& bar = foo[i]; foo.push_back(1) that last line can cause foo to reallocate the buffer and bar is then invalid, meanwhile that code will compile without warnings (well I'm guessing it will I haven't done C++ in a year :) )
- akie 10y agoThank you! That clarifies it quite a bit. My C/C++ is too rusty to remember these internals...
- saghm 10y ago> My C/C++ is too rusty Pun intended?
- satbyy 10y agoJust hit 'Run' here: http://rextester.com/BDGO47890 http://rextester.com/BDGO47890 You can see the program crash
- rubber_duck 10y agoNice, but the example crashes because you take a reference to uninitialized vector, if you initialize it and grow the vector I found that it will either keep "working fine" if you don't expect the data to change or randomly give garbage data - depending on compiler/stdlib - especially since the time when the buffer grows is implementation defined. Here is a better example : http://rextester.com/MAJDQ8375 http://rextester.com/MAJDQ8375 On VC++ it will return : Reference is 0 but value is 9 On clang it will return : Reference is 32579632 but value is 9 So let's say I wasn't mutating the 0 index, VC example would work fine at runtime and then clang would suddenly start giving garbage results. This has caused me week of debugging in a big code base.
- satbyy 10y agoThanks for the catch!
- orf 10y agoI'd hope someone with that much experience would be able to grok at least some of it? I guess it isn't explained that well though. As I understand it, if you have a structure that has a pointer to something (like a User struct pointing to an array of permissions or some such), and the memory pointed to will be freed when the structure is freed. There is a case where code can keep a reference to this pointer around longer than the user struct it belongs to, so when code tries to use it it references an invalid pointer. This is bad. Rust takes care of this in some clever ways, so you cant have a pointer/reference that outlives it's parent.
- tatterdemalion 10y agoYou've missed a critical point in the article: the structure does not have a pointer to something it has a member inline within its stack representation. Any garbage collector could (with some overhead) prevent invalid memory access when we're talking about an object with a pointer to another object. Rust is able to validate the correctness of pointers to offsets within the stack representation of an object. (Rust also validates the correctness of references to heap allocated objects as well, of course!)
- m1el 10y agoLet's translate this code to C++: class X { public: Y* getY(void) { return &y; } private: Y y; }; } `getY` returns a pointer to the struct field, this field is allocated as a part of the struct. There is no way (in C++) to verify that this pointer does not outlive the struct itself. In other words, your pointer may become stale, and the compiler has no way to check for it. Rust has a resource ownership model (borrow checker) that allows it to check at compile time that references do not outlive the data.
- akie 10y agoAh, I see! Thank you.
- humanrebar 10y agoA little closer would be to use a shared_ptr<X> and shared_ptr<Y> that share the same reference count (1). It's not exactly the same thing, though, since Rust gives you some safe usage checks that you don't get with shared_ptr. That is, vanilla Rust objects have lifetime semantics closer to unique_ptr than shared_ptr. (1): See constructor number 7 here, which was designed for this purpose: http://en.cppreference.com/w/cpp/memory/shared_ptr/shared_ptr http://en.cppreference.com/w/cpp/memory/shared_ptr/shared_pt...
- dbaupp 10y agoIt also isn't the same in that it comes with the cost of allocation and reference counting.
- nallerooth 10y agoThanks!
- blub 10y agoBoth the original example and your translation exhibit poor API design: it's an antipattern to return mutable references to member variables. This would remove all encapsulation, which is probably the most important principle of OOP. Nevertheless, a better solution was pointed out in an answer on the blog and by humanrebar.
- usrusr 10y agoIt's about safely exposing innards. Are you are familiar with the substring problem in Java? Before 1.7.0_06, substrings where created using an offset into the buffer of the original string. Nice O(1) substring creation, the textbook example for benefits of immutability. Unfortunately, if you have long-lived, but short substrings taken out of short-lived, but long parent strings, the garbage collector won't find much to collect, since the substrings keep references to their parent buffers. This was no showstopper, but without immutability or in an environment with unmanaged memory it might very well be. This Rust snippet is about similar cases of subcomponent sharing, just on a shorter timescale: the subcomponent is exposed in a way that allows the compiler to enforce that nobody keeps a reference for longer than it is guaranteed to be valid. You may look, you may even touch, but if you want to look again you have to ask again. Other languages get by without this by garbage collection, by excessive precautionary copying or by simply hoping that the caller won't do anything stupid. Rust is unique in achieving the same runtime behavior as the latter but without the uncertainty. As a bonus, those un-keepable references cannot escape across concurrency borders so that in-place mutability becomes a lot more manageable.
- mnw21cam 10y ago> What Rust Can Do That Other Languages Can't But other languages are Turing-complete too...
- Wyverald 10y agoI'm sure you understand that not all language features fall under Turing-completeness.
- deleted 10y ago[deleted]
- amelius 10y agoYes, but Rust might still be able to do what other languages can't. For example, to express a given program using at most N input symbols.
- wtallis 10y agoPerhaps more relevantly, we can talk about the relative sizes of the sets of invalid Rust programs that rustc will fail to reject, and invalid C programs that gcc will fail to reject. The halting problem tells us that neither set is empty, but there's still a lot of room for differentiation. (Actually, upon further thought, it may be that rustc rejects all invalid Rust programs and some valid ones. gcc certainly accepts some invalid C programs.)
- tatterdemalion 10y agoMy rough approximation is that the difference between a sound type system and an unsound type system is that a sound system rejects some valid programs and unsound system accepts some invalid ones.
- adrianN 10y agoBrainfuck is Turing complete too, but it's not Tetris complete. Turing completeness is a statement about computing functions on the natural numbers. It's not terribly useful when applied to real programming languages.
- qznc 10y agoD is working on it, if I understand DIP1000 [0] correctly. The example would look like: struct X { Y y; scope ref Y getY() { return y; } } [0] https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md
- ngrilly 10y ago> Most other languages simply prevent you from giving away an interior reference, or require y to refer to a distinct heap object from the X. Go lets you return an interior reference. But Go uses a garbage collector instead of lifetime checks at compile time (which is an advantage or a drawback, depending on your requirements).
- tatterdemalion 10y agoI don't know much about the Go compiler internals, but it sounds like in most cases (possibly all) the object being referenced in that way will be heap allocated; the entire point of this article is that no matter what (across function boundaries, the reference itself put into the heap, whatever) the member `y` of `X` will be inline with its stack representation. https://golang.org/doc/faq#stack_or_heap https://golang.org/doc/faq#stack_or_heap
- topspin 10y agoGo does escape analysis like most (all?) good GC implementations to detect optimization opportunities. You can allocate an array of objects, for instance, and Go will try to make a contiguous, dense allocation for all the members, enabling extremely efficient pointer arithmetic to resolve member offsets thus avoiding an indirection, and precluding the GC bookkeeping for individual elements, and perhaps improving processor cache utilization. But the key point stands wrt Go; Go is among the languages that "can't do this." First, there is no way to get a reference to a field inside a struct in Go (the workaround is reflection which has runtime overhead, and therefore doesn't count) and if you could get such a reference the compiler would be forced to reference count the member via the GC to satisfy the Go memory model.
- EdiX 10y ago> First, there is no way to get a reference to a field inside a struct in Go what? https://play.golang.org/p/1OJ5U5ZwFJ https://play.golang.org/p/1OJ5U5ZwFJ > if you could get such a reference the compiler would be forced to reference count the member via the GC to satisfy the Go memory model no reference counting in go.
- ed_blackburn 10y agoI spent an afternoon messing about with Rust and found it infuriating. The compiler kept (quite rightly) telling me how crap my code was and wouldn't compile it for me. I found it frustrating but thoroughly educational. For systems programmers, Rust looks a fantastic option. For line of business apps, it is inaccessible and the safety dial is turned up too far. But then I guess that is because Rust isn't intended to be a general purpose language for writing line of business apps? I can see it being useful for writing platform neutral, gnarly code and offer easy hooks for popular, managed languages such as JavaSript, C#, Java et al to hook into. Outside of systems programming, I wonder if Rust will be part of a silent revolution?
- tatterdemalion 10y agoI am a volunteer who works on Rust & I do almost all of my open source work in Rust. I am paid as a Ruby web developer. I personally have little interest in systems programming & want Rust to be a productive language for application development. I frequently witness this rush to judge Rust as too complex to be used for business applications based on a few hours of hacking. For this to be a valid judgment, I would think one of two things would need to be true: a) Rust remains this difficult to use once you have experience with it, or b) most business applications are worked on for only a few hours. In my experience, neither of those premises hold. I believe that the rules of Rust not only give you exceptional performance without memory issues (as this article tries to demonstrate), but also that its rules engender a higher degree of correctness and 'well-factoredness' than most languages. I believe Rust is an excellent language for writing applications at every level of the stack when 'craftmanship' matters.
- di4na 10y agoOr ... Business applications do not care about safety.
- shadowmint 10y agoIt's probably more a case that safety is critically important, but most LOB apps do not have particularly strict performance requirements. ie. java is good enough.
- shadowmint 10y agoI don't think this is a particularly useful example of what rust can do. Other languages can return inner pointers; its not an interesting feature. The only interesting thing is that it is both safe and has no gc... but really, this is just another example of safety in rust, via a relatively obscure code snippet. If you want to pitch rusts safety features, we can do something a bit more interesting, surely?
- roca 10y agoMost safe languages are unable to return inner references. Heap-allocating the inner object doesn't count; object inlining is important for systems programming. You're absolutely right that there are much more interesting examples, e.g. https://t.co/HJV7G4n0PD https://t.co/HJV7G4n0PD. Doesn't fit in six lines though :-)
- dorianm 10y agoOther languages can definitely do it: Ruby: class X attr_accessor :y end Crystal (full working example): class Y end class X def initialize(y : Y) @y = y end def y @y end end y = Y.new x = X.new(y: y) puts x.y
- willvarfar 10y agoThe article is not about returning a reference to a struct member, its about returning a reference to something on the stack rather than heap.
- imron 10y agoAnd crucially, enforcing that no references to the returned member will outlive the containing struct.
- topspin 10y agoRust ownership applies to heap allocated objects as well; if you return a reference to a member of a heap allocated struct that reference cannot outlive the struct allocation. The point of the article is more general than "stack vs heap;" other languages are either incapable of expressing a reference to a member of a heap allocated struct or incapable of ensuring the memory safety of such a reference.
- willvarfar 10y agoAs the article notes, garbage-collected languages can ensure the memory safety of a pointer into a heap object.
- TuringTest 10y agoBut not at compile time, which is what Rust provides.
- qznc 10y agoAny language where the X thing is managed by a garbage collector (and you cannot manually delete it anyways) is just as safe as Rust. What only Rust can do is to check your manually memory-managed stuff for correctness (conservatively).
- sampo 10y ago> pointer addition Isn't it a bit confusing to talk about pointer addition when the example struct is just struct X { y: Y } and thus the addresses &x and &y are (probably) the same. Maybe the author thinks that &y is obtained as &x+0, but.
- Manishearth 10y agoThe author is generalizing to when there are more fields here.
- kutkloon7 10y agoC can do exactly the same, it just won't perform the lifetime checks for you. I haven't written much Rust, but if I understand correctly you can write code that is valid, but won't compile because you're not 'following the rules' of the constraints/type checker. I find Dafny to be a more elegant and nice solution, since this language allows you to actually prove your code is valid when it's not obvious from the static analysis. But this might be because I'm somewhat more mathematically inclined (and i admit that Dafny is still by no means easy to use)
- zrm 10y agoCan anyone explain the reason why C or C++ compilers can't do this? Obviously the language specs allow you to do the unsafe thing, but suppose we add some "-Wreference-lifetime" flag to gcc that warns if it can't statically verify that a reference or pointer doesn't outlive the referenced object, and then compile everything with "-Wreference-lifetime -Werror" from now on. What aspect of the language in particular makes that impossible? Or is it?
- simias 10y agoMy guess would be that in order to be safe you'd also have to seriously constrain other aspects of the language (like casts for instance). You'd probably have to introduce something like rust's "unsafe" to turn those checks off locally. You'd have to add additional syntax to manage lifetimes. Basically you'd reimplement Rust in C++. 99% of the C++ code out there wouldn't compile anymore and a lot of it would require significant re-architecturing in order to work with the new paradigm. Or more likely the vast majority of the codebase would keep working in "legacy" mode which wouldn't play nice with the safe portions of the code (since you'd have to consider all calls to the rest of the code as unsafe). Same thing with any external library not updated to be "safe". It's like the python 2 -> 3 transition, except a lot worse because it's significantly more complicated and probably can't be automated easily. The reason C++ was so successful is because of its backward compatibility (all the way back to C). I'm not sure it'd make a lot of sense to introduce such radical changes now. Especially given how incredibly huge the language is already.
- m1el 10y agoIn principle, nothing stops C++ compilers from implementing this feature. However, this - requires additional input from the programmer, such as lifetime parameters in Rust. - C and C++ don't have ADTs which help working with lifetimes A LOT (e.g. having Option<Box<T>> as nullable pointer). - will reject 99.9% of useful libraries and programs written in C. - would be equivalent to using Rust.
- lmm 10y agoFor the record C++17 does introduce std::variant. It's horribly cumbersome but it does at least exist.
- amelius 10y agoThere's one area where Rust's typesystem will probably be counterproductive, and that's flexible and efficient graph algorithms. (Someone please prove me wrong).
- rkangel 10y agoWhich is where the 'get out of jail' card in the form of 'unsafe' code is useful. You can always fall back to not having those guarantees checked if you're writing core, low level, well tested code. You can then ensure the right guarantees are made by typing at the interface, and ensure the code is used correctly. This is why writing data structures in Rust is considered an 'advanced' topic.
- bsaul 10y agoThis last sentence sounds a bit worrying. Writing data structures such as graph is just as common as writing if then else... if this is an advanced topic, then maybe an intermediate library providing common patterns for building elaborate structure would be welcome ( that's usually what stdlib types are about, but with the borrow checker, maybe those aren't enough).
- rkangel 10y agoIn languages like C it is quite common. In languages like Rust where there is a greater power of abstraction, it is less common as there exist versions of structures that wouldn't implementable in a re-usable way in C. Think of some examples you've had to implement in the past - you'll likely find implementations either in the stdlib or in cargo. Yes, you will on occasion need to implement your own datastructure to meet a particular need. If you do that you're going to have to sit down and think about it, work out the guarantees, ensure that it doesn't leak etc.. All the things you'd have to do in C, but you have to do it rarely. There are some intermediate libraries (things like 'Cell') that allow for writing structures that wouldn't otherwise be possible.
- burntsushi 10y agoFor graphs, you can use petgraph: https://docs.rs/petgraph https://docs.rs/petgraph
- WalterBright 10y agoIn D, the example looks like (if I understood it correctly): struct X { Y y; ref Y getY() return { return y; } } This tells the compiler that getY() returns a referenced based on the implicit 'this' reference. It's available now (DIP25 was the proposal for it). We're now implementing DIP1000, which adds similar support for pointer values (DIP25 only dealt with references). To see it in action: alias Y = int; struct X { Y y; ref Y getY() return { return y; } } ref Y foo() { X x; return x.getY(); } Compiling it yields: test.d(11): Error: escaping reference to local variable x
- tatterdemalion 10y agoAll due respect, but I would be extremely surprised to learn that this feature is as capable as Rust's lifetime system. Whiel you've shown that you can't have an escaping reference to a local variable, what Rust provides goes far beyond that. You can return the references to Y (for example if your function has a reference to X as an argument), you can store them in the heap, you can reassign them to be references to different Ys, etc, all so long as the Ys they point to live longer than they do.
- WalterBright 10y ago> I would be extremely surprised to learn that this feature is as capable as Rust's lifetime system. So would I. It isn't. But we're working on much more than this in order to provide guaranteed memory safety (although using a very different approach than Rust's). It's just that Rust does not stand alone with this particular example.
- tatterdemalion 10y agoI think a key fact about this example is that `&Y` is just another type of value which can be used like any other; if you can't return it our put it in a vector or whatever its really not comparable.
- roca 10y agoThat looks interesting. Looks a lot like the C++ Standard Guidelines lifetime proposal. But it doesn't seem to consider invalidation of references via mutation. For example, could you write a growable array implementation that returns references to its elements, but ensure that those references are not allowed to persist across operations that grow (and hence reallocate) the array? Rust can do this, but I don't think you'll be able to unless you introduce a distinction between mutable and immutable references and integrate it into your safety analysis.
- augustk 10y agoIn what situation would you want to do this?
- ekidd 10y agoThis like this are extremely common in performance-critical code. In this case, you want to: 1. Allocate both X and Y in the same block of memory, maybe on the stack, perhaps in a heap. In inner loops, making calls to malloc is pure performance poison. Most common garbage collected languages are completely unable to do this, with the admirable exception of the C# and the other .NET languages (and possibly some others). 2. You want to access Y by reference, so that you can work with it without needing to make a copy. Again, this is very common in performance-critical code. You can do this in C, C++ and other existing "systems" languages. 3. You don't want to accidentally keep using Y once X (and hence the underlying storage for Y) is destroyed. This is a subtle and vile bug that can memory corruption, once-a-week crashes on production, and week-long debugging sessions, among other headaches. This is where C and C++ fail, and where Rust nails it.
- rbehrends 10y ago> In inner loops, making calls to malloc is pure performance poison. Most common garbage collected languages are completely unable to do this, with the admirable exception of the C# and the other .NET languages (and possibly some others). Modern GCs use bump allocators and (ideally) combine allocations of objects and their subobjects where that is possible.
- ekidd 10y agoEven with a bump allocator, you're still putting large amounts of data into the nursery generation. If you're in a performance critical inner loop (for each line of a 50 gigabyte CSV file, or for each scan-line in an image codec), this is still a bad idea, performance-wise. You still can't beat static stack allocation, where you increment a pointer once to create a stack frame, and decrement it to free memory. In really fast Rust code, my goal is usually to entirely eliminate all heap allocation in favor of zero-copy parsers. This can be tricky—especially for streaming I/O using buffers, because a single chunk of input might get split over two buffers—but Rust's borrow checker makes it possible to maintain references into other people's buffers without shooting yourself in the foot.
- digi_owl 10y agoSeeing the comments here i fear that Rust will end up binned alongside the likes of Ada. Because people want to write code that runs and thats it, correctness and safety be damned...
- falcolas 10y agoSo long as correct programs are incorrectly rejected, and it takes extra time to fix those mis-matches, many people will simply avoid the pain in the first place. At the end of the day, many of us are paid to ship features. For many industries, correctness and safety just can't compete with new features. It's the reason Lisp, Python, Ruby, and Go are so popular - you can quickly write programs which are both fast enough and memory safe. The gaming industry, one of the bigger C++ consumers, will also probably not move to Rust for similar reasons: hitting a schedule is much more important than not crashing or not having memory leaks. You can always patch a game.
- ccostes 10y agoThe counter-argument to this being that the safety checks imposed by Rust prevent errors in the first place, saving debugging time and getting features out more quickly.
- falcolas 10y ago> the safety checks imposed by Rust prevent errors in the first place Not all errors, only a particular class of memory errors. Most of those memory errors are those that a GCed language doesn't have to worry about, and C/C++ have a bevy of tools to find those errors before they leave the developer's hands as well. That said, Rust's borrow checker will also help protect against a class of shared memory mutation errors as well. Is the time cost of pleasing the borrow checker for every memory allocation worth this benefit? Probably something only individual developers can answer. My answer is, for now, no.
- wyldfire 10y agoThere's a real world financial consequence to security defects. That will definitely attract folks who want to make good business decisions. I agree, "people want to write code that runs and thats it" but there's a subset of folks who like to write their code in C/C++ and crank up the warnings, turn on ASan/UBSan, valgrind+electric fence, etc. To be successful, Rust doesn't have to obsolete C and C++. It just has to attract a critical group of the next generation of systems programmers. Rust has very little competition in this space (newly designed languages that can be used to write ISRs, bootloaders, OS kernels, etc). Many of the other popular languages strive to find some middle ground between C/C++ and Java/C#/Python.