10 ms·
Rust lifetimes: Getting away with things that would be reckless in C++
- keeperofdakeys 12y agoJust as an aside, the &str is not stored as two pointers, but a pointer and a length.
- deleted 12y ago[deleted]
- deleted 12y ago[deleted]
- asuffield 12y agoThere's an obvious extension here for lifetime inference - the example given doesn't need to be an error, it could compile correctly by increasing the object lifetime to the outer block. I don't know offhand whether there is a universally correct inference algorithm for that (if every other language feature was static then unification would solve it easily, but the other language features are not static and I don't know how it would interact with rust type inference).
- riffraff 12y agowouldn't extending the lifetime in this situations lead to subtle hard to trace memory leaks?
- keeperofdakeys 12y agoNo. Currently a reference in a struct or enum has no inferred lifetime, and must be explicitly stated. Having it default to the lifetime of that containing struct or enum would simply mean you don't need to specify it. The danger is it could infer wrongly, leading to lifetime errors that might be cryptic. But at the end of the day, the Rust compiler will never allow a reference to outlive the original object.
- dbaupp 12y agoYou've misinterpreted, it's not a question about reducing the annotation in a struct/enum definition, but about postponing the destruction of the String so that the later references are valid, i.e. currently we have fn test_parse_unsafe() { let v = { let text = "The cat".to_string(); tokenize_string3(text.as_slice()) }; // `text` destroyed here assert_eq!(vec![Word("The"), Other(" "), Word("cat")], v); } but the suggestion/question is about changing this to fn test_parse_unsafe() { let v = { let text = "The cat".to_string(); tokenize_string3(text.as_slice()) }; assert_eq!(vec![Word("The"), Other(" "), Word("cat")], v); } // `text` destroyed here so that the references in `v` are valid. This could lead to "memory leaks", where a destructor is implicitly postponed to a higher scope, but I don't think it would be much of a problem in practice (the promotion would only be through simple scopes, not through loops, and maybe not through `if`s). In fact, there's an yet-to-be-implemented accepted RFC covering this[1] (there's no guarantee that it will be implemented though, just that the idea is mostly sound). [1]: https://github.com/rust-lang/rfcs/blob/master/active/0031-better-temporary-lifetimes.md https://github.com/rust-lang/rfcs/blob/master/active/0031-be...
- asuffield 12y agoAh yes, that RFC is roughly what I had in mind, thanks. I believe that it's safe to promote through an if, although obviously not through a general loop.
- dbaupp 12y agoYes, I agree that it should be safe, I've softened my original text. However, it would require dynamically tracking if the destructor needs to be run, and there's currently discussion[1] about Rust possibly moving to a static model, for the highest performance. [1]: https://github.com/rust-lang/rfcs/pull/210 https://github.com/rust-lang/rfcs/pull/210
- asuffield 12y agoConsider this: On a two-way if statement, then a given storage location is either set on zero, one or both branches. If it is set on neither branch then the if statement is irrelevant and can be ignored. If it is set on one branch, then either it had an original value and hence can be treated as being set on both branches, or it must be destroyed within the branch of the if (no null pointers - think about it until it is clear that the type system guarantees this). Hence we are only interested in cases which are isomorphic with the location being set on both branches. We can treat this as a phi node following the if: there is one output value, which has been created in one of two different ways. In this case we don't know statically which value has been constructed, but we do know statically how and when to destroy it regardless of which one we get, because both branches have the same type and storage location. We don't actually need to know where it came from. Any obvious problems? I think it works...
- asuffield 12y agoThe obviously "safe" thing to do is to push it out as far as the containing function, and no further. That would be sufficient for this scenario and probably all the ones where you would ever want this to happen.
- Drakim 12y agoIn general I prefer the compiler to yell at me rather than to secretly help me behind the scenes. Otherwise, I don't learn anything and I get burned later on when there is a situation where the safeguard can't help me.
- asuffield 12y agoThis appears to be a general objection to inference algorithms in languages? Personally I find type inference to be immensely valuable.
- rwmj 12y agoOCaml recently added a bunch of "help" -- for example, if you reference a field in a struct, but the name of the field could be from one of several structs, then it will guess which struct you mean. TBH I don't find this to be that useful -- it covers up potential mistakes, and of course means that code breaks when compiled with the older version of OCaml which didn't do this. Edit: I should note that this doesn't break type safety.
- zem 12y agoI like clang's general approach, where if it can guess what you mean to do, it says "did you mean ..." in the error message, and carries on trying to compile the rest of the code on an "assume we did make this change" basis, but fails overall.
- hamstergene 12y agoIf the object has destructor with side effects, I believe that would massively complicate reasoning about program's behavior for human who's reading it.
- fanf2 12y agoThat is essentially what region inference in the MLKit compiler does. http://www.elsman.com/mlkit/ http://www.elsman.com/mlkit/ It has some unexpected properties: it can sometimes produce dangling pointers because it knows the target data will never be used, even though a GC would treat it as live. And on the other hand it will often promote data to an excessively long-lived region (because region lifetimes have to be nested). So current versions of the MLKit use GC as well as region inference.
- Yardlink 12y agoIs there are reason this language exists? They're solving a problem that's been solved many times over for at least 2 decades in the form of managed languages.
- mattdw 12y agoBecause Rust's "managed" benefits are compiled away, they have zero runtime impact. Meaning that there's no reason it couldn't be as fast as C/C++ for all workloads. So you get a safe language with bare-metal performance. (Bare metal like, people have written basic OS kernels in it.)
- wofo 12y ago> They're solving a problem that's been solved many times over for at least 2 decades in the form of managed languages. With the overhead of GC...
- ludamad 12y agoActually, many people have been waiting long for such a language to come out, and are very excited about Rust for good reasons. Managed languages solve a large set of problems, but introduce two big problems as I see it: - Cause nondeterministic overhead which is particularly problematic for kernels, games, and software with heaps over, say, 10 gigabytes. - Limit ability to safely pass objects outside their ecosystem. How many libraries written in managed code are worth using outside of the language they are written in?
- pjmlp 12y agoActually it depends on what one understands by managed. Strong typed languages with automatic memory management, having native compilers and used for writing OS, exist since Mesa/Cedar (70's at Xerox PARC) Back when C was UNIX only, no one talked about C ABI. The only ABI that mattered was the OS ABI. Amoeba, OS/400, Lisp Machines, Oberon are a few examples of having multiple languages interoperate via the OS ABI..
- ludamad 12y ago
- GoGolli 12y agoRust is the best complicated language I have seen!!!!!!
- svalorzen 12y agoOr, you know, instead of returning two C pointers which in modern C++ makes no sense, return a vector of `std::pair<size_t,size_t>` with position and length of each substring, and if needed use `std::string::substr` to extract the parts you need.
- expr- 12y agoI would say returning regular pointers is more C++-esque than your handmade range implementation. Pointers are, after all, a kind of iterator, which is an essential C++ concept. (std::string::iterator is a handful.) (Furthermore, there is nothing "C" or wrong with regular pointers. Their only "flaw" is that they can't manage an object, but they're still the semantical way to refer to one.)
- svalorzen 12y agoReturning regular pointers which point to memory which is not managed by you and that you by default know could disappear at any time is not the C++ way of doing anything. Thing is, a class has a public interface for a reason, and if you as an outsider want to start playing pointer games you know from the start what you are in for. I never implied that pointers are bad, just that this particular usage of pointers is bad, and as such should not be used as an example of why we would want something else over C++. If I ever saw a function with a signature like the one proposed I'd be extremely suspicious of what was going on. Note that this would happen in the same way if the function returned pairs of iterators. It is a choice that depends on the documented and intended usage of a function. If you feel your tokenizer will be used on temporaries (I fail to see how that would ever be useful though) you could overload the function to also take universal references or change the return parameters so that nobody can get hurt. Otherwise you return whatever delimiters you are comfortable with, with the assumption that they will not be used in some way. All code can be broken if you actively try to, so it makes no sense to protect against everything.
- repsilat 12y ago> Returning regular pointers which point to memory which is not managed by you and that you by default know could disappear at any time is not the C++ way of doing anything. Thing is, a class has a public interface for a reason, and if you as an outsider want to start playing pointer games you know from the start what you are in for. Maybe not "regular pointers", but returning iterators is very common (all throughout the STL, for one thing.) Those iterators are part of the public interface, and they provide no real guarantees over bare pointers. I don't have a horse in this race, though. I think the "nicest" solution in abstract is for `tokenize` to be a token-generator* that takes a character generator argument. If you want to store the tokens yourself you can copy them, if you're happy to stream-process them then you can do that, and if you want a reference into the original string storage you can go fly a kite. *: Where a "token" is probably another character generator.
- bsaul 12y agoWhich makes me wonder : 1/ could you build the same unsafe behiavor in Rust if you wanted to by not specifying lifetime constraints ? 2/ If yes, shouldn't lifetime constraints be mandatory ?
- dbaupp 12y agoThe compiler will not compile code it cannot verify as safe (outside an `unsafe` block), meaning it will complain about lifetime constraints that are missing, e.g. it is a compile-time error to not propagate the lifetime information out of the enum. http://play.rust-lang.org/?run=1&code=enum%20Token%20{%0A%20%20%20%20Word%28%26str%29%2C%0A%20%20%20%20Other%28%26str%29%0A}%0A%0Afn%20main%28%29%20{} http://play.rust-lang.org/?run=1&code=enum%20Token%20{%0A%20...
- nikic 12y agoThe lifetime constraint in the enum is mandatory, the code won't compile without it. The tokenization function itself doesn't have to explicitly specify lifetimes like it does in the blog post. It would also work without them and Rust would still ensure that you don't end up with a use-after-free.
- linguafranca 12y agoI'm hearing an awful lot about Rust on HN, even though afaict it still does't have a basic http package yet, limiting the main types of apps I would build with it. Maybe I'm in the minority, but perhaps we can slow down on Rust news until it's a little closer to usable?
- nicklaforge 12y ago> slow down on Rust news until it's a little closer to usable? "Usable" is an ironic way to spell "hackable". You mentioned the words "news" and "use". Does the H in HN still stand for something? Maybe HN stands for "HTTP News"?
- wismer 12y agoI like reading this stuff. Sure, I may not use Rust for any real application yet, but it's been teaching me a whole lot on different paradigms than I am used to. I think these articles are great because it pushes boundaries in understanding. And this is a science we are talking about - experimentation is a lot of fun!
- Jweb_Guru 12y agoWhile I'm certainly not going to say Rust is ready for use in production or anything like that, I'll have to dispute your points here: (1) It has at least three HTTP packages that I'm aware of (not finished ones, admittedly). (2) While perhaps for you lacking an HTTP package is a dealbreaker, many of the areas Rust is targetting (such as embedded) don't require it at all. (3) Why should people stop talking about a language because it hasn't been released yet, anyway? I'm not sure I follow the logic here.
- jey 12y agoMaybe there's just enough people interested in types of software that don't involve making HTTP requests. Programs that make REST requests is only one tributary within the taxonomy of software.
- adamnemecek 12y agoOne way a new FOSS platform can become more usable is by attracting new contributors. If a new platform is talked about, it will attract new contributors. Ergo by being talked about, a new platform becomes more usable.
- overgard 12y agoThis seems like the kind of place where std::shared_ptr would really shine. The author's point on the danger of pointers is well taken, but some of the new pointer types get around a lot of these issues. You couldn't use it to point into the middle of the string, but if you paired it with some offsets you wouldn't have to worry about the ownership of the pointer anymore.
- dbaupp 12y agoshared_ptr doesn't fix this, as soon as you access the string via any standard type (char* or string_view) you can copy that type around freely, disconnected from the shared_ptr. Hence, you can drop all the shared_ptrs and leave the string view dangling.
- pcwalton 12y agostd::shared_ptr has (atomic!) reference counting overhead.
- monocasa 12y agoNot to mention the issues with reference counting in general. Having to make sure you don't have (even indirect) circular references means you have to explicitly think about ownership anyway.
- missblit 12y agoIn C++ if the string is a rvalue reference you could std::move it into part of the return value. Think a signature like template<typename T> std::pair<std::string, std::vector<std::string_view>> tokenize_string(T &&str); This would be efficient when the user passes a temporary, and it would be safe. Which isn't to say the Rust solution isn't totally cool. Being able to easily check this class of errors at compile time is probably a lot nicer than needing to learn all the relatively complicated parts that would go into a easy to use / safe /efficient / still slightly weird C++ solution.
- eridius 12y agoSure, and then somewhere along the way you throw away the first element of the pair, because you're not using it, but you're still using the views, and oops you just reintroduced the bug you tried to fix. Which is to say, yes, you can obviously write C++ code that works. But you run the risk that one tiny mistake, or a change weeks, months, or years later, causes memory issues. Being able to completely rule out this class of error at compile-time is really amazingly useful.
- missblit 12y agoYes I agree completely. I was thinking about that after I wrote my comment, the code is only safe relative to the original sample, and it's only a band-aid for the lack of more comprehensive lifetime guarantees. Of course you can jump through more hoops, by encapsulating the string, string_view data in a proper class. But when there's tradeoffs between safety and convenience, there's a point where people will understandingly choose convenience.
- ajross 12y agoThat's a forest for the trees argument. The ability to introduce bugs ("make a tiny mistake") in future changes is an inherent property of software, and you can't fix this in the general case. Rust just fixes this for the case of free-memory-read bugs. That has value, but it's a much more limited scope than you're implying. Really I think this is the biggest problem with Rust. The stuff broken about C++ isn't really its memory model, it's simply its complexity. And every time I look at what's happening in Rust I see the same kind of complexity being introduced at every turn. I mean seriously: "named lifetime parameters"? And the syntax is a bare single quote? That's a joke, right? No one but Rust nerds is going to grok this, and real code needs to me maintained by mediocrities. Frankly Go seems to have this "don't introduce new nonsense" problem under much better control, so if I need to pick a horse in the race, it's that one, not Rust.
- shmerl 12y ago> The function get_input_string returns a temporary string, and tokenize_string2 builds an array of pointers into that string. Unfortunately, the temporary string only lives until the end of the current expression, and then the underlying memory is released. And so all our pointers in v now point into oblivion So what stops you from returning a shared pointer in case of get_input_string? Then take over that ownership and use it. It's still a potential problem that v is logically disconnected from lifetime of that pointer, but at least you could avoid the problem you described.
- enjoy-your-stay 12y agoIn C++, the best way to hand out pointers to anything where the creator may not necessarily be the last one referencing that object or chunk of RAM is to use reference counting, which would have solved the posters' problem. It would mean that you would have to wrap the incoming string in a class, and probably add the tokenize_string method to that class. Then you would also have to wrap the results vector in a class that then addrefs the original string wrapper class. But after that, handing out pointers to the contents of the string would be no problem as the results class would addref the string class and then release it when done ensuring that the string wrapper class remains alive as long as the results object has not gone out of scope. Of course Rust's approach of alerting you when your code path causes dangling pointers is also interesting, but I wonder how that would work if you were to link against a static library that handed out references to internal objects like that - could the compiler see the scoping problem?