4 ms·

> Rust is the first one I'm too stupid to understand. That's a typical experience, but it's not because Rust is that complicated, but because it's different.

by pornel 7y ago

> Rust is the first one I'm too stupid to understand. That's a typical experience, but it's not because Rust is that complicated, but because it's different. People approach Rust with their intuition from other languages, such as C's or Java's where you can have a web of objects referencing each other willy-nilly, and are stumped by Rust's insistence on clear, single ownership, and simpler tree/DAG-like structure of program's data. Single ownership itself is simple, you just need to unlearn multiple-shared-ownership habits.

wahern 7y ago
Single ownership isn't a preference, it's a constraint and a limitation. A constraint (in a good way) because that's the entire point of the language--preventing you from losing track of memory ownership. It's a limitation because there are some extremely important and common relationships (e.g. putting an object in multiple containers simultaneously) that cannot be expressed in Rust's type system without using unsafe, RC boxes, etc, which create tremendous friction. Yet a low-level systems language is precisely the environment where you most want and need to express these relationships and data structures cleanly and efficiently. A data structure isn't something you import from a module. A good program is composed of layers of bespoke data structures that culminate in a giant data structure tailored for a particular task. So the fact that you can just import a hash table or tree implementation (friction-free) is irrelevant; useful, but irrelevant because it's combining such data structures that is the primary task at hand when writing a non-trivial piece of software, and combining them necessarily results in complicated ownership dependencies. And this is all especially true for applications that do more than transform some input into some output in one shot. I'm not trying to criticize Rust. They took a good idea and ran with it, and expressing more complex ownership semantics is still an open problem (perhaps with no satisfying answer, ever). But minimizing or dismissing the sore points isn't constructive.
- Twisol 7y ago
  > It's a limitation because there are some extremely important and common relationships (e.g. putting an object in multiple containers simultaneously) that cannot be expressed in Rust's type system without using unsafe, RC boxes, etc, which create tremendous friction. That is not a difficult relationship to capture, IMHO. The problem is that it's too easy to conflate "address" and "name" in most languages. An address is the physical location of a thing. A name is a location-independent way to look up a thing. Only one thing can live at an address, but you can use a name to look up multiple addresses depending on what question you want answered. (There is a parallel here with domain names and IP addresses.) Rust associates ownership with every address. This causes serious problems when you've been using addresses as names. The solution is to use different names. For instance, let's represent a graph as a vector of nodes. Every node has an index within that vector. Given this index, we can look up any node -- but critically, having the index does not confer ownership of the node. It's a completely orthogonal concern. So a node can quite happily possess a vector of the indices of its neighbors. You might complain that this is a poor naming scheme for a graph structure, since it complicates deletion of nodes from the vector. Great! We have these exact same problems when using pointers (observe that the index plus the vector base address is the node address), but now that naming is a first-class concept, we can consider it on its own terms. There are alternative index spaces we can apply. I've used this kind of design in a compiler hobby project. The intermediate representation is a set of tables representing different information about the program. With each pass, I can easily add new tables or transform existing ones, so long as I keep the names consistent. I can easily represent relationships between entities by using their indices. And I can easily attach more information to certain entities in compiler passes by adding a new table keyed on the same index.
  - wahern 7y ago
    I've always found this a very odd rationalization. These games are unnecessary to get the same sort of type safety and double-free protection.[1] They're sufficient to subvert the Rust compiler, but that's it. The "benefit" only exists in the context of Rust because it's the only way to accomplish this. And of course the extra indirection is costly, especially for a systems language. And you're also doing much more memory management. Hacks like these are why people say handling OOM is too difficult, because such hacks turn a simple logical operation into a series of multiple memory allocations, each of which could individually fail, which explodes the number of possible failure points, creating significantly more tension than necessary between memory management and functional clarity. [1] The analog in the land of pointers is simply to NULL node member pointers and fail if a member is non-NULL at destruction or insertion, which is effectively the same as with the index tables hack. And for type safety you can of course use container-specific member types, which is exactly how the canonical linked-list and tree implementations work in BSD. (See https://man.openbsd.org/queue#LIST_EXAMPLE https://man.openbsd.org/queue#LIST_EXAMPLE and https://man.openbsd.org/tree#EXAMPLES. https://man.openbsd.org/tree#EXAMPLES.) And like with many things in Rust, but-for the single ownership limitation Rust would otherwise make doing all of this much easier because it makes it easier to maintain the NULL invariants.
    - Twisol 7y ago
      I think that we are subtly talking past each other, because what you're saying doesn't line up against what I thought I was saying. I'm not looking for additional type safety or security protections from this model. See my sibling comment to your earlier comment for a little more context [1]. As someone who works with Rust often, your assertions don't match my experience. I want to engage honestly and understand where Rust itself falters, versus where we need to invest in educating newcomers about what kinds of models are better represented in Rust than others. I have found the model I'm describing to be valuable in many ways -- more than a "hack" to get around Rust. > And of course the extra indirection is costly, especially for a systems language. To my knowledge, base-offset addressing is natively present on at least x86. With a vector-based index space like I was describing, a[b] == * (a + b) is no more costly than * b. If you would need a more complex index space, you were probably doing more than bare pointers already. > And you're also doing much more memory management. > because such hacks turn a simple logical operation into a series of multiple memory allocations No, no more than you would have needed with pointer-based naming. The top-level arena (a vector in the graph example) owns everything and can be deallocated all at once when you're done with it. Arenas were already a well-known pattern before Rust, and they can reduce the amount of manual memory management over tracking a bunch of small, separate heap allocations. Relatedly, Catherine West gave an excellent talk at RustConf 2018 on how the Entity-Component-System architecture maps nicely into Rust [2]. The address/name distinction comes from ECS, but it becomes even more useful in Rust. > The analog in the land of pointers is simply to NULL node member pointers and fail if a member is non-NULL at destruction or insertion I don't see how this is hindered by Rust. We use an Option type -- with Some(T) and None variants -- to model something that might not be present, so the logic can be exactly the same. Rust can even optimize away the "extra" space you would need for the Some/None variant flag in common situations [3]. [1] https://news.ycombinator.com/item?id=20822003 https://news.ycombinator.com/item?id=20822003 [2] https://kyren.github.io/2018/09/14/rustconf-talk.html https://kyren.github.io/2018/09/14/rustconf-talk.html [3] https://github.com/rust-lang/rust/pull/45225 https://github.com/rust-lang/rust/pull/45225
Twisol 7y ago
This jibes very well with my own experience. When working in languages like Java, I feel myself overwhelmed with keeping a grip on the runtime topology of the object graph. Who can talk to whom? Who can _affect_ whom? My usual tools for covering my bases here -- immutability and/or value types -- are just not ergonomic in Java. If I have an "immutable" object with a getter that returns a reference to some mutable object, then it's not really immutable. Quoting Joe Armstrong: > You wanted a banana but what you got was a gorilla holding the banana and the entire jungle. The clarity and simplicity of a single ownership model trumps just about everything else for me.
jandrewrogers 7y ago
There are very important and valid use cases for logical ownership both where N > 1 and N == 0. Ironically, the latter case doesn't have a use case outside of systems programming, so it is deficient for a systems programming language to not support it. (N == 0 does not imply literally "no owner", just that the owner is intrinsically non-visible to the compiler.) Yes, it is possible to awkwardly and inefficiently shoehorn these cases into an N == 1 model but it comes at a substantial cost.
- pornel 7y ago
  The main point about Rust is that you're never stuck with a dead-end substantial cost. In the worst case you slap `unsafe` and you're no worse than in C (and better everywhere else where zero-cost abstractions worked and you didn't need to use `unsafe`). There are awkward cases (self-referential structs, ugh), but let's not forget that in many many cases there are straightforward, zero-cost solutions. E.g. parent incorrectly stated you can't put one object in multiple containers. You can: one will own it, the rest will borrow it. And if you can't explain in your program which one is the owner, chances are you have the same problem in other languages: • Many C programs with very complex and performance-sensitive ownership just use arenas. This pattern works in Rust, too (at zero cost). • If you're certain you can get the custom ownership right with C pointers, Rust has C pointers, too (at zero runtime cost). • When ownership is truly dynamic, C/C++ programs also have to use some kind of refcount or a flag to track it (e.g. setting freed pointer to null == Option.take() in Rust, bool free_me = borrow::Cow in Rust). Rust's refcounting is very efficient (intrusive, atomics not required), equal or better than `shared_ptr`. With practice the frequency of "awkwardness" when writing code drops. You may still need to be more precise/restrained (especially around mutability and global state), but that pays off as soon as you want to make your code multi-threaded.
  - jandrewrogers 7y ago
    For my use cases we can ignore the case of self-referential structs. Slapping "unsafe" on everything isn't equivalent to C++ because Rust is less expressive in some important ways. I mostly work on high-end database engines, which appear to violate assumptions of Rust about what a sensible memory model can look like. In a modern database engine, almost all of your working memory is unavoidably paged, opaque, and directly DMA-ed. The concept of a lifetime cannot be attached to a memory address and virtually all of your data structures are "unsafe" in a Rust sense. You have to consider implications like destructors being effectively non-deterministic. Fortunately, schedule-based safety models are transparent to developers, do not require "borrowing" so it works well with references not visible at compile-time, and all references can be mutable. To be clear, these models don't generalize to all software either, they are just well-suited to the requirements of high-throughput server engines. High-scale database engines are effectively single-threaded, so the only cost is a relatively trivial number of non-atomic ref counts. The underlying C++ to make this be handled automagically from a developer perspective is not trivial though. Rust's memory safety model looks like it was designed for more traditional applications, like web browsers. Modern server software that is I/O intensive is going to look pretty similar to the above if performance matters, and that is a rough fit for Rust's memory model.
CJefferson 7y ago
The borrow checker is only the first problem. Some objects gave an implicit member called await, except unlike every other member accessed with . this one does some crazy thing where the function returns early. Why does this look like member access? Any other weird members I need to know about?
- bbatha 7y ago
  `await` isn't a member its a post-fix operator. Its also a keyword so it will be highlighted in most environments and you won't be able to write actual members with the name unless you use and access it with the raw syntax `r#await`. Its also only available on types that impl the future trait (either manually or async functions). The only other post-fix operator in rust is `?` which will return early and do error conversion like the old `try!` macro. Both of these operators are post-fix so that you can easily intermix them with method calls and member accesses without temporaries. Going from a sync function to an async function is often little more than putting `async` in the function prototype and sprinkling some `.await`s in. If `.await` wasn't post-fix you would need to rewrite a large number of lines to introduce new temporaries. Ya it looks a little weird at first but this is a pretty lazy criticism that doesn't hold water for anyone who's spent more than 5 minutes looking at async Rust code.
  - steveklabnik 7y ago
    > Ya it looks a little weird at first but this is a pretty lazy criticism that doesn't hold water for anyone who's spent more than 5 minutes looking at async Rust code. To be fair to your parent, this exact critique was brought up by many experienced Rustaceans during the development of async/await. In the end, it was decided that this drawback is worth it, but this is a drawback.
    - bbatha 7y ago
      > this exact critique was brought up by many experienced Rustaceans Just because they're experienced doesn't make it a great argument. Additionally they generally articulated more substantial concerns than "it looks weird." To be exact my argument is that strange syntax does not increase the complexity of the feature. The complexity of async/await, especially when compared to features like the borrow checker as CJefferson complained, is in its semantics. You need to learn general async/await semantics like the fact that awaits always have yields regardless of which side of the expression you put it on among other things. And a number of rust-specific complexities that are very different from other languages (cold by default futures, executors vs reactors, pinning, etc.)

by pornel 7y ago

wahern 7y ago

Single ownership isn't a preference, it's a constraint and a limitation. A constraint (in a good way) because that's the entire point of the language--preventing you from losing track of memory ownership. It's a limitation because there are some extremely important and common relationships (e.g. putting an object in multiple containers simultaneously) that cannot be expressed in Rust's type system without using unsafe, RC boxes, etc, which create tremendous friction. Yet a low-level systems language is precisely the environment where you most want and need to express these relationships and data structures cleanly and efficiently. A data structure isn't something you import from a module. A good program is composed of layers of bespoke data structures that culminate in a giant data structure tailored for a particular task. So the fact that you can just import a hash table or tree implementation (friction-free) is irrelevant; useful, but irrelevant because it's combining such data structures that is the primary task at hand when writing a non-trivial piece of software, and combining them necessarily results in complicated ownership dependencies. And this is all especially true for applications that do more than transform some input into some output in one shot. I'm not trying to criticize Rust. They took a good idea and ran with it, and expressing more complex ownership semantics is still an open problem (perhaps with no satisfying answer, ever). But minimizing or dismissing the sore points isn't constructive.

Twisol 7y ago
> It's a limitation because there are some extremely important and common relationships (e.g. putting an object in multiple containers simultaneously) that cannot be expressed in Rust's type system without using unsafe, RC boxes, etc, which create tremendous friction. That is not a difficult relationship to capture, IMHO. The problem is that it's too easy to conflate "address" and "name" in most languages. An address is the physical location of a thing. A name is a location-independent way to look up a thing. Only one thing can live at an address, but you can use a name to look up multiple addresses depending on what question you want answered. (There is a parallel here with domain names and IP addresses.) Rust associates ownership with every address. This causes serious problems when you've been using addresses as names. The solution is to use different names. For instance, let's represent a graph as a vector of nodes. Every node has an index within that vector. Given this index, we can look up any node -- but critically, having the index does not confer ownership of the node. It's a completely orthogonal concern. So a node can quite happily possess a vector of the indices of its neighbors. You might complain that this is a poor naming scheme for a graph structure, since it complicates deletion of nodes from the vector. Great! We have these exact same problems when using pointers (observe that the index plus the vector base address is the node address), but now that naming is a first-class concept, we can consider it on its own terms. There are alternative index spaces we can apply. I've used this kind of design in a compiler hobby project. The intermediate representation is a set of tables representing different information about the program. With each pass, I can easily add new tables or transform existing ones, so long as I keep the names consistent. I can easily represent relationships between entities by using their indices. And I can easily attach more information to certain entities in compiler passes by adding a new table keyed on the same index.
- wahern 7y ago
  I've always found this a very odd rationalization. These games are unnecessary to get the same sort of type safety and double-free protection.[1] They're sufficient to subvert the Rust compiler, but that's it. The "benefit" only exists in the context of Rust because it's the only way to accomplish this. And of course the extra indirection is costly, especially for a systems language. And you're also doing much more memory management. Hacks like these are why people say handling OOM is too difficult, because such hacks turn a simple logical operation into a series of multiple memory allocations, each of which could individually fail, which explodes the number of possible failure points, creating significantly more tension than necessary between memory management and functional clarity. [1] The analog in the land of pointers is simply to NULL node member pointers and fail if a member is non-NULL at destruction or insertion, which is effectively the same as with the index tables hack. And for type safety you can of course use container-specific member types, which is exactly how the canonical linked-list and tree implementations work in BSD. (See https://man.openbsd.org/queue#LIST_EXAMPLE https://man.openbsd.org/queue#LIST_EXAMPLE and https://man.openbsd.org/tree#EXAMPLES. https://man.openbsd.org/tree#EXAMPLES.) And like with many things in Rust, but-for the single ownership limitation Rust would otherwise make doing all of this much easier because it makes it easier to maintain the NULL invariants.
  - Twisol 7y ago
    I think that we are subtly talking past each other, because what you're saying doesn't line up against what I thought I was saying. I'm not looking for additional type safety or security protections from this model. See my sibling comment to your earlier comment for a little more context [1]. As someone who works with Rust often, your assertions don't match my experience. I want to engage honestly and understand where Rust itself falters, versus where we need to invest in educating newcomers about what kinds of models are better represented in Rust than others. I have found the model I'm describing to be valuable in many ways -- more than a "hack" to get around Rust. > And of course the extra indirection is costly, especially for a systems language. To my knowledge, base-offset addressing is natively present on at least x86. With a vector-based index space like I was describing, a[b] == * (a + b) is no more costly than * b. If you would need a more complex index space, you were probably doing more than bare pointers already. > And you're also doing much more memory management. > because such hacks turn a simple logical operation into a series of multiple memory allocations No, no more than you would have needed with pointer-based naming. The top-level arena (a vector in the graph example) owns everything and can be deallocated all at once when you're done with it. Arenas were already a well-known pattern before Rust, and they can reduce the amount of manual memory management over tracking a bunch of small, separate heap allocations. Relatedly, Catherine West gave an excellent talk at RustConf 2018 on how the Entity-Component-System architecture maps nicely into Rust [2]. The address/name distinction comes from ECS, but it becomes even more useful in Rust. > The analog in the land of pointers is simply to NULL node member pointers and fail if a member is non-NULL at destruction or insertion I don't see how this is hindered by Rust. We use an Option type -- with Some(T) and None variants -- to model something that might not be present, so the logic can be exactly the same. Rust can even optimize away the "extra" space you would need for the Some/None variant flag in common situations [3]. [1] https://news.ycombinator.com/item?id=20822003 https://news.ycombinator.com/item?id=20822003 [2] https://kyren.github.io/2018/09/14/rustconf-talk.html https://kyren.github.io/2018/09/14/rustconf-talk.html [3] https://github.com/rust-lang/rust/pull/45225 https://github.com/rust-lang/rust/pull/45225

Twisol 7y ago

This jibes very well with my own experience. When working in languages like Java, I feel myself overwhelmed with keeping a grip on the runtime topology of the object graph. Who can talk to whom? Who can _affect_ whom? My usual tools for covering my bases here -- immutability and/or value types -- are just not ergonomic in Java. If I have an "immutable" object with a getter that returns a reference to some mutable object, then it's not really immutable. Quoting Joe Armstrong: > You wanted a banana but what you got was a gorilla holding the banana and the entire jungle. The clarity and simplicity of a single ownership model trumps just about everything else for me.

jandrewrogers 7y ago

There are very important and valid use cases for logical ownership both where N > 1 and N == 0. Ironically, the latter case doesn't have a use case outside of systems programming, so it is deficient for a systems programming language to not support it. (N == 0 does not imply literally "no owner", just that the owner is intrinsically non-visible to the compiler.) Yes, it is possible to awkwardly and inefficiently shoehorn these cases into an N == 1 model but it comes at a substantial cost.

pornel 7y ago
The main point about Rust is that you're never stuck with a dead-end substantial cost. In the worst case you slap `unsafe` and you're no worse than in C (and better everywhere else where zero-cost abstractions worked and you didn't need to use `unsafe`). There are awkward cases (self-referential structs, ugh), but let's not forget that in many many cases there are straightforward, zero-cost solutions. E.g. parent incorrectly stated you can't put one object in multiple containers. You can: one will own it, the rest will borrow it. And if you can't explain in your program which one is the owner, chances are you have the same problem in other languages: • Many C programs with very complex and performance-sensitive ownership just use arenas. This pattern works in Rust, too (at zero cost). • If you're certain you can get the custom ownership right with C pointers, Rust has C pointers, too (at zero runtime cost). • When ownership is truly dynamic, C/C++ programs also have to use some kind of refcount or a flag to track it (e.g. setting freed pointer to null == Option.take() in Rust, bool free_me = borrow::Cow in Rust). Rust's refcounting is very efficient (intrusive, atomics not required), equal or better than `shared_ptr`. With practice the frequency of "awkwardness" when writing code drops. You may still need to be more precise/restrained (especially around mutability and global state), but that pays off as soon as you want to make your code multi-threaded.
- jandrewrogers 7y ago
  For my use cases we can ignore the case of self-referential structs. Slapping "unsafe" on everything isn't equivalent to C++ because Rust is less expressive in some important ways. I mostly work on high-end database engines, which appear to violate assumptions of Rust about what a sensible memory model can look like. In a modern database engine, almost all of your working memory is unavoidably paged, opaque, and directly DMA-ed. The concept of a lifetime cannot be attached to a memory address and virtually all of your data structures are "unsafe" in a Rust sense. You have to consider implications like destructors being effectively non-deterministic. Fortunately, schedule-based safety models are transparent to developers, do not require "borrowing" so it works well with references not visible at compile-time, and all references can be mutable. To be clear, these models don't generalize to all software either, they are just well-suited to the requirements of high-throughput server engines. High-scale database engines are effectively single-threaded, so the only cost is a relatively trivial number of non-atomic ref counts. The underlying C++ to make this be handled automagically from a developer perspective is not trivial though. Rust's memory safety model looks like it was designed for more traditional applications, like web browsers. Modern server software that is I/O intensive is going to look pretty similar to the above if performance matters, and that is a rough fit for Rust's memory model.

CJefferson 7y ago

The borrow checker is only the first problem. Some objects gave an implicit member called await, except unlike every other member accessed with . this one does some crazy thing where the function returns early. Why does this look like member access? Any other weird members I need to know about?

bbatha 7y ago
`await` isn't a member its a post-fix operator. Its also a keyword so it will be highlighted in most environments and you won't be able to write actual members with the name unless you use and access it with the raw syntax `r#await`. Its also only available on types that impl the future trait (either manually or async functions). The only other post-fix operator in rust is `?` which will return early and do error conversion like the old `try!` macro. Both of these operators are post-fix so that you can easily intermix them with method calls and member accesses without temporaries. Going from a sync function to an async function is often little more than putting `async` in the function prototype and sprinkling some `.await`s in. If `.await` wasn't post-fix you would need to rewrite a large number of lines to introduce new temporaries. Ya it looks a little weird at first but this is a pretty lazy criticism that doesn't hold water for anyone who's spent more than 5 minutes looking at async Rust code.
- steveklabnik 7y ago
  > Ya it looks a little weird at first but this is a pretty lazy criticism that doesn't hold water for anyone who's spent more than 5 minutes looking at async Rust code. To be fair to your parent, this exact critique was brought up by many experienced Rustaceans during the development of async/await. In the end, it was decided that this drawback is worth it, but this is a drawback.
  - bbatha 7y ago
    > this exact critique was brought up by many experienced Rustaceans Just because they're experienced doesn't make it a great argument. Additionally they generally articulated more substantial concerns than "it looks weird." To be exact my argument is that strange syntax does not increase the complexity of the feature. The complexity of async/await, especially when compared to features like the borrow checker as CJefferson complained, is in its semantics. You need to learn general async/await semantics like the fact that awaits always have yields regardless of which side of the expression you put it on among other things. And a number of rust-specific complexities that are very different from other languages (cold by default futures, executors vs reactors, pinning, etc.)