4 ms·

Extending Rust's Effect System

151 points by egnehots 3y ago

mbStavola 3y ago
I think there is a lot of value in exploring this idea, even though it's a "big idea." Rust already has a history of taking PLT ideas system programmers don't care about and making it practical and appealing. This could be another one of those things where the effort pays off many fold and we all wonder why other languages don't do this. It could be as important as generics, traits, or even lifetimes, who knows! That said, it is incredibly complex and a lot to grok. It's a completely alien concept to most programmers and there is still some confusion over what this feature even entails. As many people have said, it's questionable if you could truly unify some of these "effects" in a way that is useful or even coherent; you might have a sync and async version of a function that ostensibly does the same thing, but the implementation details could diverge a lot to the point where you might as well have just written two different functions. This sort of feature really needs an exploration of what it would look like in practice for existing, popular APIs, and not just toy examples. I really am looking forward to the full RFC with the hopes that they nail the sweet spot here and we end up getting a nice effect system that helps me write beautiful, correct, and performant APIs (more so than I can already do today). At the same time, I also want one of the "correct" outcomes of the RFC to be "let's not do this at all." I worry with the amount of excitement and promotion this initiative has that this may end up being adopted by default. I really have no basis for thinking that, but I do keep seeing this pop up and a lot of people have valid questions and concerns that just go unanswered.
- Voultapher 3y ago
  Very well said! I think there is prior art that suggests unifying, A), and implementing things on a library level, B), are not always desirable. One example each from C++: A) Generic algorithms for all containers. This is often touted as a wonderful feature of the standard library. In practice it shifts a lot of complexity into the implementation, that suddenly needs to have 6 different implementations inside the same function, with noticeably different characteristics. And at the end it often still doesn't do the right thing in the domain. For example std::find with std::string is probably not what you want when handling utf-8. Case in point std::string has a bunch of custom find functions. After having used Rust for a while, I find the distinction into type, slice and iter methods, both simpler and more reliable. Even when at a surface level it is less uniform than in C++. B) std::variant. A lot of optimizations around occupancy holes are left on the table because it's a library level abstraction. And even more impactful are match ergonomics that by design require language level integration. I'm not sating effect systems are doomed from ever working. Just so far that I've found the most prominent reasons given why a uniform effect system would be desirable, to be not that terribly convincing. Even in places like Koka a language built around that idea. So I fully agree that a "hey it's not worth it" must be a possible outcome, in the following discussions.
- ldng 3y ago
  At one point Rust as to decide if it wants to be a popular and used language to allow the masses actually do stuff. Or be an evergreen experimental language that will stay a niche language or even wither under the weight of impractical theoretically "elegant" constructs. All those years and I'm still wondering if it is going to be the next C or the next Scala (and die out). It is a real barrier to adoption.
carterschonwald 3y ago
Basically rust needs to be able to express generics that have the interface popularly known at monads.
- zozbot234 3y ago
  Effect types as currently understood are closer to Lawvere theories than monads. The biggest difference is that the former can be composed out-of-the-box with no ambiguity, whereas you can't really compose monads without resorting to a clunky "transformers" pattern.
  - carterschonwald 3y ago
    That’s been clearly disproven for like a decade… you just need to have your effect monad structure be indexed by the set of symbols in question! Especially since the effects are baked into the language. The transformers formulation is only relevant for userland defined semantics! It doesn’t matter if the monadic structure is there in user syntax, it’s there in the semantics.
    - Rusky 3y ago
      Pretty important distinction when you're specifically talking about the interface being expressed in the language.
      - carterschonwald 3y ago
        Actually less than you’d think! Granted I’ve been tinkering with ways to let users choose how much they wanna see fancy info vs let a compiler do the book keeping. Keeping track of this info in the compiler ir is trivial
        Rusky 3y ago
        What I mean is, merely plopping a Monad trait into Rust and implementing everything in userspace is famously not an effective implementation approach, so your phrasing is bound to confuse people when you are instead referring to the semantics.
IshKebab 3y ago
> Not being able to use ? inside of arbitrary closures is an example of an effect mismatch. Yes! This is probably the most annoying thing about Rust in day to day use IMO. You can work around it but it's always a pain.
pavlov 3y ago
The structure of this article makes it unreadable if you don’t already know what effects mean in this context. The introduction promises that the talk will explain what effects are. Finally, some 75% in, after discussing topics like “what happens when we can’t be generic over effects”, it does get around to answering the question of what effects actually are. But the answer is rather unsatisfying if you don’t know these definitions already. For example: “A lot of languages which have effects provide both effect types and effect handlers. These can be used together, but they are in fact distinct features. In this talk we'll only be discussing effect types.” This doesn’t really clarify much.
Ericson2314 3y ago
Algebraic effects always struck me as an overrided fad. I would much rather they first spend time and something old and proven like higher kinded types.
- zozbot234 3y ago
  Rust has GAT which have the same power as higher-kinded types. However the patterns you might expect in a language like Haskell are harder to implement due to unforeseen interactions with Rust's lower-level features (such as no obligatory GC) - the standard approaches don't lead to a zero overhead abstraction in Rust. That's one of the reasons why this kind of work is needed.
  - chowells 3y ago
    Well, no. GAT might let you fake Monad, but it doesn't let you fake Traversable. And honestly, Traversable is just as useful.
  - Ericson2314 3y ago
    I'm glad this got thoroughly downvoted. That some of the things people do in Haskell with HKTs don't work as well in Rust is poor evidence that there are no good uses for HKTs in Rust. Indeed, there are uses that would in Rust that don't exist in Haskell, like parameterizing something with a pointer type constructor (Type -> Type).
daxfohl 3y ago
Could this be done more simply by how go does exceptions: just return the effect and let the caller handle it (or not)? So a function that has exceptions would return "value, error". A function that is async would return "value, future". A function that yields would return "value, tail". I'm sure I'm missing something though.
vmchale 3y ago
> We've added major features such as the try operator (?), const generics, generic associated types (GATs), and of course: async/.await. Out of those four features, three are what can be considered to be "effects". I wouldn't call the try operator an effect, Result is a monad but it's quite different from the I/O monad in Haskell
- tialaramex 3y ago
  Indeed. I'd consider the current rendition of Try a magic conversion operator, not so different from IntoInterator (which powers Rust's for each loop sugar) for example, or the IntoFuture mechanism which drives async/await sugar or Display (which turns things into strings). A user defined function turns your type T into a possibly different type S which has some specific property and then an agreed thing happens to the S, using that property. In some cases your implementation is a no-op, you already did have the desired property, in other cases you must do a lot of work to make it happen.
- hurril 3y ago
  Let's not conflate side effects and effects.
infogulch 3y ago
Related: A lowering strategy for control effects in Rust | 26 days ago | 40 comments | https://news.ycombinator.com/item?id=39005780 https://news.ycombinator.com/item?id=39005780 Let futures be futures | 7 days ago | 96 comments | https://news.ycombinator.com/item?id=39242962 https://news.ycombinator.com/item?id=39242962 Also this thread on the Rust Internals forum has some recent discussion about async effect in particular: Case against "maybe async" | https://internals.rust-lang.org/t/case-against-maybe-async/20144 https://internals.rust-lang.org/t/case-against-maybe-async/2...
rpigab 3y ago
Nice maybe future result of RFC!

151 points by egnehots 3y ago

I think there is a lot of value in exploring this idea, even though it's a "big idea." Rust already has a history of taking PLT ideas system programmers don't care about and making it practical and appealing. This could be another one of those things where the effort pays off many fold and we all wonder why other languages don't do this. It could be as important as generics, traits, or even lifetimes, who knows! That said, it is incredibly complex and a lot to grok. It's a completely alien concept to most programmers and there is still some confusion over what this feature even entails. As many people have said, it's questionable if you could truly unify some of these "effects" in a way that is useful or even coherent; you might have a sync and async version of a function that ostensibly does the same thing, but the implementation details could diverge a lot to the point where you might as well have just written two different functions. This sort of feature really needs an exploration of what it would look like in practice for existing, popular APIs, and not just toy examples. I really am looking forward to the full RFC with the hopes that they nail the sweet spot here and we end up getting a nice effect system that helps me write beautiful, correct, and performant APIs (more so than I can already do today). At the same time, I also want one of the "correct" outcomes of the RFC to be "let's not do this at all." I worry with the amount of excitement and promotion this initiative has that this may end up being adopted by default. I really have no basis for thinking that, but I do keep seeing this pop up and a lot of people have valid questions and concerns that just go unanswered.

Voultapher 3y ago
Very well said! I think there is prior art that suggests unifying, A), and implementing things on a library level, B), are not always desirable. One example each from C++: A) Generic algorithms for all containers. This is often touted as a wonderful feature of the standard library. In practice it shifts a lot of complexity into the implementation, that suddenly needs to have 6 different implementations inside the same function, with noticeably different characteristics. And at the end it often still doesn't do the right thing in the domain. For example std::find with std::string is probably not what you want when handling utf-8. Case in point std::string has a bunch of custom find functions. After having used Rust for a while, I find the distinction into type, slice and iter methods, both simpler and more reliable. Even when at a surface level it is less uniform than in C++. B) std::variant. A lot of optimizations around occupancy holes are left on the table because it's a library level abstraction. And even more impactful are match ergonomics that by design require language level integration. I'm not sating effect systems are doomed from ever working. Just so far that I've found the most prominent reasons given why a uniform effect system would be desirable, to be not that terribly convincing. Even in places like Koka a language built around that idea. So I fully agree that a "hey it's not worth it" must be a possible outcome, in the following discussions.
ldng 3y ago
At one point Rust as to decide if it wants to be a popular and used language to allow the masses actually do stuff. Or be an evergreen experimental language that will stay a niche language or even wither under the weight of impractical theoretically "elegant" constructs. All those years and I'm still wondering if it is going to be the next C or the next Scala (and die out). It is a real barrier to adoption.

carterschonwald 3y ago

Basically rust needs to be able to express generics that have the interface popularly known at monads.

zozbot234 3y ago
Effect types as currently understood are closer to Lawvere theories than monads. The biggest difference is that the former can be composed out-of-the-box with no ambiguity, whereas you can't really compose monads without resorting to a clunky "transformers" pattern.
- carterschonwald 3y ago
  That’s been clearly disproven for like a decade… you just need to have your effect monad structure be indexed by the set of symbols in question! Especially since the effects are baked into the language. The transformers formulation is only relevant for userland defined semantics! It doesn’t matter if the monadic structure is there in user syntax, it’s there in the semantics.
  - Rusky 3y ago
    Pretty important distinction when you're specifically talking about the interface being expressed in the language.
    - carterschonwald 3y ago
      Actually less than you’d think! Granted I’ve been tinkering with ways to let users choose how much they wanna see fancy info vs let a compiler do the book keeping. Keeping track of this info in the compiler ir is trivial
      - Rusky 3y ago
        What I mean is, merely plopping a Monad trait into Rust and implementing everything in userspace is famously not an effective implementation approach, so your phrasing is bound to confuse people when you are instead referring to the semantics.

IshKebab 3y ago

> Not being able to use ? inside of arbitrary closures is an example of an effect mismatch. Yes! This is probably the most annoying thing about Rust in day to day use IMO. You can work around it but it's always a pain.

pavlov 3y ago

The structure of this article makes it unreadable if you don’t already know what effects mean in this context. The introduction promises that the talk will explain what effects are. Finally, some 75% in, after discussing topics like “what happens when we can’t be generic over effects”, it does get around to answering the question of what effects actually are. But the answer is rather unsatisfying if you don’t know these definitions already. For example: “A lot of languages which have effects provide both effect types and effect handlers. These can be used together, but they are in fact distinct features. In this talk we'll only be discussing effect types.” This doesn’t really clarify much.

Ericson2314 3y ago

Algebraic effects always struck me as an overrided fad. I would much rather they first spend time and something old and proven like higher kinded types.

zozbot234 3y ago
Rust has GAT which have the same power as higher-kinded types. However the patterns you might expect in a language like Haskell are harder to implement due to unforeseen interactions with Rust's lower-level features (such as no obligatory GC) - the standard approaches don't lead to a zero overhead abstraction in Rust. That's one of the reasons why this kind of work is needed.
- chowells 3y ago
  Well, no. GAT might let you fake Monad, but it doesn't let you fake Traversable. And honestly, Traversable is just as useful.
- Ericson2314 3y ago
  I'm glad this got thoroughly downvoted. That some of the things people do in Haskell with HKTs don't work as well in Rust is poor evidence that there are no good uses for HKTs in Rust. Indeed, there are uses that would in Rust that don't exist in Haskell, like parameterizing something with a pointer type constructor (Type -> Type).

daxfohl 3y ago

Could this be done more simply by how go does exceptions: just return the effect and let the caller handle it (or not)? So a function that has exceptions would return "value, error". A function that is async would return "value, future". A function that yields would return "value, tail". I'm sure I'm missing something though.

vmchale 3y ago

> We've added major features such as the try operator (?), const generics, generic associated types (GATs), and of course: async/.await. Out of those four features, three are what can be considered to be "effects". I wouldn't call the try operator an effect, Result is a monad but it's quite different from the I/O monad in Haskell

tialaramex 3y ago
Indeed. I'd consider the current rendition of Try a magic conversion operator, not so different from IntoInterator (which powers Rust's for each loop sugar) for example, or the IntoFuture mechanism which drives async/await sugar or Display (which turns things into strings). A user defined function turns your type T into a possibly different type S which has some specific property and then an agreed thing happens to the S, using that property. In some cases your implementation is a no-op, you already did have the desired property, in other cases you must do a lot of work to make it happen.
hurril 3y ago
Let's not conflate side effects and effects.

infogulch 3y ago

Related: A lowering strategy for control effects in Rust | 26 days ago | 40 comments | https://news.ycombinator.com/item?id=39005780 https://news.ycombinator.com/item?id=39005780 Let futures be futures | 7 days ago | 96 comments | https://news.ycombinator.com/item?id=39242962 https://news.ycombinator.com/item?id=39242962 Also this thread on the Rust Internals forum has some recent discussion about async effect in particular: Case against "maybe async" | https://internals.rust-lang.org/t/case-against-maybe-async/20144 https://internals.rust-lang.org/t/case-against-maybe-async/2...

rpigab 3y ago

Nice maybe future result of RFC!