4 ms·

What is the actual problem that this effect-oriented approach addresses? The effects of methods that are called by other methods will be concealed, so at the ca

What is the actual problem that this effect-oriented approach addresses? The effects of methods that are called by other methods will be concealed, so at the calling site, you'll have no idea that the high-level API `do_stuff` is going to utilize `io`. That is, unless the effect decoration "infects" the caller like `async`, in which case every complex API method will be decorated with a huge number of effects. Simply put, what does this buy us?

wavemode 2y ago
Effect systems have always felt to me like the functional programming community re-inventing dependency injection, but at the type level. At the term level, you could just let your function have a function as a parameter `read_file: IO -> String`, vs. annotating it with some kind of type-level "filesystem effect". And the former is a lot more flexible over time.
- mitt_romney_12 2y ago
  This is an interesting approach but it has some issues with more more advanced effects. For example if you have a function that uses mutable state you would need to have the state both as an input and an output (for the original and the updated state) `update_mut: State -> String -> State` this isn't very economical as is probably worse than just `update_mut: String -> State ()`. If you commit to just having the parameters act as "markers" that don't actually relate to the implementation then you can get away with `update_mut: State -> String -> ()` but then you use one of the big benefits of effect systems which is that you can change the way effects work and have multiple different implementations.
- 12_throw_away 2y ago
  > Effect systems have always felt to me like the functional programming community re-inventing dependency injection Yeah, so, in python, I've been playing around with "structured dependency injection", basically DI + ideas from structured concurrency (i.e., dependency injection bound to well-defined lexical scopes). And whenever I read about effects systems, it definitely feels like they're talking about, if not the same thing, at least something in the same nuclear family of things.
fwouts 2y ago
Unless I completely misunderstood the post, a function that calls another function with effects will indeed be "infected". That is, unless the effect ends up being local (such as passing a mutable reference to a local variable).
sparkie 2y ago
> The effects of methods that are called by other methods will be concealed, so at the calling site, you'll have no idea that the high-level API `do_stuff` is going to utilize `io`. The point is to make the caller aware of any side-effects so that the callee can't conceal them. > That is, unless the effect decoration "infects" the caller like `async`, in which case every complex API method will be decorated with a huge number of effects. Effects are infectious, precisely because if `foo` has effects, which must be known to `bar` which call's `foo`, then `baz` which calls `bar` must also be aware of the effects of foo, else they would be hidden. > Simply put, what does this buy us? For one, it makes it explicit what can and cannot be done by some code, as a means to prevent obvious programmer errors. You might consider it analogous to a static versus dynamic type system. In the dynamic system we can call `(foo 123)`, even though `foo` might expect strings rather than integers - and we get a failure at runtime. In a static typing system, we can catch this error much earlier - at compile time. `foo(s : String)` makes it clear what foo expects. Effects attempt to augment functions not only with the type of value they expect, but also with the capabilities that the function has. If a function is pure, it's not capable of going into your filesystem and deleting data, for instance. While the effects are infective, the languages encourage you to write as much as you can using pure functions and only use effects where strictly necessary. Essentially, they follow the principle of least privilege[1]. There are ways to avoid infecting a program with the `IO` effect where only "some" IO is needed. For instance, in Clean, which uses uniqueness types instead of effects, a function which writes to a particular file can be given the privilege for just this file, eg `write_foo : *File String -> *File`. That would prevent `write_foo` from opening up another file and writing to it. The file itself must be opened using a `*Filesystem` type, which `*World` - the primordial source of uniqueness for your program implements, which is passed into the program's entry point. The other advantage referential transparency gives us is optimization. If a function is called multiple times with the same arguments, it always produces the same result. If we can detect at compile time that a function is called more than once with the same arguments, we can cache the result of the first call, and replace the second call with the cached value. This same optimization can't be done with unknown side-effects because those side-effects may be desirable - we don't want the compiler to attempt to remove them. Clean's uniqueness types give us another opportunity for optimization. A uniqueness type is guaranteed by the compiler to not be aliased, which means that we can mutate a value in place whilst retaining referential transparency, though only in a single-threaded manner. In regards to threading, threads are also side-effects. If we call into some library function written by someone else, we don't want their functions to start spawning threads and potentially mutating values we give it, else it could cause any number of race conditions. If any functions which may spawn a thread have a big red flag on them, we can avert these problems earlier, rather than finding out in production and spending a lot of time debugging. Another advantage that purity provides (but not uniqueness) is the ability to reuse parts of data structures for multiple values. For example, if we have a linked list `l`, we can write both `x = cons foo l` and `y = cons bar l`, to obtain two new lists which both have `l` as their tail, but they both refer to the same `l` in memory. It's safe to do this because we can't mutate this tail - we wouldn't want a mutation of the list `x` to also mutate `y`. In a language which doesn't prevent arbitrary side-effects, we would need to make a copy of `l` when creating `x` and `y` to prevent this from occurring. Purely functional data structures have many uses - if we want to keep a versioned record of states (like git) for example. Would strongly recommend reading Okasaki's Purely Functional Data Structures[2] for more insight. [1]:https://en.wikipedia.org/wiki/Principle_of_least_privilege https://en.wikipedia.org/wiki/Principle_of_least_privilege [2]:https://www.cs.tufts.edu/~nr/cs257/archive/chris-okasaki/dissertation.pdf https://www.cs.tufts.edu/~nr/cs257/archive/chris-okasaki/dis...
empath75 2y ago
Containing effects _in general_ is the natural next step after containing unsafe code. Memory safety issues are just one category of side effect that can have unpredictable effects on your code -- any time your code interacts with the "outside world" there's an opportunity for unpredictable behavior, and the more you can quarantine it, the easier it is to reason about the remaining code.
- wavemode 2y ago
  The difference is that a compiler knows all about the memory model of the language it's compiling. So it can enforce rules that guarantee you handle memory safely. But it knows nothing about your other effects. The most it can do is force you to annotate that an effect is happening. It can't help you verify correct behavior of that effect.
  - jiggawatts 2y ago
    The obvious next step is to give the compiler the information it needs. The effect types could contain constraints similar to how Rust traits work.
  - swiftcoder 2y ago
    The compiler knew next to nothing about the memory model before we started annotating lifetimes. Annotating effects is a potentially reasonable next step. (I say "potentially", because even just annotating lifetimes is already considered a DX nightmare by a lot of people)
    - gergo_barany 2y ago
      > The compiler knew next to nothing about the memory model before we started annotating lifetimes. Compilers for garbage compiled languages have always had to reason about the memory model. Compilers for languages with obligatory bounds checks have always had to reason about the memory model. Compilers for multi-threaded languages have always had to reason about the memory model. Even "good old C" compilers had to reason about the memory model when it came to volatile variables. Rust didn't invent the concept of a memory model, and Rust's lifetimes are only a small part of its memory model.
Animats 2y ago
> What is the actual problem that this effect-oriented approach addresses? Yes. The blog post is hazy on the actual problem. Effect containment is an attempt to limit spooky action at a distance. The idea is to constrain what happens way over there from negatively affecting what happens here. Most difficult bugs come from that. It helps to keep this in mind when propounding language theories. Globals and mutable references are the usual legit mechanisms for effects at a distance. Less legit mechanisms are pointer manipulation in C and messing with the dictionaries of distant objects in Python. Non-legit mechanisms are subscripting out of range and dangling pointers. The trend is away from modifying mutable data. There's the pure functional approach, but then you have to do too many gyrations to have an effect on anything. Someone proposed single-assignment languages back in the 1980s, where each variable could be assigned to only once. That was considered silly at the time, but that's where we mostly ended up. In Rust, you write "let" far more often than you assign to a mutable variable. C++ has "auto". Everybody does it that way now, most of the time. It's roughly equivalent to a functional form, but things have names and there's a place to put dumping, logging, and profiling code. That's mostly where things have settled down. It's not a bad place to be. There's a whole other area of type theory that revolves around generics. This starts at "complicated" and ends at "incomprehensible". The lesson in this area, from LISP macros to C++ templates to the more exotic languages, is that it's easy to create unreadable code.
swiftcoder 2y ago
Effect systems and capability systems generally both try and solve very similar issues - getting a handle on the side effects of deeply nested and/or 3rd party code.
3836293648 2y ago
Only local effects (basically only mutation) are concealed

wavemode 2y ago

Effect systems have always felt to me like the functional programming community re-inventing dependency injection, but at the type level. At the term level, you could just let your function have a function as a parameter `read_file: IO -> String`, vs. annotating it with some kind of type-level "filesystem effect". And the former is a lot more flexible over time.

mitt_romney_12 2y ago
This is an interesting approach but it has some issues with more more advanced effects. For example if you have a function that uses mutable state you would need to have the state both as an input and an output (for the original and the updated state) `update_mut: State -> String -> State` this isn't very economical as is probably worse than just `update_mut: String -> State ()`. If you commit to just having the parameters act as "markers" that don't actually relate to the implementation then you can get away with `update_mut: State -> String -> ()` but then you use one of the big benefits of effect systems which is that you can change the way effects work and have multiple different implementations.
12_throw_away 2y ago
> Effect systems have always felt to me like the functional programming community re-inventing dependency injection Yeah, so, in python, I've been playing around with "structured dependency injection", basically DI + ideas from structured concurrency (i.e., dependency injection bound to well-defined lexical scopes). And whenever I read about effects systems, it definitely feels like they're talking about, if not the same thing, at least something in the same nuclear family of things.

fwouts 2y ago

Unless I completely misunderstood the post, a function that calls another function with effects will indeed be "infected". That is, unless the effect ends up being local (such as passing a mutable reference to a local variable).

sparkie 2y ago

> The effects of methods that are called by other methods will be concealed, so at the calling site, you'll have no idea that the high-level API `do_stuff` is going to utilize `io`. The point is to make the caller aware of any side-effects so that the callee can't conceal them. > That is, unless the effect decoration "infects" the caller like `async`, in which case every complex API method will be decorated with a huge number of effects. Effects are infectious, precisely because if `foo` has effects, which must be known to `bar` which call's `foo`, then `baz` which calls `bar` must also be aware of the effects of foo, else they would be hidden. > Simply put, what does this buy us? For one, it makes it explicit what can and cannot be done by some code, as a means to prevent obvious programmer errors. You might consider it analogous to a static versus dynamic type system. In the dynamic system we can call `(foo 123)`, even though `foo` might expect strings rather than integers - and we get a failure at runtime. In a static typing system, we can catch this error much earlier - at compile time. `foo(s : String)` makes it clear what foo expects. Effects attempt to augment functions not only with the type of value they expect, but also with the capabilities that the function has. If a function is pure, it's not capable of going into your filesystem and deleting data, for instance. While the effects are infective, the languages encourage you to write as much as you can using pure functions and only use effects where strictly necessary. Essentially, they follow the principle of least privilege[1]. There are ways to avoid infecting a program with the `IO` effect where only "some" IO is needed. For instance, in Clean, which uses uniqueness types instead of effects, a function which writes to a particular file can be given the privilege for just this file, eg `write_foo : *File String -> *File`. That would prevent `write_foo` from opening up another file and writing to it. The file itself must be opened using a `*Filesystem` type, which `*World` - the primordial source of uniqueness for your program implements, which is passed into the program's entry point. The other advantage referential transparency gives us is optimization. If a function is called multiple times with the same arguments, it always produces the same result. If we can detect at compile time that a function is called more than once with the same arguments, we can cache the result of the first call, and replace the second call with the cached value. This same optimization can't be done with unknown side-effects because those side-effects may be desirable - we don't want the compiler to attempt to remove them. Clean's uniqueness types give us another opportunity for optimization. A uniqueness type is guaranteed by the compiler to not be aliased, which means that we can mutate a value in place whilst retaining referential transparency, though only in a single-threaded manner. In regards to threading, threads are also side-effects. If we call into some library function written by someone else, we don't want their functions to start spawning threads and potentially mutating values we give it, else it could cause any number of race conditions. If any functions which may spawn a thread have a big red flag on them, we can avert these problems earlier, rather than finding out in production and spending a lot of time debugging. Another advantage that purity provides (but not uniqueness) is the ability to reuse parts of data structures for multiple values. For example, if we have a linked list `l`, we can write both `x = cons foo l` and `y = cons bar l`, to obtain two new lists which both have `l` as their tail, but they both refer to the same `l` in memory. It's safe to do this because we can't mutate this tail - we wouldn't want a mutation of the list `x` to also mutate `y`. In a language which doesn't prevent arbitrary side-effects, we would need to make a copy of `l` when creating `x` and `y` to prevent this from occurring. Purely functional data structures have many uses - if we want to keep a versioned record of states (like git) for example. Would strongly recommend reading Okasaki's Purely Functional Data Structures[2] for more insight. [1]:https://en.wikipedia.org/wiki/Principle_of_least_privilege https://en.wikipedia.org/wiki/Principle_of_least_privilege [2]:https://www.cs.tufts.edu/~nr/cs257/archive/chris-okasaki/dissertation.pdf https://www.cs.tufts.edu/~nr/cs257/archive/chris-okasaki/dis...

empath75 2y ago

Containing effects _in general_ is the natural next step after containing unsafe code. Memory safety issues are just one category of side effect that can have unpredictable effects on your code -- any time your code interacts with the "outside world" there's an opportunity for unpredictable behavior, and the more you can quarantine it, the easier it is to reason about the remaining code.

wavemode 2y ago
The difference is that a compiler knows all about the memory model of the language it's compiling. So it can enforce rules that guarantee you handle memory safely. But it knows nothing about your other effects. The most it can do is force you to annotate that an effect is happening. It can't help you verify correct behavior of that effect.
- jiggawatts 2y ago
  The obvious next step is to give the compiler the information it needs. The effect types could contain constraints similar to how Rust traits work.
- swiftcoder 2y ago
  The compiler knew next to nothing about the memory model before we started annotating lifetimes. Annotating effects is a potentially reasonable next step. (I say "potentially", because even just annotating lifetimes is already considered a DX nightmare by a lot of people)
  - gergo_barany 2y ago
    > The compiler knew next to nothing about the memory model before we started annotating lifetimes. Compilers for garbage compiled languages have always had to reason about the memory model. Compilers for languages with obligatory bounds checks have always had to reason about the memory model. Compilers for multi-threaded languages have always had to reason about the memory model. Even "good old C" compilers had to reason about the memory model when it came to volatile variables. Rust didn't invent the concept of a memory model, and Rust's lifetimes are only a small part of its memory model.

Animats 2y ago

> What is the actual problem that this effect-oriented approach addresses? Yes. The blog post is hazy on the actual problem. Effect containment is an attempt to limit spooky action at a distance. The idea is to constrain what happens way over there from negatively affecting what happens here. Most difficult bugs come from that. It helps to keep this in mind when propounding language theories. Globals and mutable references are the usual legit mechanisms for effects at a distance. Less legit mechanisms are pointer manipulation in C and messing with the dictionaries of distant objects in Python. Non-legit mechanisms are subscripting out of range and dangling pointers. The trend is away from modifying mutable data. There's the pure functional approach, but then you have to do too many gyrations to have an effect on anything. Someone proposed single-assignment languages back in the 1980s, where each variable could be assigned to only once. That was considered silly at the time, but that's where we mostly ended up. In Rust, you write "let" far more often than you assign to a mutable variable. C++ has "auto". Everybody does it that way now, most of the time. It's roughly equivalent to a functional form, but things have names and there's a place to put dumping, logging, and profiling code. That's mostly where things have settled down. It's not a bad place to be. There's a whole other area of type theory that revolves around generics. This starts at "complicated" and ends at "incomprehensible". The lesson in this area, from LISP macros to C++ templates to the more exotic languages, is that it's easy to create unreadable code.

swiftcoder 2y ago

Effect systems and capability systems generally both try and solve very similar issues - getting a handle on the side effects of deeply nested and/or 3rd party code.

3836293648 2y ago

Only local effects (basically only mutation) are concealed