3 ms·

> These kinds of mistakes are easy to make and hard to debug, especially when the sleepable call is deeply nested. This means you either have to pass around pe

> These kinds of mistakes are easy to make and hard to debug, especially when the sleepable call is deeply nested. This means you either have to pass around permission-to-sleep tokens or you need additional function colors to make sure that it is enforced on the entire function call tree. The former might be possible with the lifetime trickery that chostcell does (I'm not certain, advanced type system hacks aren't my forte) but it's cumbersome. The latter would require an effects system which rust currently doesn't have.

JonChesterfield 4y ago
That helped a little, thanks. I can see how having an argument passed down the call tree is equivalent to colouring. Make sleep forms take the token, no token, no sleep. Maybe generic over counter embedded in the token, that's kind of ugly either way. "Effects system" seems to mean different things in different contexts. I found a few rust specific blog posts but they didn't obviously apply to this. Probably distinct from dependent types. The rust compiler has the counting machinery to do this, perhaps it's not surfaced to the language outside of built-in types (arc?). In fairness I can't think of many languages which are up for that sort of hooking into the compiler. It looks to me like a callgraph walk accumulating some state on known function calls plus error on taking address-of the mutator functions would work for sufficiently static graphs, which is probably what you have for careful concurrent code. Semantic linters running across llvm IR are probably useful things to have. Not really related to rust at that point though.
- iudqnolq 4y ago
  I think you were confused by someone who described that "counting machinery" at too high a level of abstraction. Compile-time counting is implemented in the compiler, runtime counting requires you to opt-in by wrapping the type in a special wrapper type. This wrapper type requires two special language features: a way to run code when a type is "dropped" (think destroyed), and a way to have a type forward method calls to another type. But any regular code can use these features. That lets you write a straightforward wrapper that forwards calls and manages the reference counting (increment on clone, decrement on drop). There's a bit more cleverness in the math, but it's all regular rust code. See <https://doc.rust-lang.org/src/alloc/rc.rs.html#15896 https://doc.rust-lang.org/src/alloc/rc.rs.html#15896> This lack of special standard-library-only magic is a key part of rust's appeal to me. Those same features let the crate parking_lot implement a better mutex than the standard library, for example.
eqvinox 4y ago
> This means you either have to pass around permission-to-sleep or you need additional function colors I wonder if it's possible to do this better. Somehow applying the type system onto the "execution context" feels like it would be the "ideal" solution here. From just 2 minutes of superficial thought, it would really be quite similar to permission-to-something tokens. Just a bunch of syntactic sugar on how it's done, plus making it easier to extend for additional tokens/colors/...?
- ameliaquining 4y ago
  I think you want something like https://tmandry.gitlab.io/blog/posts/2021-12-21-context-capabilities/ https://tmandry.gitlab.io/blog/posts/2021-12-21-context-capa.... The Rust maintainers have expressed interest in this possibility, but I don't think there's anything concrete yet.
  - JonChesterfield 4y ago
    That ^ is interesting in that it's (a syntax heavy version of) dynamic scoping with an implicit restriction that the call graph must be static. I.e. as described that does not work with function pointers (there is a todo reference to 'dyn', which looks like it means vtable dispatch, i.e. a special case of a function pointer call). This suggests a general pattern given the above on memory allocation - some things are expressible on static call graphs. On dynamic call graphs, they need to either fail (could be at compile time) or induce a runtime cost.

JonChesterfield 4y ago

That helped a little, thanks. I can see how having an argument passed down the call tree is equivalent to colouring. Make sleep forms take the token, no token, no sleep. Maybe generic over counter embedded in the token, that's kind of ugly either way. "Effects system" seems to mean different things in different contexts. I found a few rust specific blog posts but they didn't obviously apply to this. Probably distinct from dependent types. The rust compiler has the counting machinery to do this, perhaps it's not surfaced to the language outside of built-in types (arc?). In fairness I can't think of many languages which are up for that sort of hooking into the compiler. It looks to me like a callgraph walk accumulating some state on known function calls plus error on taking address-of the mutator functions would work for sufficiently static graphs, which is probably what you have for careful concurrent code. Semantic linters running across llvm IR are probably useful things to have. Not really related to rust at that point though.

iudqnolq 4y ago
I think you were confused by someone who described that "counting machinery" at too high a level of abstraction. Compile-time counting is implemented in the compiler, runtime counting requires you to opt-in by wrapping the type in a special wrapper type. This wrapper type requires two special language features: a way to run code when a type is "dropped" (think destroyed), and a way to have a type forward method calls to another type. But any regular code can use these features. That lets you write a straightforward wrapper that forwards calls and manages the reference counting (increment on clone, decrement on drop). There's a bit more cleverness in the math, but it's all regular rust code. See <https://doc.rust-lang.org/src/alloc/rc.rs.html#15896 https://doc.rust-lang.org/src/alloc/rc.rs.html#15896> This lack of special standard-library-only magic is a key part of rust's appeal to me. Those same features let the crate parking_lot implement a better mutex than the standard library, for example.

eqvinox 4y ago

> This means you either have to pass around permission-to-sleep or you need additional function colors I wonder if it's possible to do this better. Somehow applying the type system onto the "execution context" feels like it would be the "ideal" solution here. From just 2 minutes of superficial thought, it would really be quite similar to permission-to-something tokens. Just a bunch of syntactic sugar on how it's done, plus making it easier to extend for additional tokens/colors/...?

ameliaquining 4y ago
I think you want something like https://tmandry.gitlab.io/blog/posts/2021-12-21-context-capabilities/ https://tmandry.gitlab.io/blog/posts/2021-12-21-context-capa.... The Rust maintainers have expressed interest in this possibility, but I don't think there's anything concrete yet.
- JonChesterfield 4y ago
  That ^ is interesting in that it's (a syntax heavy version of) dynamic scoping with an implicit restriction that the call graph must be static. I.e. as described that does not work with function pointers (there is a todo reference to 'dyn', which looks like it means vtable dispatch, i.e. a special case of a function pointer call). This suggests a general pattern given the above on memory allocation - some things are expressible on static call graphs. On dynamic call graphs, they need to either fail (could be at compile time) or induce a runtime cost.