5 ms·

This sounds interesting: > Claro’s declarative, DAG-based structured concurrency model provides an easy-to-use abstraction to statically guarantee that program

This sounds interesting: > Claro’s declarative, DAG-based structured concurrency model provides an easy-to-use abstraction to statically guarantee that programs are: Non-blocking Data-race free Deadlock free Optimally scheduled Scalable by default Any language that can simplify the static guarantees of all those is worth a look.

kaba0 3y ago
I don’t think deadlock free is possible without some very serious limitations. Like not even share-nothing, communicate via messages only is safe from deadlocks.
- bmitc 3y ago
  You can deadlock processes in Erlang/Elixir, but as far as I know, that deadlock will not block all the other processes running.
  - kaba0 3y ago
    I mean, that’s how it works in Java as well (other threads keep running), but you are not much helped if your “critical business logic” deadlocked and doesn’t do its job. Though I guess you can try restarting it with Erlang.
- saghm 3y ago
  From glancing over their guide, it seems like it achieves this by essentially requiring you to manually specify a state machine that it can verify won't cycle by composing together "graph functions" that return futures (which are not allowed to block on the results of any futures directly or transitively). If I understand correctly, combining this with only allowing immutable data to be shared (like you described) makes it possible to parallelize graph function calls when that don't depend on each others results.
  - kaba0 3y ago
    Well, that’s true. Though then it is pretty similar to the “build tool” problem, which is actually an interesting parallel :D Nonetheless, I do think that big limitations at most places, with optional, explicit escape hatches is a good pattern.
    - jasonsteving 3y ago
      Definitely a clear similarity to the problems that incremental build tools (i.e. Bazel) are solving! At the moment, Claro doesn't allow any "escape hatches" but the logic there has really been to start with a strong/safe foundation, and then afterwards evaluate whether escape hatches are necessary, and if so what they should look like. At the moment, with specific regard to Claro's concurrency story, I'd say it's unlikely that there will be any escape hatches for writing obviously thread-unsafe code whatsoever explicitly available to user code. It's worth taking a look at this section of the docs to see Claro's current stance on the StdLib itself being given the unique right to "bless" certain mutable data structures to be shared between threads after they've been explicitly verified to be thread-safe. https://docs.clarolang.com/guaranteed_data_race_free/guaranteed_data_race_free.generated_docs.html#thread-safe-mutable-data-structures-blessed-by-the-stdlib https://docs.clarolang.com/guaranteed_data_race_free/guarant...
  - rakoo 3y ago
    From their doc, immutability is used for avoiding data races. Deadlocks are avoided by only starting a procedure when all dependencies are ready.
    - saghm 3y ago
      Yeah, I didn't mean to imply that immutability was needed for avoiding deadlocks; I just thought it was interesting that it "composes" with their deadlock avoidance to provide the follow-up "optimally scheduled" and "scalable by default" features.
  - jasonsteving 3y ago
    This is basically spot on. I'm glad you were able to come away with that understanding from your initial read of the docs.
gleenn 3y ago
I wonder how constrained the code has to be to satisfy all those properties. Anyone had any experience using this?
- jasonsteving 3y ago
  Hey! I wrote Claro so I can speak to this :). There are certainly some constraints to get these properties. To eliminate deadlocking, Claro tracks any explicit "blocking" on the completion of a future<T> and requires the containing procedure (and any dependents) to be explicitly annotated as "blocking", and then these blocking procedures aren't allowed to be called within a Graph procedure OR be scheduled on the Executor directly in the form of a lambda for example. Data races are prevented by simply making it impossible for two threads to share a reference to mutable data. The most obvious consequence here is that mutable data can't be passed into a Graph Procedure, and none of its internal nodes can evaluate to a mutable data type. Less obvious is that lambdas cannot "capture" mutable data. If they were allowed to capture mutable data, then they would effectively become stateful "objects" themselves in a way that the type system doesn't track (a `function<int -> int>` has the same signature regardless of what it captures) so this would effectively be a backdoor to allowing mutable data to leak between threads. More details here: https://docs.clarolang.com/fearless_concurrency/fearless_concurrency.html https://docs.clarolang.com/fearless_concurrency/fearless_con...
thesz 3y ago
SQL with SERIALIZABLE isolation level. It also has an upshot that you can execute your quieries on GPU [1]. [1] https://github.com/BlazingDB/blazingsql https://github.com/BlazingDB/blazingsql
- jonesnc 3y ago
  What does this have to do with Claro?
  - thesz 3y ago
    It has to do with the following part of the comment I replied to: > guaranteed... that programs are: Non-blocking Data-race free Deadlock free Optimally scheduled Scalable by default And the call that "any language that does this worth a look." SQL (transaction processing) does all this and more for quite some time [1] now. [1] https://cs.yale.edu/homes/thomson/publications/calvin-sigmod12.pdf https://cs.yale.edu/homes/thomson/publications/calvin-sigmod...
jasonsteving 3y ago
Thanks for your interest in this! Claro's Graph Procedures are certainly something I'm very proud of. I think they provide a powerful concurrency abstraction that limit the spread of concurrency-related complexity in a way that I find very enjoyable to work with.

kaba0 3y ago

I don’t think deadlock free is possible without some very serious limitations. Like not even share-nothing, communicate via messages only is safe from deadlocks.

bmitc 3y ago
You can deadlock processes in Erlang/Elixir, but as far as I know, that deadlock will not block all the other processes running.
- kaba0 3y ago
  I mean, that’s how it works in Java as well (other threads keep running), but you are not much helped if your “critical business logic” deadlocked and doesn’t do its job. Though I guess you can try restarting it with Erlang.
saghm 3y ago
From glancing over their guide, it seems like it achieves this by essentially requiring you to manually specify a state machine that it can verify won't cycle by composing together "graph functions" that return futures (which are not allowed to block on the results of any futures directly or transitively). If I understand correctly, combining this with only allowing immutable data to be shared (like you described) makes it possible to parallelize graph function calls when that don't depend on each others results.
- kaba0 3y ago
  Well, that’s true. Though then it is pretty similar to the “build tool” problem, which is actually an interesting parallel :D Nonetheless, I do think that big limitations at most places, with optional, explicit escape hatches is a good pattern.
  - jasonsteving 3y ago
    Definitely a clear similarity to the problems that incremental build tools (i.e. Bazel) are solving! At the moment, Claro doesn't allow any "escape hatches" but the logic there has really been to start with a strong/safe foundation, and then afterwards evaluate whether escape hatches are necessary, and if so what they should look like. At the moment, with specific regard to Claro's concurrency story, I'd say it's unlikely that there will be any escape hatches for writing obviously thread-unsafe code whatsoever explicitly available to user code. It's worth taking a look at this section of the docs to see Claro's current stance on the StdLib itself being given the unique right to "bless" certain mutable data structures to be shared between threads after they've been explicitly verified to be thread-safe. https://docs.clarolang.com/guaranteed_data_race_free/guaranteed_data_race_free.generated_docs.html#thread-safe-mutable-data-structures-blessed-by-the-stdlib https://docs.clarolang.com/guaranteed_data_race_free/guarant...
- rakoo 3y ago
  From their doc, immutability is used for avoiding data races. Deadlocks are avoided by only starting a procedure when all dependencies are ready.
  - saghm 3y ago
    Yeah, I didn't mean to imply that immutability was needed for avoiding deadlocks; I just thought it was interesting that it "composes" with their deadlock avoidance to provide the follow-up "optimally scheduled" and "scalable by default" features.
- jasonsteving 3y ago
  This is basically spot on. I'm glad you were able to come away with that understanding from your initial read of the docs.

gleenn 3y ago

I wonder how constrained the code has to be to satisfy all those properties. Anyone had any experience using this?

jasonsteving 3y ago
Hey! I wrote Claro so I can speak to this :). There are certainly some constraints to get these properties. To eliminate deadlocking, Claro tracks any explicit "blocking" on the completion of a future<T> and requires the containing procedure (and any dependents) to be explicitly annotated as "blocking", and then these blocking procedures aren't allowed to be called within a Graph procedure OR be scheduled on the Executor directly in the form of a lambda for example. Data races are prevented by simply making it impossible for two threads to share a reference to mutable data. The most obvious consequence here is that mutable data can't be passed into a Graph Procedure, and none of its internal nodes can evaluate to a mutable data type. Less obvious is that lambdas cannot "capture" mutable data. If they were allowed to capture mutable data, then they would effectively become stateful "objects" themselves in a way that the type system doesn't track (a `function<int -> int>` has the same signature regardless of what it captures) so this would effectively be a backdoor to allowing mutable data to leak between threads. More details here: https://docs.clarolang.com/fearless_concurrency/fearless_concurrency.html https://docs.clarolang.com/fearless_concurrency/fearless_con...

thesz 3y ago

SQL with SERIALIZABLE isolation level. It also has an upshot that you can execute your quieries on GPU [1]. [1] https://github.com/BlazingDB/blazingsql https://github.com/BlazingDB/blazingsql

jonesnc 3y ago
What does this have to do with Claro?
- thesz 3y ago
  It has to do with the following part of the comment I replied to: > guaranteed... that programs are: Non-blocking Data-race free Deadlock free Optimally scheduled Scalable by default And the call that "any language that does this worth a look." SQL (transaction processing) does all this and more for quite some time [1] now. [1] https://cs.yale.edu/homes/thomson/publications/calvin-sigmod12.pdf https://cs.yale.edu/homes/thomson/publications/calvin-sigmod...

jasonsteving 3y ago

Thanks for your interest in this! Claro's Graph Procedures are certainly something I'm very proud of. I think they provide a powerful concurrency abstraction that limit the spread of concurrency-related complexity in a way that I find very enjoyable to work with.