4 ms·

I put here an aside: All this is super-complicated for most use-cases. What I have never used, but is inferred from https://en.wikipedia.org/wiki/List_of_concu

by mamcx 3y ago

I put here an aside: All this is super-complicated for most use-cases. What I have never used, but is inferred from https://en.wikipedia.org/wiki/List_of_concurrent_and_parallel_programming_languages https://en.wikipedia.org/wiki/List_of_concurrent_and_paralle... and in special the original CSP is how all this lacks total control in build your network patterns. In special, is my dream to have a language that allows to express better what is explained starting here: https://zguide.zeromq.org/docs/chapter2/ https://zguide.zeromq.org/docs/chapter2/ There is not (AFAIK) languages that make nice to properly wire this AND keep the semantic understanding/composability of it. For example, is hard to know if a function if is running in a thread/process, what is their priority, etc. I imagine that a language with orchestration (like Erlang/Elixir) in a explicit way will be far easier, but is hard to figure how do it with good composability

frumiousirc 3y ago
For static composability, a "ported graph" model makes a lot of sense. This model has graph nodes representing a unit of processing with its input and output data coming through labeled "ports" and edges connecting the ports. Subgraphs can be defined which then appear as a single node in constructing a larger graph (the composability). Such a model can then be mapped to an implementation (ie, used as configuration) in an abstract way that leaves the ability to select the data transport (edges) and graph execution independently from the graph itself. That is, the same graph may execute in a single multi-thread (MT) process or as a multi-process (MP) distributed application. Most simplistically, ports are designated as "input" or "output" (from point of view of the node). But, these designations can be expanded so that ports can carry designations matching the ZeroMQ socket patterns. The implementation can then leverage the transport agnosticity of ZeroMQ when supporting adaptation to MT vs MP but with the cost that all edges require data serialization instead of pointer passing.

by mamcx 3y ago

frumiousirc 3y ago

For static composability, a "ported graph" model makes a lot of sense. This model has graph nodes representing a unit of processing with its input and output data coming through labeled "ports" and edges connecting the ports. Subgraphs can be defined which then appear as a single node in constructing a larger graph (the composability). Such a model can then be mapped to an implementation (ie, used as configuration) in an abstract way that leaves the ability to select the data transport (edges) and graph execution independently from the graph itself. That is, the same graph may execute in a single multi-thread (MT) process or as a multi-process (MP) distributed application. Most simplistically, ports are designated as "input" or "output" (from point of view of the node). But, these designations can be expanded so that ports can carry designations matching the ZeroMQ socket patterns. The implementation can then leverage the transport agnosticity of ZeroMQ when supporting adaptation to MT vs MP but with the cost that all edges require data serialization instead of pointer passing.