3 ms·

This looks like a nice language with lots of cool features. I especially like the attempts at GPU and off-heap integration. However I feel like at least the la

This looks like a nice language with lots of cool features. I especially like the attempts at GPU and off-heap integration. However I feel like at least the landing page dodges a lot of hard questions. For example the biggest reason anyone should want to write a concurrent program over a synchronous single-threaded one is performance, so talking about how your features are implemented efficiently is important. Specific examples: Having my isolates semantically copy all their inputs sure might avoid race conditions, but I hope that there's some fancy copy-on-write and sub-slicing stuff going on so that if I spawn 10 threads to process 10 parts of my array I don't make 10 copies of the whole array. For GPU support I want to The off-heap support mentions "all objects are quickly serializable and deserializable to and from this format by default". Anyone can claim "quickly", I want to know exactly how efficient this serialization is. Does it use zero-copy casting with some patching? Is it just a memcpy to the GC heap? Does each type get code-generated serializers/deserializers or maybe it's done with reflection? Whether this feature is useful entirely depends on exactly how fast this serialization is. How does the remote code execution transmit code? Do I need to make sure that my work units are big enough that every remote JVM can warm up and JIT transmitted bytecode? Is there caching so I don't transmit the standard library every time?

jtatton 6y ago
Hi thanks for raising this. I will add more details around benchmarking and the points you have raised to the documentation. Some answers for now: - One may use the `shared` keyword in order to override the default copying semantic of Concurnas for mutable data. In this way one may spawn 10 isolates operating on an array without copying it 10 times. Of course this may introduce non deterministic behavior if they overlap in terms of how data is proceed in that array, so care must be applied. Also, a class may be marked as being shared, again suppressing copying, this is handy if if one is importing already thread safe Java code. - In terms of off heap support all classes are augmented with runtime generated serializers and deserializers (https://concurnas.com/docs/offHeap.html#serialization-of-objects https://concurnas.com/docs/offHeap.html#serialization-of-obj...). Off heap memory support is an area where I'd like to improve Concurnas some more. I think that some form of value type support would be beneficial here. The goal being for it to be possible for one to implement an efficient RDBMS or NoSQL system in Concurnas itself. - For remote computing, yes there is caching of code to avoid retransmission (the documentation could be better regarding this, I will update the following in due course: https://concurnas.com/docs/distComp.html#request-dependencies https://concurnas.com/docs/distComp.html#request-dependencie...) - The point about work unit size for remote computing is interesting, I wonder if there is a framework/general method for solving this sort of problem (feels like a solved problem).
- viluon 6y ago
  The solution to avoiding unnecessary copying via `shared` sounds naive to me. I don't want the language's safety guarantees to disappear once I try to add such simple optimisations, especially when writing high-performance concurrent code.
- krapht 6y ago
  This isn't really easy to solve. I'm only (a little) familiar with Chapel which I think is the most widely used high level HPC language; how data is distributed needs a lot of thought and is specified by the programmer according to language constructs of "locale" and "distribution". The other hard problem an HPC language needs to handle is error handling. What happens when a processor goes down, data isn't received in a timely fashion, etc. How a data processing job might get rerouted to a different server and how intermediate data calculations might get checkpointed so the loss of one result doesn't mean redoing the whole computation.
eikenberry 6y ago
> For example the biggest reason anyone should want to write a concurrent program over a synchronous single-threaded one is performance[..] I don't agree with this sentiment. Concurrency is a great abstraction, even if not run in parallel. After using CSP (in Go) for a time I don't care for other (mainstream) languages as much due to their lack of good concurrency primitives. I love writing programs as a bunch of services that work together. Feels like shell/pipe scripting or having daemons work together over sockets. Fits my brain very well.
- macintux 6y ago
  And Erlang relies on concurrency for robustness, not performance.

jtatton 6y ago

Hi thanks for raising this. I will add more details around benchmarking and the points you have raised to the documentation. Some answers for now: - One may use the `shared` keyword in order to override the default copying semantic of Concurnas for mutable data. In this way one may spawn 10 isolates operating on an array without copying it 10 times. Of course this may introduce non deterministic behavior if they overlap in terms of how data is proceed in that array, so care must be applied. Also, a class may be marked as being shared, again suppressing copying, this is handy if if one is importing already thread safe Java code. - In terms of off heap support all classes are augmented with runtime generated serializers and deserializers (https://concurnas.com/docs/offHeap.html#serialization-of-objects https://concurnas.com/docs/offHeap.html#serialization-of-obj...). Off heap memory support is an area where I'd like to improve Concurnas some more. I think that some form of value type support would be beneficial here. The goal being for it to be possible for one to implement an efficient RDBMS or NoSQL system in Concurnas itself. - For remote computing, yes there is caching of code to avoid retransmission (the documentation could be better regarding this, I will update the following in due course: https://concurnas.com/docs/distComp.html#request-dependencies https://concurnas.com/docs/distComp.html#request-dependencie...) - The point about work unit size for remote computing is interesting, I wonder if there is a framework/general method for solving this sort of problem (feels like a solved problem).

viluon 6y ago
The solution to avoiding unnecessary copying via `shared` sounds naive to me. I don't want the language's safety guarantees to disappear once I try to add such simple optimisations, especially when writing high-performance concurrent code.
krapht 6y ago
This isn't really easy to solve. I'm only (a little) familiar with Chapel which I think is the most widely used high level HPC language; how data is distributed needs a lot of thought and is specified by the programmer according to language constructs of "locale" and "distribution". The other hard problem an HPC language needs to handle is error handling. What happens when a processor goes down, data isn't received in a timely fashion, etc. How a data processing job might get rerouted to a different server and how intermediate data calculations might get checkpointed so the loss of one result doesn't mean redoing the whole computation.

eikenberry 6y ago

> For example the biggest reason anyone should want to write a concurrent program over a synchronous single-threaded one is performance[..] I don't agree with this sentiment. Concurrency is a great abstraction, even if not run in parallel. After using CSP (in Go) for a time I don't care for other (mainstream) languages as much due to their lack of good concurrency primitives. I love writing programs as a bunch of services that work together. Feels like shell/pipe scripting or having daemons work together over sockets. Fits my brain very well.

macintux 6y ago
And Erlang relies on concurrency for robustness, not performance.