4 ms·

This seems pretty cool, but I'm left with one question. In the example from the repo it states, "The following computes Pi in parallel, using any number of mach

This seems pretty cool, but I'm left with one question. In the example from the repo it states, "The following computes Pi in parallel, using any number of machines and processors." However, after reading through all the docs, I see no reference whatsoever to any multi-machine support, only multi-processing on a single machine. Can this span over a cluster? Although they make the claim that it will "scale to hundreds of thousands of cores" (which I want to believe, I would love to use this) without the API documentation to show me how to do this, their fancy library doesnt do much good.

juanjgalvez 8y ago
Hi. Yes, applications can span multiple nodes (e.g. in clusters and supercomputers), and is one of the main use cases of charm++/charmpy. The fact that you don't see anything in the API or examples is that application code is basically transparent to the amount of processes that are launched. What determines the number of processes used is the launcher (e.g. charmrun, or something like aprun or ibrun on other systems). During initialization, the charmpy runtime will figure out internally how many charmpy processes are active in the job. With charmrun, you can launch multiple processes in one host, but also across multiple hosts (by ssh'ing into each one and spawning the processes). This is done automatically by charmrun assuming you specify a list of hosts (called nodelist, see http://charm.cs.illinois.edu/manuals/html/charm++/C.html http://charm.cs.illinois.edu/manuals/html/charm++/C.html). Again, the application code is not affected by this. Similarly, on other systems you can launch charmpy applications with the system job launcher (e.g. aprun, sbatch, ibrun…). We have done so for example on Cray supercomputers. It is simple enough but we have to update the documentation to at least show an example of this.
- wedn3sday 8y ago
  Thanks for the response! This is great, makes it pretty much a better version of mpi. Could you add this to the documentation, or if its already there maybe a link on the main doc page about running on a cluster?

juanjgalvez 8y ago

Hi. Yes, applications can span multiple nodes (e.g. in clusters and supercomputers), and is one of the main use cases of charm++/charmpy. The fact that you don't see anything in the API or examples is that application code is basically transparent to the amount of processes that are launched. What determines the number of processes used is the launcher (e.g. charmrun, or something like aprun or ibrun on other systems). During initialization, the charmpy runtime will figure out internally how many charmpy processes are active in the job. With charmrun, you can launch multiple processes in one host, but also across multiple hosts (by ssh'ing into each one and spawning the processes). This is done automatically by charmrun assuming you specify a list of hosts (called nodelist, see http://charm.cs.illinois.edu/manuals/html/charm++/C.html http://charm.cs.illinois.edu/manuals/html/charm++/C.html). Again, the application code is not affected by this. Similarly, on other systems you can launch charmpy applications with the system job launcher (e.g. aprun, sbatch, ibrun…). We have done so for example on Cray supercomputers. It is simple enough but we have to update the documentation to at least show an example of this.

wedn3sday 8y ago
Thanks for the response! This is great, makes it pretty much a better version of mpi. Could you add this to the documentation, or if its already there maybe a link on the main doc page about running on a cluster?