Ray is a distributed computing framework that has a fast scheduler, 0 copy / serialization between tasks (shared memory) and stateful "actors", making it great

Ray is a distributed computing framework that has a fast scheduler, 0 copy / serialization between tasks (shared memory) and stateful "actors", making it great for RL but it's more general than that. I'd recommend checking out their architecture whitepaper: https://docs.google.com/document/d/1tBw9A4j62ruI5omIJbMxly-la5w4q_TjyJgJL_jN2fI/preview?tab=t.0#heading=h.iyrm5j2gcdoq https://docs.google.com/document/d/1tBw9A4j62ruI5omIJbMxly-l... Imagine spark without the JVM baggage and with no need to spill to disk / serialize between steps unless it's necessary.