3 ms·
Great work and kudos to the Ray team! It's definitely a fresh look with a lot of lessons learned from previous generations (e.g. spark). There are a few nice
by phissenschaft 5y ago
Great work and kudos to the Ray team!
It's definitely a fresh look with a lot of lessons learned from previous generations (e.g. spark).
There are a few nice features I wish Ray would eventually get to.
On the user experience side, it would be nice to have task level logs: often time it's easier for users to reason at task level, especially the task is a facade that triggers other complicated library/subprocess calls.
For the scheduler, if there's more native support for sharded/bundled/partitioned tasks and
https://cloud.google.com/blog/products/gcp/no-shard-left-behind-dynamic-work-rebalancing-in-google-cloud-dataflow https://cloud.google.com/blog/products/gcp/no-shard-left-beh...