2 ms·
Here's an example. You're doing a big machine learning workflow and you've got a gazillion large files representing training data. You run a large training job
by CobrastanJorji 5y ago
Here's an example. You're doing a big machine learning workflow and you've got a gazillion large files representing training data. You run a large training job in Amazon's cloud somewhere that is composed of one worker that lists the files and delegates work to various learning jobs, and then those jobs each stream in one training file, process it, and then grab the next item and repeat. That's a pretty common type of work.