5 ms·

Wouldn’t you lose, for example, streaming capabilities once you use Celery? You would have to first upload the whole video, then enqueue the job, and then figur

Wouldn’t you lose, for example, streaming capabilities once you use Celery? You would have to first upload the whole video, then enqueue the job, and then figure out a mechanism to send the thumbnails back to that client, while with FLAME you get a better user experience by streaming thumbnails as soon as the upload starts. I believe the main point though is that background workers and FLAME are orthogonal concepts. You can use FLAME for autoscaling, you can use Celery for durability, and you could use Celery with FLAME to autoscale your background workers based on queue size. So being able to use these components individually will enable different patterns and use cases.

seabrookmx 3y ago
Yes fair point. Celery has to pickle parameters so that they can flow through redis or rabbit to the worker pool. It's worth pointing out this transparent remote function call ability is unique to the BEAM. The FLAME pattern in other languages (as described in the article with Javascript) would also require serializable parameters. > I believe the main point though is that background workers and FLAME are orthogonal concepts Yeah I think this is what the author is driving at. I appreciate you helping me try and wrap my brain around it :)