3 ms·

> Have a simple webserver and want to do some things out of band? In jvm I would use a concurrent queue and add tasks to it, and spin up some threads consuming

> Have a simple webserver and want to do some things out of band? In jvm I would use a concurrent queue and add tasks to it, and spin up some threads consuming that. In python you either spin up a new process and deal with the communication problems, become a master of greenlet and python internals, or most likely default to celery which need it's own deployment. This is probably risky in production anyway, because the load balancer (typically) has no insight into these background tasks and will happily kill a process/pod/etc that is running a background process. You should probably dispatch the workload to an external task runner (e.g., Lambda or a Kubernetes Job or similar) unless you really don't care if the background task gets killed mid-flight. (I've had a few dev teams ignore these warnings and then blame infrastructure when their background jobs got killed mid-flight occasionally).

matsemann 4y ago
Yes, that is true. My case here was about pushing some metrics somewhere, where losing some weren't an issue, but I didn't want to block main thread doing it. But this single java app on some EC2 instance could do what you need 10 "apps" and possibly a complicated k8s deployment to handle with python. Some just because the raw performance of python is far worse, but most of it because of cases like this, where simple things can't be shared. So lots of unnecessary complexity compared to "old and verbose" java. Another example is prometheus metrics. The current app I'm working on doesn't have a webserver. Which makes it really awkward in python to add prometheus. Since adding an endpoint to my app creates a new process and needs to be deployed almost as a sidecart, there is no smooth way to actually get the metrics from my main app to the endpoint that can be scraped since they don't share the same process.

matsemann 4y ago

Yes, that is true. My case here was about pushing some metrics somewhere, where losing some weren't an issue, but I didn't want to block main thread doing it. But this single java app on some EC2 instance could do what you need 10 "apps" and possibly a complicated k8s deployment to handle with python. Some just because the raw performance of python is far worse, but most of it because of cases like this, where simple things can't be shared. So lots of unnecessary complexity compared to "old and verbose" java. Another example is prometheus metrics. The current app I'm working on doesn't have a webserver. Which makes it really awkward in python to add prometheus. Since adding an endpoint to my app creates a new process and needs to be deployed almost as a sidecart, there is no smooth way to actually get the metrics from my main app to the endpoint that can be scraped since they don't share the same process.