4 ms·
Can we avoid the overhead of starting and shutting down processes by running a single Python process and communicating using something like grpc [0] (or even JS
by lbn 10y ago
Can we avoid the overhead of starting and shutting down processes by running a single Python process and communicating using something like grpc [0] (or even JSON-RPC for maximum simplicity)?
How do web frameworks like Flask handle multiple concurrent requests? Would performance increase if we started multiple instances of this Python web server on the same machine and load balanced them? The code would be much simpler if there was no need to handle process management.
[0]: http://www.grpc.io/ http://www.grpc.io/
- klibertp 10y ago> Can we avoid the overhead of starting and shutting down processes by running a single Python process and communicating using something like grpc Actually, that's how ErlPort works. You start a Python process and then you call some functions inside that process. How many functions you'd like to run using a single Python process is up to you - you can spawn a new worker for every call or spawn only one worker and stick to it unless it's killed somehow. You can also register callbacks on either side (Elixir or Python) and with a bit more effort you can make Python process accept normal Elixir messages and answer in the same way. > How do web frameworks like Flask handle multiple concurrent requests? In my experience, they mostly rely on uWSGI and a pool of processes...
- deleted 10y ago[deleted]
- jeremyjh 10y agoThe whole point here is we need parallel processing, and Python cannot provide this in a single process due to the GIL. Flask applications depend on the workload being I/O bound, so it can achieve concurrency where parallelism is not required. If you built a process that is CPU bound in Python code in Flask, you'd find it could not achieve much concurrency at all.
- bpicolo 10y agoThere are plenty of multiprocessing wsgi servers, flask builtin included