3 ms·
If you're having RAM problems, you don't want to start up each processor independently as a "port". You'd want to write a Python server that offers a socket, st
by jerf 10y ago
If you're having RAM problems, you don't want to start up each processor independently as a "port". You'd want to write a Python server that offers a socket, start one instance of that that listens to the socket and then forks (since you control the system fully, probably let it fork once per connection rather than worrying about "preforking" or other complicated scenarios for when you don't control the concurrency), and then write the glue code to talk across that socket. [1]
That way, your Python processes load all their modules and do all their initialization, and the forking automatically takes care of Copy-On-Write semantics in the RAM and you end up with only more-or-less one copy of them in RAM.
[1]: I don't know if there's a "perfect" off-the-shelf solution for you, but there's enough existing code in the world with all the pieces that it's pretty easy to do that nowadays. For instance, one very easy protocol that leaps to mind is:
4 bytes to say how much JSON there is
a JSON object containing the metadata for your connection
4 or 8 bytes to say how long the incoming webpage is
the contents of the webpage you're telling Python to process
It's not perfectly efficient, but it has great bang-for-the-buck, and will carry you a long way before you start needing to do anything else fancier.
You might also be able to bash ErlPort into compliance and get the client and server side going, but, well, this isn't very difficult code to write and it doesn't take much "screwing around with opaque library code that doesn't really want to be used separately" before you could have just written this.