AKA, "model serving". The reason is that the kernel gives you this featuer and it's really powerful, so why not take advantage of it? During dev work you ofte

AKA, "model serving". The reason is that the kernel gives you this featuer and it's really powerful, so why not take advantage of it? During dev work you often want an easily restarted stack (code changes). Anyway, if you use this approach, you can just have the python server stay resident and then have another app mmap it (shared memory) instead of doing inference over an API, whcih is always awkward.