5 ms·
I have never found a good example of a Python web server that provides some mechanism for statefulness. Is it just fundamentally not possible to have shared st
by Waterluvian 10y ago
I have never found a good example of a Python web server that provides some mechanism for statefulness. Is it just fundamentally not possible to have shared state among requests handled by the threads of a process? Sanic's examples seem to be the same as Flask's: self-contained function calls attached to endpoints.
I keep hitting a wall with Python when I want to do something like:
1. subscribe to a websocket connection and keep the last received message in state
2. expose an http endpoint to let a client GET that last message.
- quinnftw 10y agoYou can use caching to mimic this behaviour in Flask, I.e. http://flask.pocoo.org/docs/0.12/patterns/caching/ http://flask.pocoo.org/docs/0.12/patterns/caching/ I'm not sure how this works with multiple threads though, I imagine you would have to synchronize it yourself.
- tyingq 10y agoRedis's lpush, and rpop plus some naming scheme might suffice. Everything is atomic, so the thread bit is covered.
- deleted 10y ago[deleted]
- raziel2p 10y agoIs that really something you want to do in-memory? Once you have to start multiple worker processes or application servers behind a load balancer, you'll have to re-implement it with some sort of shared persistant store like Redis.
- icebraining 10y agoNot really, you can use a multiprocessing.Manager to share a plain old dict or list across multiple worker processes: https://docs.python.org/3/library/multiprocessing.html#sharing-state-between-processes https://docs.python.org/3/library/multiprocessing.html#shari...
- cwilkes 10y agoI think the OP was talking about different VMs behind a load balancer where there is no shared memory at all.
- icebraining 10y agoWasn't my impression (using worker processes in the same machine is common), but fair enough. On the other hand, message passing across machines is overrated. We run a SaaS service on 25 VMs with no communication between them for regular operation.
- bobbyi_settv 10y agoYou normally use something like redis to store the state. If you were going to share state in memory between threads, how would you handle the case where the second request goes to a different server or that the process has restarted? You'd need redis anyway, so you might as well just use it in all cases.
- Waterluvian 10y agoI get that everyone's responses are thinking some big public thing. I'm thinking small toy implementation for my home network. The toy experiment is how to do what's trivial in Node with Python. Mainly because I like working with python. I think the answer might be: Python is the wrong tool for the job.
- rcarmo 10y agoErm, no. You can do shared thread storage, in Python, it's just that it doesn't really scale. I've done it for small daemons without significant hassle, and even wrote my own Go-like CSP helper: https://github.com/rcarmo/python-utils/blob/master/taskkit.py https://github.com/rcarmo/python-utils/blob/master/taskkit.p...
- zeptomu 10y agoThe problem is that accessing shared state concurrently in a multi-process context is a non-trivial problem, so specific software emerged that handles these problems for you. The simplest solution is to use a small DB system like sqlite. It is built into Python (import sqlite3) performs reasonably well and you do not have to run an additional service. Now if a small DB like sqlite already feels overblown to you (and it really is simple and small) you might not need concurrent access either, so the simplest solution is to just use a file where you store your state.
- icebraining 10y agoI'd say the simplest solution is a global (or just shared) variable using a thread-safe container like queue.Queue. There's also multiprocessing.Queue, which supports sharing the queue across multiple workers.
- weberc2 10y agoFlask lets you share state between requests; just have the route methods reference some global variable. You could also apply the route decorator to an instance method (although probably not using the decorator syntax).
- deleted 10y ago[deleted]
- chronial 10y agoI am a little confused here. What's keeping you from storing your state in a global variable?
- pdmccormick 10y agoYour request has stayed on my mind over the past few days, so I put this together for you: https://github.com/pdmccormick/sample-socketio-chat-app https://github.com/pdmccormick/sample-socketio-chat-app Enjoy!