5 ms·

You normally use something like redis to store the state. If you were going to share state in memory between threads, how would you handle the case where the s

You normally use something like redis to store the state. If you were going to share state in memory between threads, how would you handle the case where the second request goes to a different server or that the process has restarted? You'd need redis anyway, so you might as well just use it in all cases.

Waterluvian 9y ago
I get that everyone's responses are thinking some big public thing. I'm thinking small toy implementation for my home network. The toy experiment is how to do what's trivial in Node with Python. Mainly because I like working with python. I think the answer might be: Python is the wrong tool for the job.
- rcarmo 9y ago
  Erm, no. You can do shared thread storage, in Python, it's just that it doesn't really scale. I've done it for small daemons without significant hassle, and even wrote my own Go-like CSP helper: https://github.com/rcarmo/python-utils/blob/master/taskkit.py https://github.com/rcarmo/python-utils/blob/master/taskkit.p...
- zeptomu 9y ago
  The problem is that accessing shared state concurrently in a multi-process context is a non-trivial problem, so specific software emerged that handles these problems for you. The simplest solution is to use a small DB system like sqlite. It is built into Python (import sqlite3) performs reasonably well and you do not have to run an additional service. Now if a small DB like sqlite already feels overblown to you (and it really is simple and small) you might not need concurrent access either, so the simplest solution is to just use a file where you store your state.
  - icebraining 9y ago
    I'd say the simplest solution is a global (or just shared) variable using a thread-safe container like queue.Queue. There's also multiprocessing.Queue, which supports sharing the queue across multiple workers.
- bpicolo 9y ago
  Having multiple nodejs processes is the same thing as having multiple python processes w/ regards to sharing state. What you're referring to works equally well in the single-process case for both.
- deleted 9y ago
  [deleted]
- rbutcher 9y ago
  This maybe seems complicated because Node has 1 obvious way to run (single threaded with asynchronous functions) but Python has a few ways (single threaded, multithreaded, ioloops kind of like Node, greenlets). Python is excellent for toy implementations, and real ones too in many cases.
- rbutcher 9y ago
  This maybe seems complicated because Node has 1 obvious way to run (single threaded with asynchronous functions) but Python has a few ways (single threaded, multithreaded, ioloops kind of like Node, greenlets). Python is excellent for toy implementations, and real ones too in many cases.
- netheril96 9y ago
  Is multithreading really necessary for a toy?
  - mkj 9y ago
    It probably is for long-polling or websockets? https://github.com/mkj/wort-templog/blob/master/web/templog.py#L184 https://github.com/mkj/wort-templog/blob/master/web/templog.... is my not-quite-toy example - a single process runs from uwsgi with Bottle (like Flask) and gevent. The long polling waits on a global Event variable that's updated by another request, nice and simple.

Waterluvian 9y ago

I get that everyone's responses are thinking some big public thing. I'm thinking small toy implementation for my home network. The toy experiment is how to do what's trivial in Node with Python. Mainly because I like working with python. I think the answer might be: Python is the wrong tool for the job.

rcarmo 9y ago
Erm, no. You can do shared thread storage, in Python, it's just that it doesn't really scale. I've done it for small daemons without significant hassle, and even wrote my own Go-like CSP helper: https://github.com/rcarmo/python-utils/blob/master/taskkit.py https://github.com/rcarmo/python-utils/blob/master/taskkit.p...
zeptomu 9y ago
The problem is that accessing shared state concurrently in a multi-process context is a non-trivial problem, so specific software emerged that handles these problems for you. The simplest solution is to use a small DB system like sqlite. It is built into Python (import sqlite3) performs reasonably well and you do not have to run an additional service. Now if a small DB like sqlite already feels overblown to you (and it really is simple and small) you might not need concurrent access either, so the simplest solution is to just use a file where you store your state.
- icebraining 9y ago
  I'd say the simplest solution is a global (or just shared) variable using a thread-safe container like queue.Queue. There's also multiprocessing.Queue, which supports sharing the queue across multiple workers.
bpicolo 9y ago
Having multiple nodejs processes is the same thing as having multiple python processes w/ regards to sharing state. What you're referring to works equally well in the single-process case for both.
deleted 9y ago
[deleted]
rbutcher 9y ago
This maybe seems complicated because Node has 1 obvious way to run (single threaded with asynchronous functions) but Python has a few ways (single threaded, multithreaded, ioloops kind of like Node, greenlets). Python is excellent for toy implementations, and real ones too in many cases.
rbutcher 9y ago
This maybe seems complicated because Node has 1 obvious way to run (single threaded with asynchronous functions) but Python has a few ways (single threaded, multithreaded, ioloops kind of like Node, greenlets). Python is excellent for toy implementations, and real ones too in many cases.
netheril96 9y ago
Is multithreading really necessary for a toy?
- mkj 9y ago
  It probably is for long-polling or websockets? https://github.com/mkj/wort-templog/blob/master/web/templog.py#L184 https://github.com/mkj/wort-templog/blob/master/web/templog.... is my not-quite-toy example - a single process runs from uwsgi with Bottle (like Flask) and gevent. The long polling waits on a global Event variable that's updated by another request, nice and simple.