4 ms·

Memory is typically the scaling bottleneck for Python workers for web servers. One way to cut back drastically on this is to load as much as possible in startup

Memory is typically the scaling bottleneck for Python workers for web servers. One way to cut back drastically on this is to load as much as possible in startup and then fork for request serving processes. The problem is that python refcounting causes a lot of copy-on-writes for data that's really just used as read-only data. This change allows you to run more workers with less memory. Instagram has had a few good posts about how they've approached this problem, here's another: https://engineering.instagram.com/dismissing-python-garbage-collection-at-instagram-4dca40b29172 https://engineering.instagram.com/dismissing-python-garbage-...

sametmax 9y ago
> Memory is typically the scaling bottleneck for Python workers for web servers. Wut ? When I have performances problems, my RAM is full with varnish cache, redis stored objects and postgres buffer. Python is like, low, low on the list. Again, most people are NOT instagram or Google. They have a very atypical load.
- bpicolo 9y ago
  That's latency, not bandwidth. This is to help scale out concurrent request counts per box, rather than time to serve a single request (though obviously with request queuing, failure to serve requests fast enough makes concurrency a latency bottleneck). And sure, most people aren't instagram. This is a python-at-big-scale problem, when the cost benefits of fitting more requests onto fewer servers actually matter. One of the biggest reasons this actually makes a difference is that, in SoA, internal API scaling against your monolith can become expensive.
- ComputerGuru 9y ago
  > Again, most people are NOT Instagram or Google And, again, can we stop with these pointless comparisons? Past 1 front end server the only cost that matters is how many requests your one node can handle and how much that one node costs. If you’re not a VPS and you have your frontend http cache correctly configured, then it doesn’t matter how much smaller than Google you are, comparisons are valid (although, of course, the fewer servers you have the more you can afford to spend on them; though you probably aren’t making as much money as IG/FB/Google either...).
  - sametmax 9y ago
    The only metrics is economic. The question is how much does those performance constrains cost you ?

sametmax 9y ago

> Memory is typically the scaling bottleneck for Python workers for web servers. Wut ? When I have performances problems, my RAM is full with varnish cache, redis stored objects and postgres buffer. Python is like, low, low on the list. Again, most people are NOT instagram or Google. They have a very atypical load.

bpicolo 9y ago
That's latency, not bandwidth. This is to help scale out concurrent request counts per box, rather than time to serve a single request (though obviously with request queuing, failure to serve requests fast enough makes concurrency a latency bottleneck). And sure, most people aren't instagram. This is a python-at-big-scale problem, when the cost benefits of fitting more requests onto fewer servers actually matter. One of the biggest reasons this actually makes a difference is that, in SoA, internal API scaling against your monolith can become expensive.
ComputerGuru 9y ago
> Again, most people are NOT Instagram or Google And, again, can we stop with these pointless comparisons? Past 1 front end server the only cost that matters is how many requests your one node can handle and how much that one node costs. If you’re not a VPS and you have your frontend http cache correctly configured, then it doesn’t matter how much smaller than Google you are, comparisons are valid (although, of course, the fewer servers you have the more you can afford to spend on them; though you probably aren’t making as much money as IG/FB/Google either...).
- sametmax 9y ago
  The only metrics is economic. The question is how much does those performance constrains cost you ?