3 ms·

Context: I write mostly write Python for a living and run a Django app as a major responsibility I always wonder if python, auto scaling, and cloud change the

Context: I write mostly write Python for a living and run a Django app as a major responsibility I always wonder if python, auto scaling, and cloud change the kinds of businesses (and margins) required. Running 60+ pods of gunicorn for an app incurs lots of overhead, especially if one is using small VMs. I can't count the number of war stories I've heard from SRE friends who joined some company and realized they were taking over a Django stack for the core of a business that was blowing gaskets left and right. This sort of thing just leaves me wondering if margins have to be high to support a slower language, slower framework, and complicated deployment model to work around the Python GIL. Call me jaded but I just wonder if it's worth the time to market to do Python these days versus say Go and be able to deploy a static binary and use all your CPU cores simply. K8s is way less tempting for a "one-process-architecture " (minus database and maybe nginx) until the system is much larger (instead a couple of systemd units would sort you)

lifeisstillgood 2y ago
I’m not sure I understand - pythonnis “just” a process once you kick it off, want to run 8 processes on 8 CPUs, kick off 8 processes. Are you saying that to run this 8 processes you suddenly need 8 VMs running in the cloud - yeah. That does seem expensive. It’s very tempting to say one should not start in the cloud but host locally, but that seems an unpopular view. But maybe as we start to see more and more local workspaces catering to people working from home, we will see more and more “mini data centres” - it might even by Oxides sweet spot. Edit: “”” because it actually winds up being less DevOps work, on average, to support open-source systems running on bare VMs, than to try to keep up with Google’s deprecation treadmill “”” https://steve-yegge.medium.com/dear-google-cloud-your-deprecation-policy-is-killing-you-ee7525dc05dc https://steve-yegge.medium.com/dear-google-cloud-your-deprec... Mr Yegge makes my point much better ofc
- schrodinger 2y ago
  We went through a migration from Ruby on Rails (as a json api) to Go and it just really does make a difference when you need 1/50th the servers to run (don’t remember exact multiple, but it really was that dramatic—the concurrency model enabled so many parallel requests per instance compared to Ruby). It’s kinda like using a map reduce cluster when a beefy server with a few Linus commands piped could have handled the “big data” needs. Both have their time and place, but it’s amazing how far a simple setup can go, so don’t overengineer prematurely.
p_l 2y ago
Stack Overflow used to claim that the money they spent on licensing etc. to run their stack on .NET was well recouped in reduced hardware and energy spend, because .NET AOT compilation meant they got way more performance per buck.
CuriouslyC 2y ago
Pro tip, the strangler fig pattern is great for taming those oversized Django monoliths. FastAPI is a good replacement if you want to stay python, it's much faster.
- knowsuchagency 2y ago
  Django can be run on ASGI. It works especially well with Granian. Django-Ninja if you want REST API's. I like it better than FastAPI, personally, and I think it's more appropriate if you're already on Django
schrodinger 2y ago
Totally agreed. Such a fan of Go for this. It obviates just about any reason to use docker. You compile a single binary, for any platform from any platform, and just… copy it onto a server. Hell there’s even first class support to compile assets into it if you wanna throw your whole site into the binary, or many other use cases that would otherwise make docker nice. Damn I miss the startup that was a Go build and deploy to 3 static VMs over my current company’s dozens of microservices + ecs + kafka + aurora postgres. Startup A had the servers all sitting around at 2% cpu all day, trivially scalable by adding more vms. No downtime over 3 years. Startup B had 10x the users, but an elaborate docker + ecs + dozens of microservices (that could have just been libraries, avoiding all that complexity) + aurora postgres. Downtime all the time. Most recent was so hard to detect: a db server running fine at 40% load and our application around the same, yet db timeouts left and right so considered an official outage. Aws had throttled the network throughout between the DB and App with no warning! We even had an AWS support person on the call and took them 2 hours to figure it out.
- williamdclt 2y ago
  I don’t see how using a go binary vs dockerized microservices is related to database concerns? You need a database either way, and this problem you describe doesn’t seem related to the application level
redman25 2y ago
Web apps are generally IO bound by the database. Hardly any time gets spent in cpu even in python. There could be a number of things contributing to having to horizontally scale python more than compiled frameworks. Many apps are still stuck on non-async frameworks. Many devs are also bad at knowing how not to block server processes.
marcosdumay 2y ago
Python can waste 90% or 99% of your ops budget. But auto-scaling starts from 99%, and can go all the way to 4 or 5 9s of wastage. Of course, if you combine them, you add up the 9s. Anyway, it's patently obvious that increasing your unit costs by 7 orders of magnitude will constrain the kinds of business you can run. A few people will deny that, but the cloud is so impactful that those are very few nowadays. What a lot of people will deny is the relative importance of those two. If you take your C code from an auto-scaling environment and replace it with bare metal Python, you often get a few orders of magnitude gain.
nostrademons 2y ago
This is sort of like why plumbers hate chemical drain cleaners: you call the plumber only on the times that it doesn't work, and then pay them handsomely to clean up your mess. For all the times that Drano works, you never call the plumber in the first place, and so they never see it. Likewise, you only hire a SRE if a.) you have the budget to hire at all, which the vast majority of startups do not and b.) reliability is shitty enough that you need to hire someone to fix it. All the Django setups that hum along fine do not need SREs; the founders continue to rake in the money and don't need to hire anyone.

lifeisstillgood 2y ago

I’m not sure I understand - pythonnis “just” a process once you kick it off, want to run 8 processes on 8 CPUs, kick off 8 processes. Are you saying that to run this 8 processes you suddenly need 8 VMs running in the cloud - yeah. That does seem expensive. It’s very tempting to say one should not start in the cloud but host locally, but that seems an unpopular view. But maybe as we start to see more and more local workspaces catering to people working from home, we will see more and more “mini data centres” - it might even by Oxides sweet spot. Edit: “”” because it actually winds up being less DevOps work, on average, to support open-source systems running on bare VMs, than to try to keep up with Google’s deprecation treadmill “”” https://steve-yegge.medium.com/dear-google-cloud-your-deprecation-policy-is-killing-you-ee7525dc05dc https://steve-yegge.medium.com/dear-google-cloud-your-deprec... Mr Yegge makes my point much better ofc

schrodinger 2y ago
We went through a migration from Ruby on Rails (as a json api) to Go and it just really does make a difference when you need 1/50th the servers to run (don’t remember exact multiple, but it really was that dramatic—the concurrency model enabled so many parallel requests per instance compared to Ruby). It’s kinda like using a map reduce cluster when a beefy server with a few Linus commands piped could have handled the “big data” needs. Both have their time and place, but it’s amazing how far a simple setup can go, so don’t overengineer prematurely.

p_l 2y ago

Stack Overflow used to claim that the money they spent on licensing etc. to run their stack on .NET was well recouped in reduced hardware and energy spend, because .NET AOT compilation meant they got way more performance per buck.

CuriouslyC 2y ago

Pro tip, the strangler fig pattern is great for taming those oversized Django monoliths. FastAPI is a good replacement if you want to stay python, it's much faster.

knowsuchagency 2y ago
Django can be run on ASGI. It works especially well with Granian. Django-Ninja if you want REST API's. I like it better than FastAPI, personally, and I think it's more appropriate if you're already on Django

schrodinger 2y ago

Totally agreed. Such a fan of Go for this. It obviates just about any reason to use docker. You compile a single binary, for any platform from any platform, and just… copy it onto a server. Hell there’s even first class support to compile assets into it if you wanna throw your whole site into the binary, or many other use cases that would otherwise make docker nice. Damn I miss the startup that was a Go build and deploy to 3 static VMs over my current company’s dozens of microservices + ecs + kafka + aurora postgres. Startup A had the servers all sitting around at 2% cpu all day, trivially scalable by adding more vms. No downtime over 3 years. Startup B had 10x the users, but an elaborate docker + ecs + dozens of microservices (that could have just been libraries, avoiding all that complexity) + aurora postgres. Downtime all the time. Most recent was so hard to detect: a db server running fine at 40% load and our application around the same, yet db timeouts left and right so considered an official outage. Aws had throttled the network throughout between the DB and App with no warning! We even had an AWS support person on the call and took them 2 hours to figure it out.

williamdclt 2y ago
I don’t see how using a go binary vs dockerized microservices is related to database concerns? You need a database either way, and this problem you describe doesn’t seem related to the application level

redman25 2y ago

Web apps are generally IO bound by the database. Hardly any time gets spent in cpu even in python. There could be a number of things contributing to having to horizontally scale python more than compiled frameworks. Many apps are still stuck on non-async frameworks. Many devs are also bad at knowing how not to block server processes.

marcosdumay 2y ago

Python can waste 90% or 99% of your ops budget. But auto-scaling starts from 99%, and can go all the way to 4 or 5 9s of wastage. Of course, if you combine them, you add up the 9s. Anyway, it's patently obvious that increasing your unit costs by 7 orders of magnitude will constrain the kinds of business you can run. A few people will deny that, but the cloud is so impactful that those are very few nowadays. What a lot of people will deny is the relative importance of those two. If you take your C code from an auto-scaling environment and replace it with bare metal Python, you often get a few orders of magnitude gain.

nostrademons 2y ago

This is sort of like why plumbers hate chemical drain cleaners: you call the plumber only on the times that it doesn't work, and then pay them handsomely to clean up your mess. For all the times that Drano works, you never call the plumber in the first place, and so they never see it. Likewise, you only hire a SRE if a.) you have the budget to hire at all, which the vast majority of startups do not and b.) reliability is shitty enough that you need to hire someone to fix it. All the Django setups that hum along fine do not need SREs; the founders continue to rake in the money and don't need to hire anyone.