9 ms·

Whether intended or not, there's an undercurrent of "you're all so dumb for using Python" (or Ruby, or PHP, or other similarly performant language) here. I wan

by bjt 6y ago

Whether intended or not, there's an undercurrent of "you're all so dumb for using Python" (or Ruby, or PHP, or other similarly performant language) here. I want to surface that and question it a bit. It's totally reasonable for a company to choose the Python/Gunicorn option if they already have a bunch of people who know Python and they don't need to serve tons of requests per second. Even if they do need to serve tons of requests per second, it's totally reasonable for them to still choose Python/Gunicorn if the cost of the additional servers is less than the cost of having to support multiple languages. Or if they get a lot of value from libraries that are unique to the Python ecosystem. Or if they care more about quickly iterating on features than driving down server costs. I agree that there's a point where it stops making sense, and there are plenty of engineers who don't recognize when they're past that point because they keep doubling down on sunk costs and things they're familiar with. But let's not be too quick to assume people are in that camp when we don't know all the tradeoffs they're facing.

rumanator 6y ago
> Even if they do need to serve tons of requests per second, it's totally reasonable for them to still choose Python/Gunicorn if the cost of the additional servers is less than the cost of having to support multiple languages. How hard is it to get up to speed on any other tech stack? ASP.NET Core is extremely fast and the learning curve is close to none, for example. If someone was able to wrap his head around backend development with Python I'm pretty sure they have the mental fortitude to onboard a tech stack that doesn't suffer from major performance problems.
- stev_stev 6y ago
  This is where my head always goes as someone who does mostly Asp.Net Core. Why is it always between something like C++ or something like Python? Nowadays with middleware and endpoint routing, asp.net core can be almost as simple as flask (even if that’s not idiomatic or what the docs show you).
  - flyinglizard 6y ago
    Or Azure Functions if you want to go the serverless route. Coupled with Visual Studio, you can build high quality, large scale apps at about the same effort of blinking a LED on Arduino.
    - freeone3000 6y ago
      Are we using the same Azure Functions? I spend more time messing with functions.json and determining binding types and waiting five minutes for error messages to come in than I ever did when I simply provisioned an app service and deployed code.
      - deleted 6y ago
        [deleted]
      - xeroalt 6y ago
        Feel the same way with lambda.
  - xeroalt 6y ago
    I’m most comfortable in .net core but I recently learned Django because I kept hearing how productive it was from HN and all the startups around me use it, but Im starting to feel like I chose to move the wrong way. I will say I like how Postgres is the default for python and the library situation is much better over there as well.
    - thdrdt 6y ago
      I am a little confused by your post. Moving to Django is the wrong way?
      - xeroalt 6y ago
        Yeah Django feels terrible to me compared to .net core. I already wrote a similar answer to someone else so I will just paste it here "The developer experience in python was just way worse for me. Autogenerating swagger docs didn't seem possible without manually adding annotations to all my code (for flask at least), the battle-tested python libraries like django and sqlalchemy aren't built for async, mypy is obviously way worse than an actual type system and having static types really helps with understanding a large new code base, model validation (the thing forms/serializers handle) was more tedious, vscode was worse than visual studio (code navigation was very hard to do in python. I could jump 1 layer into library code, but when I tried to jump deeper vscode couldn't find anything.)" I've seen people say you should only choose rails or django for a startup, but .net core mvc provides the same batteries included approach and is built with the modern web ecosystem in mind. It also runs on linux and easily integrates with postgres so its just as cheap now as well. I think people that write off C# haven't really worked with it in its present form. I didn't find python significantly more terse than C# (other than having to define properties on types, which I already stated I prefer), just less feature rich.
- turtlebits 6y ago
  I disagree. I tried to learn F# a few months ago and the experience was horrible. I followed the tutorial using the aspnet CLI tool and got nothing but errors. I Google’d for a while until I gave up.
  - jdc 6y ago
    Can you link to the tutorial you followed?
  - tasogare 6y ago
    Of course it’s gonna be hard if you are trying to learn a language plus a framework at the same time, especially given that F# is treated as a second class citizen by Microsoft. An easier path is to break that learning into more manageable tasks: on one hand getting familiar with F# (which is already some work as it has both a functional side, and a CLR one; you need more the OOP part for doing ASP.net) and in the other ASP.net, for which the golden road is C#. Otherwise the doc is quite good for ASP.net Core, and it’s rare I get stuck on a problem for too long with it.
  - rumanator 6y ago
    > I tried to learn F# a few months ago and the experience was horrible. Switching to a programmig language based on an entirely different programming paradigm is not comparable to switching to a language based on the exact same programming paradigm to develop the exact same application using the exact same design patterns.
    - turtlebits 6y ago
      I’m completely comfortable with the language coming from Elixir. The problem was the tooling/docs didn’t match the experience. This was from the official site and many of the commands threw errors. I think it was a case of incomplete Mac support, which should have been called out. IIRC I posted on HN about it and got an apology for the state of tooling
- hedora 6y ago
  > How hard is it to get up to speed on any other tech stack? If I find myself debugging python tools, I usually just add debug statements to figure out WTF it is trying to do, and reimplement it in bash. It invariably is less than 10% as many lines of code, and also more debuggable / readable than the original. Granted, most of the python scripts I see these days are build processes or cluster coordinators.
  - EdwardDiego 6y ago
    > reimplement it in bash > more debuggable / readable More debuggable / readable for you, perhaps.
    - arpa 6y ago
      Hey, for me too. But then again I used to write bash frameworks for fun so there's that.
  - pjc50 6y ago
    Given how badly bash goes wrong if spaces in filenames sneak in, or if you want to do error handling, at several places I've had a policy of rewriting scripts from bash into python. It has the advantage that it's usually a better cross-platform solution than running bash on Windows.
- heelix 6y ago
  I'll find out myself. Just got handed a codebase that is a mix of Python/Gunicorn/node when this sprint started off. I've seen the word gunicorn in the repo... but still don't know what it does yet. First time for this old back end Java/C++/XQuery programmer, so how hard can it be? (On the plus side, seems my Jetbrain kit includes PyCharm, so I've even got an IDE!)
  - neurostimulant 6y ago
    Gunicorn is just a WSGI server, basically used to spawn a pool of webserver processes for your python backend. A python webserver is more optimal when used in multiprocess configuration (as opposed to multithreaded configuration, where python sorely suck at), and gunicorn will do that for you automatically, routing each http request to available worker process in the pool.
    - pdonis 6y ago
      > Gunicorn is just a WSGI server, basically used to spawn a pool of webserver processes for your python backend. Or a pool of asynchronous "green threads" using any of a number of libraries (gevent is the one I'm most familiar with). The thing to avoid is mixing the two (multiple processes and green threads). > A python webserver is more optimal when used in multiprocess configuration For CPU bound worker tasks, yes, this is true. For I/O bound applications, not so much; asynchronous I/O can handle the same I/O load with much fewer resources (particularly as forking Python processes uses a lot more memory because so much of the memory in the Python interpreter is dynamic, so you don't get a lot of benefit from what in a compiled language would be shared read-only code that doesn't need to be copied for every fork). > (as opposed to multithreaded configuration, where python sorely suck at) Yes, the limitation of the GIL is one of Python's worst warts. (It looks like there are finally efforts to remove it, but it's taken a long, long time.)
      - neurostimulant 6y ago
        Ah yes, I haven't used gevent for years and now associate gunicorn with wsgi. I have far easier experience with nodejs and golang when I need to do async parts on the backend (usually for websocket stuff), then use some message queue/passing system with celery or zeromq to communicate back and forth with the python backend.
- deleted 6y ago
  [deleted]
- neurostimulant 6y ago
  That's because I'm more productive with Python (been using it for ten years to implement many backend services), never hit any of its performance limitation yet (I mostly use it to develop b2b app with medium traffic at most, and always leverage distributed task queue for anything that might excessively blocks the webserver process), and also didn't want to maintain a fleet of windows servers (at least in the past when .net was windows only).
- PeterisP 6y ago
  It does not take that much time to get up to speed on another tech stack, but it does take some time, and servers are really cheap. Even if the learning curve is close to none, it's just cheaper to rent a dozen machines than have an engineering team spend a day or two looking into a new ecosystem.
- pmoleri 6y ago
  Learning curve close to none? I like that many times there's "the way" of doing things, but when I have to do something slightly outside the recommended way, I feel that the effort required outweights all the benefits.
  - rumanator 6y ago
    Isn't that true for all frameworks?
  - tarsinge 6y ago
    “The way” of doing things depends on the language, and for example for me Python is very hard. It’s not my main stack but I use it regularly and after years I still can’t find "the way" of doing things by myself, I feel like I end up on stackoverflow way too often because of anxiety of not being idiomatic. It’s like there is an enormous meta knowledge very specific to the language. That’s not something I experience with other stacks.
- 29athrowaway 6y ago
  1) .NET APIs change very frequently. 2) C# is a very verbose language, that requires a lot of typing. 3) F#, the best language in .NET, is largely ignored by the .NET community.
  - EdwardDiego 6y ago
    I can't speak from recent experience, but on point 1) when I was teaching myself to code, I focused on .NET due to its prevalence in my local market. I took part in the .NET user group (called DUG, natch), attended the meetups etc. And this was at the time when MS would announce a new blessed way to do things on a reasonably frequent basis. When I started, the blessed way to access data was DAOs, then it was ADO.NET (note, those could be around the wrong way, I have trouble figuring it out now in hindsight), then it was Linq2Sql, then that was deprecated for Entity Framework (which, I'll give credit, they seem to have stuck with, even if it does feel like a half-cribbed NHibernate). I was frantically trying to learn the blessed thing, because the MS shops I was familiar with only used the blessed thing - the server was IIS, the database was SQL Server, the language was C# (I was the only member of the DUG who coded in F#, and few others knew of it), and you used the blessed patterns and the blessed frameworks. Incidentally, this is why FOSS has had such a hard time in .NET, as soon as MS releases something that is reasonably feature complete, a lot of single-vendor minded companies switch to it. And I met a lot of developers in their early 40s who were quietly terrified of getting left behind on the MS technology treadmill, and trying just as frantically to learn the new blessed thing as I was. And then I got hired by a Java shop, and faced a paradigm where shit code cough java.util.Calendar, java.util.Date cough stuck around for yonks because it was good enough and replacing it had to be done very thoughtfully and gently. Point #2 isn't super relevant in the age of IDEs, but I agree wholeheartedly that F# deserves a lot more love than it gets.
  - rumanator 6y ago
    > 1) .NET APIs change very frequently. ASP.NET Core 2.1 was released in 2018 and will be supported until late 2021. ASP.NET Core 3.1 was released a few months ago and there is no end of support in sight. Moreover, the changes between 2.1 and 3.1 were not that many. I've migrated a whole ASP.NET Core 2.1 web service to 3.1 in less than 1 hour. > 2) C# is a very verbose language, that requires a lot of typing. Nonsense. The only added verbosity to C# when compared with Python are the type declarations, which arguably are a problem plaguing Python. The first class support for events and async programming and properties in C# more than make up for it. > 3) F#, the best language in .NET, is largely ignored by the .NET community. I fail to see what point you were trying to make.
- vidarh 6y ago
  A lot of developers never get proficient on one stack. Much less more than one.
  - rumanator 6y ago
    You don't need to master one stack to get up to speed. For that investment to pay off you only need to be almost as good as you were with your old stack, which isn't hard if you already are not proficient.
    - vidarh 6y ago
      I'd argue that time would be better spent actually learning any single stack properly.
- tetha 6y ago
  Writing code is one thing though. Now you have to figure out how to run a CI on a build server. Deployment on your production systems, be it containers or even not. Monitoring, alerting, profiling, tuning under load. You have to support a new database connection library with new shenanigans. In general, you need to integrate the new language into the existing ecosystem. The latter may even be impossible depending on the stack and solution chosen. Just look up those weird Java only caching servers. All of that is possible, sure. Due to company acquisitions and specialized teams in some areas, we're kinda running the full bingo card of languages. But there's little denying: We overall spent less time handling language runtimes and language-specific monitoring back when we were java+mysql and that's it.
  - rumanator 6y ago
    > Now you have to figure out how to run a CI on a build server. Deployment on your production systems, be it containers or even not. Monitoring, alerting, profiling, tuning under load. You have to support a new database connection library with new shenanigans. Most if not all of those items are either trivial or non-issues. In fact, I would argue that deploying a Python app is a far more convoluted process than getting an ASP.NET Core app up and running. With Docker, the problem simply disappears. Getting it to build and test on a CICD pipeline is as hard as typing $ dotnet build, or $ dotnet test. > All of that is possible, sure. Not only it is possible, it's laughably easy. We are supposed to avoid our problems, not perpetuate and aggravate them as eternum because we are too lazy to look for ways to make our life easier.
anonymoushn 6y ago
It seems like doubling down is the standard thing to do. Here's a video about how Instagram bugs engineers to do fewer string manipulations in Python instead of using a faster language https://youtu.be/hnpzNAPiC0E https://youtu.be/hnpzNAPiC0E
- bsder 6y ago
  You ... are not ... Instagram. Okay. Get over it. Python is fine. Threads are fine. Go is fine. Clojure is fine. Java is fine. PHP is ... okay I won't go that far. :) You fix the problem when the cost required to fix the problem is finally less than the opportunity cost of fixing something else.
  - anonymoushn 6y ago
    I would prefer not to deal with "scaling up" for workloads that could run in a single process on commodity hardware. I've mainly worked at companies that would be bankrupt if they used Python the way Instagram does.
    - bsder 6y ago
      Django and Python somehow handled websites for entire newspapers on far wimpier hardware just fine--which makes me wonder if "cloud" (ie. non-deterministic memory and I/O accesses due to sharing with other tenants) isn't the problem rather than Python. You need a market. You need paying customers. You need cashflow. You need features. You need a business plan. You don't need scaling. Ever. To first, second and third order approximations. Your company has a higher probability of bankruptcy than needing scaling.
      - earthboundkid 6y ago
        Newspapers run everything behind a CDN, so Django isn’t a bottleneck. The things they do that can’t just be CDN buffered (like ad targeting and paywalls and comments) end up being supplied by outside vendors. I think Django fits well for a news org, but it’s good to understand why it fits well.
mdszy 6y ago
Twitter ran on Rails until 10 million users.
- samatman 6y ago
  Ah yes, the Fail Whale era.
  - fnord123 6y ago
    They should have kept the fail whale instead of offering "Sorry the tweet couldn't be loaded. Please retry" which I get 3/4 of the time on my phone.
- randomdata 6y ago
  Missing an order of magnitude? Twitter was claiming to be still running Rails in the mid-2010s, when they apparently had 200+ million active users. The service was finally sunsetted during the big Scala rewrite. They abandoned their message queue (Starling) written in Ruby early on. That abandonment was often misattributed to them abandoning Rails, I guess because of the shared usage of Ruby. The confusion was compounded to back when someone at Twitter (may have been Odeo at time?) posted a message board rant about how Rails does not scale because ActiveRecord did not allow connecting to multiple databases out of the box. That event seems to be the origins of the "Rails does not scale" mantra that swept the internet for a time. But, humorously, someone replied with a solution a few minutes later.
- kelnos 6y ago
  But how many instances did they have running that app, and at what cost? Did they have to build a ridiculous amount of caching in? And wasn't that the time period where they were incredibly unreliable to the point that their "fail whale" server error page became a running gag?
  - vidarh 6y ago
    Twitters architecture was a bad joke. Many to many communication in a scalable manner has been a solved problem from decades: you federate. You assign users to buckets, you assign buckets to servers. You route messages like you'd route e-mail. Been there, done that. The federation does not need to be outwardly visible. Twitters problem was a too centralized architecture not Rails. And I say that as someone who at the time hated Rails and who still hates Rails. I find it bloated and over-complicated. It may even have led them to make bad architectural choices because of how it was structured. But they still did make bad architectural choices, and they fixed those choices at the same time they moved off Rails.
theelous3 6y ago
I don't really get what the point of this post is. Is it really a dig at python? Python can handle thousands of connections in a single thread no problem with basic enough stuff. Is what the author did supposed to be impressive? Is it supposed to make python look bad? I don't get it. Seems like run of the mill stuff. Python might struggle at the same level of concurrency (was it like 15k?) but you can still do 10k connections easy enough iirc.
- rejschaap 6y ago
  I read it more like: "what if we just use threads instead of avoiding them at all cost and complicate things to get concurrency"
  - theelous3 6y ago
    Which I read as "I have no experience whatsoever with modern python, and async is something that catches water". Async is typically less prone to error and complexity than thread code, and also typically faster/lighter for io (in python). I can't see a reason for this not to be the case in other languages too.
    - dnautics 6y ago
      Note: these opinions are mine In terms of error-proneness, I would say the hierarchy is this: Actors < CSP ~ Async << Threads In terms of "getting started" difficulty, I would put the order like this: Async < Actors ~ CSP < Threads In terms of first-order maintainability in large projects I would put the order like this: Actors >> CSP ~ Threads > Async Async is on its face easy to grok, and saves you a ton of problems with locking and the like, and you can run it on a single threaded system with the right type of dispatcher. However, small amounts of complexity rapidly devolve into towers of async calls that cannot be untangled from a spaghetti pile, and there's not really satisfying ways of figuring out how to unwind error-handling in async, except for the basic case of "I truly don't care if this async fails". I will say though, it's spectacularly easy to write messy and garbage code in all four of these concurrency models (I certainly have). My preference for actors comes from four opinions: - some systems (if you want to be flippant, microservices architecture) use actors to encapsulate failure domains, which is really fantasic, and truly the #1 reason to use actors - 99.8% of the time no need to write mutexes, and good actor systems basically won't deadlock unless you really try hard. (also true for async system btw) - gives you an organizational framework to write well-designed and well-engineered systems. - with a small amount of discipline "not having a spaghetti ball" scales with complexity (I find it takes a lot of discipline to not have a spaghetti ball with async in the more complex cases)
flyinglizard 6y ago
The only reason to use Python for anything more than few hundreds lines worth of utility is if you're working on a codebase that's already in Python, and even then it's debatable. There simply isn't an excuse for using Python for any infrastructure. It does nothing particularly well - or even right - other than very purpose-specific scripting. It can tie things together well enough. And your codebase becomes a liability rather than an asset. I always say this opinion when it comes to Python discussion in HN and I always get downvoted but hey, "all it takes for evil to triumph...".
- bsder 6y ago
  > The only reason to use Python for anything more than few hundreds lines worth of utility is if you're working on a codebase that's already in Python, and even then it's debatable. One important reason for using Python is that it almost always forces people and companies to release the source code. I will take a shitty Python script over shitty C/Rust/Go/Java code any day for that reason ALONE. With source code, I can fix your shitty program (and all programs are shitty--even mine). If it's compiled, that path is blocked.
- itwy 6y ago
  Any which language do you use for web apps if I may ask? Let me guess. Rust.
  - flyinglizard 6y ago
    I primarily do all sorts of systems programming, but the few web backends I did were C# on Azure Functions and it was spectacular. I’m a very big proponent of Microsoft’s tooling and language development. The difference in productivity between C# and Python is such that they might as well have been developed by different civilizations. (I have occasionally used Python since about 2003 and C# since 2018)
    - jefurii 6y ago
      > The difference in productivity between C# and Python is such that they might as well have been developed by different civilizations. I think they were. Is that a bad thing?
Doxin 6y ago
Not to mention that python is plenty fast compared to the time it takes to write stuff to the network. Of course heavyweight frameworks like django don't help the equation, but writing fast network code in python isn't exactly hard either.
- kccqzy 6y ago
  > python is plenty fast compared to the time it takes to write stuff to the network With the proliferation of microservices, I find this increasingly not true. Sure, python definitely is plenty fast when you need to send something to a user many miles away. But with microservices, writing to the network might mean writing to a machine in the same data center, or even the same host in a different container. That's as fast as a few memory copies and a few context switches.
- vidarh 6y ago
  This is something a lot of people don't get with most higher level languages. My first commercial use of Ruby was in 2005. Not web facing, but messaging middleware. As in a pub-sub type passing of messages between various endpoints. We had a C version. It was about 7k lines to support the bare minimum we needed. As an experiment to teach myself Ruby I wrote a Ruby implementation. With the usual caveats (it's often easy to make a rewrite better in all kinds of ways, including size), it was ~700 lines, far easier to read, and supported far more functionality, so I put it in production. Was it slower? As usual that depends what you mean by "slower". It consumed 10x more CPU, but it also did much more work (e.g. supporting more flexible routing of messages etc.). The throughput, however was the same, and 10x more CPU means that maxing out the network connection took 10% of a single core instead of 1% of a single core. For some types of tasks CPU is the most important thing, but for a lot of tasks you'll be IO limited. And for a lot of tasks that people think are CPU limited are really down to poor IO handling (causing excessive context switches e.g. through lots of small reads is a common one)
  - pjmlp 6y ago
    And other people don't get it is possible to have high level languages and almost C like performance. You don't need to give up on JIT and AOT compilation to use high level languages, and this is where current tooling for Ruby and Python ends up losing.
pdonis 6y ago
> It's totally reasonable for a company to choose the Python/Gunicorn option Yes, if done properly. The issues with the particular Python/Gunicorn setup the author described in an earlier article (linked to in this one) were not so much Python/Gunicorn issues as "not understanding how to properly use Python/Gunicorn" issues, or more generally "not understanding how the tool you are trying to use actually works" issues. (I actually shudder to think what such a group would have done trying to program the same application in C.)

by bjt 6y ago

rumanator 6y ago

> Even if they do need to serve tons of requests per second, it's totally reasonable for them to still choose Python/Gunicorn if the cost of the additional servers is less than the cost of having to support multiple languages. How hard is it to get up to speed on any other tech stack? ASP.NET Core is extremely fast and the learning curve is close to none, for example. If someone was able to wrap his head around backend development with Python I'm pretty sure they have the mental fortitude to onboard a tech stack that doesn't suffer from major performance problems.

stev_stev 6y ago
This is where my head always goes as someone who does mostly Asp.Net Core. Why is it always between something like C++ or something like Python? Nowadays with middleware and endpoint routing, asp.net core can be almost as simple as flask (even if that’s not idiomatic or what the docs show you).
- flyinglizard 6y ago
  Or Azure Functions if you want to go the serverless route. Coupled with Visual Studio, you can build high quality, large scale apps at about the same effort of blinking a LED on Arduino.
  - freeone3000 6y ago
    Are we using the same Azure Functions? I spend more time messing with functions.json and determining binding types and waiting five minutes for error messages to come in than I ever did when I simply provisioned an app service and deployed code.
    - deleted 6y ago
      [deleted]
    - xeroalt 6y ago
      Feel the same way with lambda.
- xeroalt 6y ago
  I’m most comfortable in .net core but I recently learned Django because I kept hearing how productive it was from HN and all the startups around me use it, but Im starting to feel like I chose to move the wrong way. I will say I like how Postgres is the default for python and the library situation is much better over there as well.
  - thdrdt 6y ago
    I am a little confused by your post. Moving to Django is the wrong way?
    - xeroalt 6y ago
      Yeah Django feels terrible to me compared to .net core. I already wrote a similar answer to someone else so I will just paste it here "The developer experience in python was just way worse for me. Autogenerating swagger docs didn't seem possible without manually adding annotations to all my code (for flask at least), the battle-tested python libraries like django and sqlalchemy aren't built for async, mypy is obviously way worse than an actual type system and having static types really helps with understanding a large new code base, model validation (the thing forms/serializers handle) was more tedious, vscode was worse than visual studio (code navigation was very hard to do in python. I could jump 1 layer into library code, but when I tried to jump deeper vscode couldn't find anything.)" I've seen people say you should only choose rails or django for a startup, but .net core mvc provides the same batteries included approach and is built with the modern web ecosystem in mind. It also runs on linux and easily integrates with postgres so its just as cheap now as well. I think people that write off C# haven't really worked with it in its present form. I didn't find python significantly more terse than C# (other than having to define properties on types, which I already stated I prefer), just less feature rich.
turtlebits 6y ago
I disagree. I tried to learn F# a few months ago and the experience was horrible. I followed the tutorial using the aspnet CLI tool and got nothing but errors. I Google’d for a while until I gave up.
- jdc 6y ago
  Can you link to the tutorial you followed?
- tasogare 6y ago
  Of course it’s gonna be hard if you are trying to learn a language plus a framework at the same time, especially given that F# is treated as a second class citizen by Microsoft. An easier path is to break that learning into more manageable tasks: on one hand getting familiar with F# (which is already some work as it has both a functional side, and a CLR one; you need more the OOP part for doing ASP.net) and in the other ASP.net, for which the golden road is C#. Otherwise the doc is quite good for ASP.net Core, and it’s rare I get stuck on a problem for too long with it.
- rumanator 6y ago
  > I tried to learn F# a few months ago and the experience was horrible. Switching to a programmig language based on an entirely different programming paradigm is not comparable to switching to a language based on the exact same programming paradigm to develop the exact same application using the exact same design patterns.
  - turtlebits 6y ago
    I’m completely comfortable with the language coming from Elixir. The problem was the tooling/docs didn’t match the experience. This was from the official site and many of the commands threw errors. I think it was a case of incomplete Mac support, which should have been called out. IIRC I posted on HN about it and got an apology for the state of tooling
hedora 6y ago
> How hard is it to get up to speed on any other tech stack? If I find myself debugging python tools, I usually just add debug statements to figure out WTF it is trying to do, and reimplement it in bash. It invariably is less than 10% as many lines of code, and also more debuggable / readable than the original. Granted, most of the python scripts I see these days are build processes or cluster coordinators.
- EdwardDiego 6y ago
  > reimplement it in bash > more debuggable / readable More debuggable / readable for you, perhaps.
  - arpa 6y ago
    Hey, for me too. But then again I used to write bash frameworks for fun so there's that.
- pjc50 6y ago
  Given how badly bash goes wrong if spaces in filenames sneak in, or if you want to do error handling, at several places I've had a policy of rewriting scripts from bash into python. It has the advantage that it's usually a better cross-platform solution than running bash on Windows.
heelix 6y ago
I'll find out myself. Just got handed a codebase that is a mix of Python/Gunicorn/node when this sprint started off. I've seen the word gunicorn in the repo... but still don't know what it does yet. First time for this old back end Java/C++/XQuery programmer, so how hard can it be? (On the plus side, seems my Jetbrain kit includes PyCharm, so I've even got an IDE!)
- neurostimulant 6y ago
  Gunicorn is just a WSGI server, basically used to spawn a pool of webserver processes for your python backend. A python webserver is more optimal when used in multiprocess configuration (as opposed to multithreaded configuration, where python sorely suck at), and gunicorn will do that for you automatically, routing each http request to available worker process in the pool.
  - pdonis 6y ago
    > Gunicorn is just a WSGI server, basically used to spawn a pool of webserver processes for your python backend. Or a pool of asynchronous "green threads" using any of a number of libraries (gevent is the one I'm most familiar with). The thing to avoid is mixing the two (multiple processes and green threads). > A python webserver is more optimal when used in multiprocess configuration For CPU bound worker tasks, yes, this is true. For I/O bound applications, not so much; asynchronous I/O can handle the same I/O load with much fewer resources (particularly as forking Python processes uses a lot more memory because so much of the memory in the Python interpreter is dynamic, so you don't get a lot of benefit from what in a compiled language would be shared read-only code that doesn't need to be copied for every fork). > (as opposed to multithreaded configuration, where python sorely suck at) Yes, the limitation of the GIL is one of Python's worst warts. (It looks like there are finally efforts to remove it, but it's taken a long, long time.)
    - neurostimulant 6y ago
      Ah yes, I haven't used gevent for years and now associate gunicorn with wsgi. I have far easier experience with nodejs and golang when I need to do async parts on the backend (usually for websocket stuff), then use some message queue/passing system with celery or zeromq to communicate back and forth with the python backend.
deleted 6y ago
[deleted]
neurostimulant 6y ago
That's because I'm more productive with Python (been using it for ten years to implement many backend services), never hit any of its performance limitation yet (I mostly use it to develop b2b app with medium traffic at most, and always leverage distributed task queue for anything that might excessively blocks the webserver process), and also didn't want to maintain a fleet of windows servers (at least in the past when .net was windows only).
PeterisP 6y ago
It does not take that much time to get up to speed on another tech stack, but it does take some time, and servers are really cheap. Even if the learning curve is close to none, it's just cheaper to rent a dozen machines than have an engineering team spend a day or two looking into a new ecosystem.
pmoleri 6y ago
Learning curve close to none? I like that many times there's "the way" of doing things, but when I have to do something slightly outside the recommended way, I feel that the effort required outweights all the benefits.
- rumanator 6y ago
  Isn't that true for all frameworks?
- tarsinge 6y ago
  “The way” of doing things depends on the language, and for example for me Python is very hard. It’s not my main stack but I use it regularly and after years I still can’t find "the way" of doing things by myself, I feel like I end up on stackoverflow way too often because of anxiety of not being idiomatic. It’s like there is an enormous meta knowledge very specific to the language. That’s not something I experience with other stacks.
29athrowaway 6y ago
1) .NET APIs change very frequently. 2) C# is a very verbose language, that requires a lot of typing. 3) F#, the best language in .NET, is largely ignored by the .NET community.
- EdwardDiego 6y ago
  I can't speak from recent experience, but on point 1) when I was teaching myself to code, I focused on .NET due to its prevalence in my local market. I took part in the .NET user group (called DUG, natch), attended the meetups etc. And this was at the time when MS would announce a new blessed way to do things on a reasonably frequent basis. When I started, the blessed way to access data was DAOs, then it was ADO.NET (note, those could be around the wrong way, I have trouble figuring it out now in hindsight), then it was Linq2Sql, then that was deprecated for Entity Framework (which, I'll give credit, they seem to have stuck with, even if it does feel like a half-cribbed NHibernate). I was frantically trying to learn the blessed thing, because the MS shops I was familiar with only used the blessed thing - the server was IIS, the database was SQL Server, the language was C# (I was the only member of the DUG who coded in F#, and few others knew of it), and you used the blessed patterns and the blessed frameworks. Incidentally, this is why FOSS has had such a hard time in .NET, as soon as MS releases something that is reasonably feature complete, a lot of single-vendor minded companies switch to it. And I met a lot of developers in their early 40s who were quietly terrified of getting left behind on the MS technology treadmill, and trying just as frantically to learn the new blessed thing as I was. And then I got hired by a Java shop, and faced a paradigm where shit code cough java.util.Calendar, java.util.Date cough stuck around for yonks because it was good enough and replacing it had to be done very thoughtfully and gently. Point #2 isn't super relevant in the age of IDEs, but I agree wholeheartedly that F# deserves a lot more love than it gets.
- rumanator 6y ago
  > 1) .NET APIs change very frequently. ASP.NET Core 2.1 was released in 2018 and will be supported until late 2021. ASP.NET Core 3.1 was released a few months ago and there is no end of support in sight. Moreover, the changes between 2.1 and 3.1 were not that many. I've migrated a whole ASP.NET Core 2.1 web service to 3.1 in less than 1 hour. > 2) C# is a very verbose language, that requires a lot of typing. Nonsense. The only added verbosity to C# when compared with Python are the type declarations, which arguably are a problem plaguing Python. The first class support for events and async programming and properties in C# more than make up for it. > 3) F#, the best language in .NET, is largely ignored by the .NET community. I fail to see what point you were trying to make.
vidarh 6y ago
A lot of developers never get proficient on one stack. Much less more than one.
- rumanator 6y ago
  You don't need to master one stack to get up to speed. For that investment to pay off you only need to be almost as good as you were with your old stack, which isn't hard if you already are not proficient.
  - vidarh 6y ago
    I'd argue that time would be better spent actually learning any single stack properly.
tetha 6y ago
Writing code is one thing though. Now you have to figure out how to run a CI on a build server. Deployment on your production systems, be it containers or even not. Monitoring, alerting, profiling, tuning under load. You have to support a new database connection library with new shenanigans. In general, you need to integrate the new language into the existing ecosystem. The latter may even be impossible depending on the stack and solution chosen. Just look up those weird Java only caching servers. All of that is possible, sure. Due to company acquisitions and specialized teams in some areas, we're kinda running the full bingo card of languages. But there's little denying: We overall spent less time handling language runtimes and language-specific monitoring back when we were java+mysql and that's it.
- rumanator 6y ago
  > Now you have to figure out how to run a CI on a build server. Deployment on your production systems, be it containers or even not. Monitoring, alerting, profiling, tuning under load. You have to support a new database connection library with new shenanigans. Most if not all of those items are either trivial or non-issues. In fact, I would argue that deploying a Python app is a far more convoluted process than getting an ASP.NET Core app up and running. With Docker, the problem simply disappears. Getting it to build and test on a CICD pipeline is as hard as typing $ dotnet build, or $ dotnet test. > All of that is possible, sure. Not only it is possible, it's laughably easy. We are supposed to avoid our problems, not perpetuate and aggravate them as eternum because we are too lazy to look for ways to make our life easier.

anonymoushn 6y ago

It seems like doubling down is the standard thing to do. Here's a video about how Instagram bugs engineers to do fewer string manipulations in Python instead of using a faster language https://youtu.be/hnpzNAPiC0E https://youtu.be/hnpzNAPiC0E

bsder 6y ago
You ... are not ... Instagram. Okay. Get over it. Python is fine. Threads are fine. Go is fine. Clojure is fine. Java is fine. PHP is ... okay I won't go that far. :) You fix the problem when the cost required to fix the problem is finally less than the opportunity cost of fixing something else.
- anonymoushn 6y ago
  I would prefer not to deal with "scaling up" for workloads that could run in a single process on commodity hardware. I've mainly worked at companies that would be bankrupt if they used Python the way Instagram does.
  - bsder 6y ago
    Django and Python somehow handled websites for entire newspapers on far wimpier hardware just fine--which makes me wonder if "cloud" (ie. non-deterministic memory and I/O accesses due to sharing with other tenants) isn't the problem rather than Python. You need a market. You need paying customers. You need cashflow. You need features. You need a business plan. You don't need scaling. Ever. To first, second and third order approximations. Your company has a higher probability of bankruptcy than needing scaling.
    - earthboundkid 6y ago
      Newspapers run everything behind a CDN, so Django isn’t a bottleneck. The things they do that can’t just be CDN buffered (like ad targeting and paywalls and comments) end up being supplied by outside vendors. I think Django fits well for a news org, but it’s good to understand why it fits well.

mdszy 6y ago

Twitter ran on Rails until 10 million users.

samatman 6y ago
Ah yes, the Fail Whale era.
- fnord123 6y ago
  They should have kept the fail whale instead of offering "Sorry the tweet couldn't be loaded. Please retry" which I get 3/4 of the time on my phone.
randomdata 6y ago
Missing an order of magnitude? Twitter was claiming to be still running Rails in the mid-2010s, when they apparently had 200+ million active users. The service was finally sunsetted during the big Scala rewrite. They abandoned their message queue (Starling) written in Ruby early on. That abandonment was often misattributed to them abandoning Rails, I guess because of the shared usage of Ruby. The confusion was compounded to back when someone at Twitter (may have been Odeo at time?) posted a message board rant about how Rails does not scale because ActiveRecord did not allow connecting to multiple databases out of the box. That event seems to be the origins of the "Rails does not scale" mantra that swept the internet for a time. But, humorously, someone replied with a solution a few minutes later.
kelnos 6y ago
But how many instances did they have running that app, and at what cost? Did they have to build a ridiculous amount of caching in? And wasn't that the time period where they were incredibly unreliable to the point that their "fail whale" server error page became a running gag?
- vidarh 6y ago
  Twitters architecture was a bad joke. Many to many communication in a scalable manner has been a solved problem from decades: you federate. You assign users to buckets, you assign buckets to servers. You route messages like you'd route e-mail. Been there, done that. The federation does not need to be outwardly visible. Twitters problem was a too centralized architecture not Rails. And I say that as someone who at the time hated Rails and who still hates Rails. I find it bloated and over-complicated. It may even have led them to make bad architectural choices because of how it was structured. But they still did make bad architectural choices, and they fixed those choices at the same time they moved off Rails.

theelous3 6y ago

I don't really get what the point of this post is. Is it really a dig at python? Python can handle thousands of connections in a single thread no problem with basic enough stuff. Is what the author did supposed to be impressive? Is it supposed to make python look bad? I don't get it. Seems like run of the mill stuff. Python might struggle at the same level of concurrency (was it like 15k?) but you can still do 10k connections easy enough iirc.

rejschaap 6y ago
I read it more like: "what if we just use threads instead of avoiding them at all cost and complicate things to get concurrency"
- theelous3 6y ago
  Which I read as "I have no experience whatsoever with modern python, and async is something that catches water". Async is typically less prone to error and complexity than thread code, and also typically faster/lighter for io (in python). I can't see a reason for this not to be the case in other languages too.
  - dnautics 6y ago
    Note: these opinions are mine In terms of error-proneness, I would say the hierarchy is this: Actors < CSP ~ Async << Threads In terms of "getting started" difficulty, I would put the order like this: Async < Actors ~ CSP < Threads In terms of first-order maintainability in large projects I would put the order like this: Actors >> CSP ~ Threads > Async Async is on its face easy to grok, and saves you a ton of problems with locking and the like, and you can run it on a single threaded system with the right type of dispatcher. However, small amounts of complexity rapidly devolve into towers of async calls that cannot be untangled from a spaghetti pile, and there's not really satisfying ways of figuring out how to unwind error-handling in async, except for the basic case of "I truly don't care if this async fails". I will say though, it's spectacularly easy to write messy and garbage code in all four of these concurrency models (I certainly have). My preference for actors comes from four opinions: - some systems (if you want to be flippant, microservices architecture) use actors to encapsulate failure domains, which is really fantasic, and truly the #1 reason to use actors - 99.8% of the time no need to write mutexes, and good actor systems basically won't deadlock unless you really try hard. (also true for async system btw) - gives you an organizational framework to write well-designed and well-engineered systems. - with a small amount of discipline "not having a spaghetti ball" scales with complexity (I find it takes a lot of discipline to not have a spaghetti ball with async in the more complex cases)

flyinglizard 6y ago

The only reason to use Python for anything more than few hundreds lines worth of utility is if you're working on a codebase that's already in Python, and even then it's debatable. There simply isn't an excuse for using Python for any infrastructure. It does nothing particularly well - or even right - other than very purpose-specific scripting. It can tie things together well enough. And your codebase becomes a liability rather than an asset. I always say this opinion when it comes to Python discussion in HN and I always get downvoted but hey, "all it takes for evil to triumph...".

bsder 6y ago
> The only reason to use Python for anything more than few hundreds lines worth of utility is if you're working on a codebase that's already in Python, and even then it's debatable. One important reason for using Python is that it almost always forces people and companies to release the source code. I will take a shitty Python script over shitty C/Rust/Go/Java code any day for that reason ALONE. With source code, I can fix your shitty program (and all programs are shitty--even mine). If it's compiled, that path is blocked.
itwy 6y ago
Any which language do you use for web apps if I may ask? Let me guess. Rust.
- flyinglizard 6y ago
  I primarily do all sorts of systems programming, but the few web backends I did were C# on Azure Functions and it was spectacular. I’m a very big proponent of Microsoft’s tooling and language development. The difference in productivity between C# and Python is such that they might as well have been developed by different civilizations. (I have occasionally used Python since about 2003 and C# since 2018)
  - jefurii 6y ago
    > The difference in productivity between C# and Python is such that they might as well have been developed by different civilizations. I think they were. Is that a bad thing?

Doxin 6y ago

Not to mention that python is plenty fast compared to the time it takes to write stuff to the network. Of course heavyweight frameworks like django don't help the equation, but writing fast network code in python isn't exactly hard either.

kccqzy 6y ago
> python is plenty fast compared to the time it takes to write stuff to the network With the proliferation of microservices, I find this increasingly not true. Sure, python definitely is plenty fast when you need to send something to a user many miles away. But with microservices, writing to the network might mean writing to a machine in the same data center, or even the same host in a different container. That's as fast as a few memory copies and a few context switches.
vidarh 6y ago
This is something a lot of people don't get with most higher level languages. My first commercial use of Ruby was in 2005. Not web facing, but messaging middleware. As in a pub-sub type passing of messages between various endpoints. We had a C version. It was about 7k lines to support the bare minimum we needed. As an experiment to teach myself Ruby I wrote a Ruby implementation. With the usual caveats (it's often easy to make a rewrite better in all kinds of ways, including size), it was ~700 lines, far easier to read, and supported far more functionality, so I put it in production. Was it slower? As usual that depends what you mean by "slower". It consumed 10x more CPU, but it also did much more work (e.g. supporting more flexible routing of messages etc.). The throughput, however was the same, and 10x more CPU means that maxing out the network connection took 10% of a single core instead of 1% of a single core. For some types of tasks CPU is the most important thing, but for a lot of tasks you'll be IO limited. And for a lot of tasks that people think are CPU limited are really down to poor IO handling (causing excessive context switches e.g. through lots of small reads is a common one)
- pjmlp 6y ago
  And other people don't get it is possible to have high level languages and almost C like performance. You don't need to give up on JIT and AOT compilation to use high level languages, and this is where current tooling for Ruby and Python ends up losing.

pdonis 6y ago

> It's totally reasonable for a company to choose the Python/Gunicorn option Yes, if done properly. The issues with the particular Python/Gunicorn setup the author described in an earlier article (linked to in this one) were not so much Python/Gunicorn issues as "not understanding how to properly use Python/Gunicorn" issues, or more generally "not understanding how the tool you are trying to use actually works" issues. (I actually shudder to think what such a group would have done trying to program the same application in C.)