3 ms·
When I worked at Yelp circa 2013, we were an entirely Python shop (for backend). At that time, we had millions of LOC in Python. I love Python to death, but it
by SoylentOrange 6y ago
When I worked at Yelp circa 2013, we were an entirely Python shop (for backend). At that time, we had millions of LOC in Python. I love Python to death, but it became very clear that Python was simply a very bad fit for a project as large as Yelp.
I remember trying to write a program to simply understand our dependency graph so we could decouple some of our services (we were just starting our SOA efforts at that time). This is something other languages give you for free.
Our build times were exceptionally slow, as a direct result of Python's dynamic nature. For example, test discovery took over 2 minutes on each test run. This also made it painful to have distributed test runs, as splitting up tests across multiple processes would require the test discovery step to be run in each process, incurring a 2 minute penalty per process.
We also had deployment issues, because of Python's dynamic nature. Some code paths were only exercised during a production deployment, and were not sufficiently tested before deployment. This is despite the fact that we had tens of thousands of tests and sunk many hundreds of person-hours into testing as much as was possible. Despite a very professional engineering culture, and a constantly expanding test suite covering these corner cases, we had deployments fail about once a week for this reason.
I adore Python and still use it extensively most days (mostly its data science/ML libraries). But I now think that large projects would really benefit from static typing and strict compiler checks, as well as just being able to actually compile a binary (which would have solved many of our testing woes). In contrast to some other smart people on this thread, I don't think that even disciplined developer teams should use Python for large projects, at least not in the long-run.
- musingsole 6y agoYour description here is often the same argument for microservices over a justification for any specific language.
- SoylentOrange 6y agoDefinitely migrating to microservices helped alleviate the pressure from some of these problems: easier testability, smaller individual codebases with simpler dependency graphs, faster test and deploy times for each service, etc. But the language problem remains. I think there are a lot more good language options today for backend services than there were in 2010, and if I were writing new backend code for a big website/service today, I might pick one of those.