4 ms·

Always the same people that get their names in the news. I would say that the amount of tweaks that prop up Python is no cause for celebration.

by mlrpfx 3y ago

blitzar 3y ago
Yet here it is ... powering a global scale app. Too weak for someones hobby project, strong enough for global scale.
- mlrtph100 3y ago
  So Google and Dropbox, who switched to Golang, are hobby projects.
  - cmrdporcupine 3y ago
    Choice of language is not what makes things succeed or fail in production. Team best practices, enforcement of coding style and technique, good project management, infrastructure, and team cohesion etc. And weeding out bad engineers who think that "switching to a new [language|framework|religion]" will solve all problems. 10+ years ago Facebook itself did a remarkable job of scaling up, with hardly any outages, a PHP LAMP stack thingy to one of the hugest traffic websites in the world. Meanwhile back then Twitter mucked around with every novel technology they could find or invent, and had constant outages. Also Google didn't "switch to Golang." There's plenty of C++, Java, Python there. I worked there for 10 years and encountered only a handful of Go projects. Lots of cloud services in C++. Would I personally choose Python for a project? No, I don't like it, and I work in Rust full time because I prefer it. But if I worked at Meta / Instagram and had a team of Python engineers and existing libraries & infrastructure, this would be the right approach.
    - aeyes 3y ago
      Also, what's really different from Instagram? You change the picture for a short text and many other features already existed - comments, likes, subscriptions, recommendations, personal messages... I think this is just an Instagram skin with some feature flags.
- Yoric 3y ago
  My few years of personal experience of using Python with even small-ish scale professional projects have been... underwhelming so far. I wonder how Meta (and Google, and all the others who write serious Python code) manage to keep things running – and I'm not talking (only) of performance, but largely of bugs. My personal suspicion is that the secret sauce is simply to have a sufficient budget to hire an army of code quality engineers. I could be wrong, though.
  - ambivalence 3y ago
    In short: static analysis, unit tests, staging environments, effective log aggregation. The first one is a big productivity booster as it shows you bugs before you commit them. Here's a post about this, not exactly new but still describing the general principles very well: https://instagram-engineering.com/static-analysis-at-scale-an-instagram-story-8f498ab71a0c https://instagram-engineering.com/static-analysis-at-scale-a...
    - VWWHFSfQ 3y ago
      I would say that those things don't really contribute to the actual runtime scalability of the application. Those things ensure that the code is at least correct for what it's supposed to do, not necessarily whether it's performant. Even the most perfectly-written Python code will still be dog-slow and require an enormous budget to actually run at scale. It's obviously fine to do that and there are many success stories, but all of them required massive infrastructure investment.
      - ambivalence 3y ago
        > Even the most perfectly-written Python code will still be dog-slow and require an enormous budget to actually run at scale. Nah, asyncio on uvloop is plenty fast.
      - Yoric 3y ago
        For context, I was asking about scaling code complexity, not scaling performance. That being said, my personal experience suggests that both are really hard in Python, and not independent.
- nobleach 3y ago
  But this is what FB/Meta are known for. They managed to scale PHP for years. They did fork it into Hack. They do have their own extensions. But they've put a ton of time and engineering effort into it. They didn't do it because they thought PHP was a great language. It was because they already had a massive infrastructure and tons of engineers that were already productive. Doing the math they realized it'd be cheaper to keep using PHP. The same is true here. They spent so much money getting Instagram to scale. They created Cinder to try and fold their hacks/tweaks back into Python. Spinning up some Rust/Java/Nim/Zig/Pony/Whatever stack sounds like it'd be fun... but spinning up a stack with which you're already VERY familiar, sounds like a money-maker. I'm totally agreeing btw, I realize this came off like it might be a counterpoint.
- n_ary 3y ago
  Off-topic: it is not yet [global scale] because it is only available in US. Here in EU, I still get “not available in your country”.
  - blitzar 3y ago
    It is only unavailable in the EU. Something, something privacy regulations.
  - reaperducer 3y ago
    it is not yet [global scale] because it is only available in US. Here in EU, I still get “not available in your country”. It's not only available in the U.S. It's available in 100 countries. There's more to "global" than the E.U.
  - o1y32 3y ago
    Anything that is available in over 100 countries is global to me.
- superq 3y ago
  Throw a trillion cores at anything and it can be global scale.

Always the same people that get their names in the news. I would say that the amount of tweaks that prop up Python is no cause for celebration.

by mlrpfx 3y ago

Always the same people that get their names in the news. I would say that the amount of tweaks that prop up Python is no cause for celebration.

blitzar 3y ago

Yet here it is ... powering a global scale app. Too weak for someones hobby project, strong enough for global scale.

mlrtph100 3y ago
So Google and Dropbox, who switched to Golang, are hobby projects.
- cmrdporcupine 3y ago
  Choice of language is not what makes things succeed or fail in production. Team best practices, enforcement of coding style and technique, good project management, infrastructure, and team cohesion etc. And weeding out bad engineers who think that "switching to a new [language|framework|religion]" will solve all problems. 10+ years ago Facebook itself did a remarkable job of scaling up, with hardly any outages, a PHP LAMP stack thingy to one of the hugest traffic websites in the world. Meanwhile back then Twitter mucked around with every novel technology they could find or invent, and had constant outages. Also Google didn't "switch to Golang." There's plenty of C++, Java, Python there. I worked there for 10 years and encountered only a handful of Go projects. Lots of cloud services in C++. Would I personally choose Python for a project? No, I don't like it, and I work in Rust full time because I prefer it. But if I worked at Meta / Instagram and had a team of Python engineers and existing libraries & infrastructure, this would be the right approach.
  - aeyes 3y ago
    Also, what's really different from Instagram? You change the picture for a short text and many other features already existed - comments, likes, subscriptions, recommendations, personal messages... I think this is just an Instagram skin with some feature flags.
Yoric 3y ago
My few years of personal experience of using Python with even small-ish scale professional projects have been... underwhelming so far. I wonder how Meta (and Google, and all the others who write serious Python code) manage to keep things running – and I'm not talking (only) of performance, but largely of bugs. My personal suspicion is that the secret sauce is simply to have a sufficient budget to hire an army of code quality engineers. I could be wrong, though.
- ambivalence 3y ago
  In short: static analysis, unit tests, staging environments, effective log aggregation. The first one is a big productivity booster as it shows you bugs before you commit them. Here's a post about this, not exactly new but still describing the general principles very well: https://instagram-engineering.com/static-analysis-at-scale-an-instagram-story-8f498ab71a0c https://instagram-engineering.com/static-analysis-at-scale-a...
  - VWWHFSfQ 3y ago
    I would say that those things don't really contribute to the actual runtime scalability of the application. Those things ensure that the code is at least correct for what it's supposed to do, not necessarily whether it's performant. Even the most perfectly-written Python code will still be dog-slow and require an enormous budget to actually run at scale. It's obviously fine to do that and there are many success stories, but all of them required massive infrastructure investment.
    - ambivalence 3y ago
      > Even the most perfectly-written Python code will still be dog-slow and require an enormous budget to actually run at scale. Nah, asyncio on uvloop is plenty fast.
    - Yoric 3y ago
      For context, I was asking about scaling code complexity, not scaling performance. That being said, my personal experience suggests that both are really hard in Python, and not independent.
nobleach 3y ago
But this is what FB/Meta are known for. They managed to scale PHP for years. They did fork it into Hack. They do have their own extensions. But they've put a ton of time and engineering effort into it. They didn't do it because they thought PHP was a great language. It was because they already had a massive infrastructure and tons of engineers that were already productive. Doing the math they realized it'd be cheaper to keep using PHP. The same is true here. They spent so much money getting Instagram to scale. They created Cinder to try and fold their hacks/tweaks back into Python. Spinning up some Rust/Java/Nim/Zig/Pony/Whatever stack sounds like it'd be fun... but spinning up a stack with which you're already VERY familiar, sounds like a money-maker. I'm totally agreeing btw, I realize this came off like it might be a counterpoint.
n_ary 3y ago
Off-topic: it is not yet [global scale] because it is only available in US. Here in EU, I still get “not available in your country”.
- blitzar 3y ago
  It is only unavailable in the EU. Something, something privacy regulations.
- reaperducer 3y ago
  it is not yet [global scale] because it is only available in US. Here in EU, I still get “not available in your country”. It's not only available in the U.S. It's available in 100 countries. There's more to "global" than the E.U.
- o1y32 3y ago
  Anything that is available in over 100 countries is global to me.
superq 3y ago
Throw a trillion cores at anything and it can be global scale.