9 ms·

Cinder: Meta's internal performance-oriented production version of CPython

68 points by cpeterso 4y ago

robomartin 4y ago
The one thing that keeps bugging me about Python (ignoring speed for a moment) is the lack of attention to seamless pain-free deployment. There are two types: Web and desktop. Both are a mess. On the web the "easy button" is to pay for a dedicated service that is setup to specifically deal with the fact that deploying a Python/Django app to a Linux server is a nightmare and a mess. You can deploy a PHP application to any server you care to choose from any hosting company in the world. Grab a cheap GoDaddy server and off you go. It's that simple. With Python? Forget it. To some extent I blame this on the simplistic integrated development server trick. Sure, it's great, you install Django and you can see a site on your computer with no effort. And then comes the reality of moving whatever you created to the real world. On the desktop, well, again, it's a mess. As much as people love to dump on Visual Basic, the one thing they got absolutely right was that you could develop an application, test it locally, and have it in installable form to use internally or send to customers with zero effort. Back in the day we created dozens of tools, both for internal and external applications both with VB and VC++ (these were Windows-based) and Objective-C for Mac. Python does not make this simple at all. In my opinion, Python developers really need to take this aspect of how software is uses seriously. The only use for a language in isolation of creating tools people can use is in academia. Outside of that, we need to be able to deploy applications, be it web, desktop or mobile. Sometimes it seems zero thought or work has been given to this reality. How crazy is it? Try to create just a simple desktop console application. Something silly, like a console-based calculator or hangman game. Now try to package it in installable form for delivery to Mac or Windows users who need to double-click on an icon and install/uninstall the application as they would with anything else on their systems. Why can't Python do that when VB was doing it decades ago?
fragmede 4y ago
previously: https://news.ycombinator.com/item?id=27043217 https://news.ycombinator.com/item?id=27043217
- dang 4y ago
  Thanks! Macroexpanded: Cinder: Instagram's performance-oriented fork of CPython - https://news.ycombinator.com/item?id=27043217 https://news.ycombinator.com/item?id=27043217 - May 2021 (261 comments)
adamnemecek 4y ago
So much effort has gone into optimizing the worlds-most-ok language.
- cutierust 4y ago
  I am counting on Julia to remove this language from the fac e of earth. Fun fact, when the creator announced python beta he clearly said it was for building prototypes
  - scintill76 4y ago
    I've no love lost for Python, but in regards to what Python's creator may have said at the beginning, I'm reminded of the Linux announcement -- "just a hobby, won't be big and professional" and "NOT portable", "probably never will support anything other than AT-harddisks."
  - anakaine 4y ago
    Julia sounds wonderful on paper, but as soon as I want to work with the sorts of data that is my bread and butter, the ecosystem in Julia is empty, whilst Python is fully featured. Hard sell without huge investment.
    - snovv_crash 4y ago
      Which field is this? I find for my field it's really full featured, the part that is missing is the production-level stability and deployment process.
  - incrudible 4y ago
    Nah. You can always count on the scientific community producing tools that software engineers will scoff at. The opposite is not true.
    - jakobnissen 4y ago
      Spoken with the arrogance of a true software engineer. It's common in science to see computer scientists implement domain-specific software with a hopelessly naive understanding of the domain, leading to biased, wrong, or misinterpreted results. Like, go to any bioinformatics conference and you will find these people. "What do you mean I can't just use a database of clinical pathogens to make a tool that generalises to all bacteria?"
kosolam 4y ago
- kosolam 4y ago
nijave 4y ago
A lot of comments are suggesting to use better performing alternatives but they're ignoring the potential cost savings in hardware alone with small, incremental improvements. Performance might be fine but saving off 1-2% in hardware costs could be significant for a large scale app (like Instagram)
- BiteCode_dev 4y ago
  The thing is, it takes a huge scale for web apps before python becomes your bottleneck. Between caching, async, task queue, reverse proxes, databases and compiled extensions, you have so many performant options to glue with python that any hardware saving is more likely affected by your choice between redis, postgres and clickhouse than by the language you query them with.
  - dimator 4y ago
    Exactly. It's fine that the PSF and others are tackling performance seriously. But in addition to your point, in the 90% (made up on the spot) of the problems python is perfectly suited for, performance is simply not a consideration.
bratao 4y ago
Considering the Python team's paradigm shift regarding the tradeoff between simplicity and speed, and the creation of the Faster CPython team (Thanks Microsoft !) that is getting everything upstream very quickly, I expect a massive evolution in Python's speed in the next few years.
mihaic 4y ago
I still believe that the main reason Python is slow is because people kept repeating that Python is slow and just accepted this as unavoidable. Combined with how nobody sponsored the main work on something like this, even though everyone would have benefited, and sparked all these independent projects. Given what insane performance has been extracted from Javascript, I'd expect the limits for Python being within an order of magnitude from C++/Rust.
zloop 4y ago
The SBCL Lisp compiler (incidentally, the compiler is also called "python", which predates the PSF trademark ...) achieves much better results without millions of corporate dollars sunk into it. Speeding up Python has been a lucrative business for more than 10 years, with dozens of projects and very little results. I'd just use another language if speed is the issue.
- BiteCode_dev 4y ago
  And yet despite 30 years of people telling everyone how lisp is amazing, most people keep avoiding it despite that it's free, stable, and available everywhere. Meanwhile, new languages like rust or zig have no problem to be adopted, C#/Php/JS/python are still used massively and produced a lot of value. But it's probably everybody that is wrong, there is no way Python has a value proposal that is worth something to a lot of people.

68 points by cpeterso 4y ago

The one thing that keeps bugging me about Python (ignoring speed for a moment) is the lack of attention to seamless pain-free deployment. There are two types: Web and desktop. Both are a mess. On the web the "easy button" is to pay for a dedicated service that is setup to specifically deal with the fact that deploying a Python/Django app to a Linux server is a nightmare and a mess. You can deploy a PHP application to any server you care to choose from any hosting company in the world. Grab a cheap GoDaddy server and off you go. It's that simple. With Python? Forget it. To some extent I blame this on the simplistic integrated development server trick. Sure, it's great, you install Django and you can see a site on your computer with no effort. And then comes the reality of moving whatever you created to the real world. On the desktop, well, again, it's a mess. As much as people love to dump on Visual Basic, the one thing they got absolutely right was that you could develop an application, test it locally, and have it in installable form to use internally or send to customers with zero effort. Back in the day we created dozens of tools, both for internal and external applications both with VB and VC++ (these were Windows-based) and Objective-C for Mac. Python does not make this simple at all. In my opinion, Python developers really need to take this aspect of how software is uses seriously. The only use for a language in isolation of creating tools people can use is in academia. Outside of that, we need to be able to deploy applications, be it web, desktop or mobile. Sometimes it seems zero thought or work has been given to this reality. How crazy is it? Try to create just a simple desktop console application. Something silly, like a console-based calculator or hangman game. Now try to package it in installable form for delivery to Mac or Windows users who need to double-click on an icon and install/uninstall the application as they would with anything else on their systems. Why can't Python do that when VB was doing it decades ago?

fragmede 4y ago

previously: https://news.ycombinator.com/item?id=27043217 https://news.ycombinator.com/item?id=27043217

dang 4y ago
Thanks! Macroexpanded: Cinder: Instagram's performance-oriented fork of CPython - https://news.ycombinator.com/item?id=27043217 https://news.ycombinator.com/item?id=27043217 - May 2021 (261 comments)

adamnemecek 4y ago

So much effort has gone into optimizing the worlds-most-ok language.

cutierust 4y ago
I am counting on Julia to remove this language from the fac e of earth. Fun fact, when the creator announced python beta he clearly said it was for building prototypes
- scintill76 4y ago
  I've no love lost for Python, but in regards to what Python's creator may have said at the beginning, I'm reminded of the Linux announcement -- "just a hobby, won't be big and professional" and "NOT portable", "probably never will support anything other than AT-harddisks."
- anakaine 4y ago
  Julia sounds wonderful on paper, but as soon as I want to work with the sorts of data that is my bread and butter, the ecosystem in Julia is empty, whilst Python is fully featured. Hard sell without huge investment.
  - snovv_crash 4y ago
    Which field is this? I find for my field it's really full featured, the part that is missing is the production-level stability and deployment process.
- incrudible 4y ago
  Nah. You can always count on the scientific community producing tools that software engineers will scoff at. The opposite is not true.
  - jakobnissen 4y ago
    Spoken with the arrogance of a true software engineer. It's common in science to see computer scientists implement domain-specific software with a hopelessly naive understanding of the domain, leading to biased, wrong, or misinterpreted results. Like, go to any bioinformatics conference and you will find these people. "What do you mean I can't just use a database of clinical pathogens to make a tool that generalises to all bacteria?"

kosolam 4y ago

kosolam 4y ago

nijave 4y ago

A lot of comments are suggesting to use better performing alternatives but they're ignoring the potential cost savings in hardware alone with small, incremental improvements. Performance might be fine but saving off 1-2% in hardware costs could be significant for a large scale app (like Instagram)

BiteCode_dev 4y ago
The thing is, it takes a huge scale for web apps before python becomes your bottleneck. Between caching, async, task queue, reverse proxes, databases and compiled extensions, you have so many performant options to glue with python that any hardware saving is more likely affected by your choice between redis, postgres and clickhouse than by the language you query them with.
- dimator 4y ago
  Exactly. It's fine that the PSF and others are tackling performance seriously. But in addition to your point, in the 90% (made up on the spot) of the problems python is perfectly suited for, performance is simply not a consideration.

bratao 4y ago

Considering the Python team's paradigm shift regarding the tradeoff between simplicity and speed, and the creation of the Faster CPython team (Thanks Microsoft !) that is getting everything upstream very quickly, I expect a massive evolution in Python's speed in the next few years.

mihaic 4y ago

I still believe that the main reason Python is slow is because people kept repeating that Python is slow and just accepted this as unavoidable. Combined with how nobody sponsored the main work on something like this, even though everyone would have benefited, and sparked all these independent projects. Given what insane performance has been extracted from Javascript, I'd expect the limits for Python being within an order of magnitude from C++/Rust.

zloop 4y ago

The SBCL Lisp compiler (incidentally, the compiler is also called "python", which predates the PSF trademark ...) achieves much better results without millions of corporate dollars sunk into it. Speeding up Python has been a lucrative business for more than 10 years, with dozens of projects and very little results. I'd just use another language if speed is the issue.

BiteCode_dev 4y ago
And yet despite 30 years of people telling everyone how lisp is amazing, most people keep avoiding it despite that it's free, stable, and available everywhere. Meanwhile, new languages like rust or zig have no problem to be adopted, C#/Php/JS/python are still used massively and produced a lot of value. But it's probably everybody that is wrong, there is no way Python has a value proposal that is worth something to a lot of people.