23 ms·

Cinder: Instagram's performance-oriented fork of CPython

msoad 5y ago
I love how Facebook and Instagram never went the route of "full rewrite" for their apps as they scaled. I my experience "Language X is slow and we could save Y switching to Z" is always a false promise. You can pick parts of the system that are costing a lot and port them to other language/frameworks to capture the bulk of savings while keeping your developers happy working on familiar code. Or if you'e big enough like Facebook, you can go see why X is slow and if it is possible to improve at the compiler level. Never disrupt the developers flow (Twitter did this with their Scala craze back in the day)
- re 5y ago
  > keeping your developers happy working on familiar code That turns pretty quickly into "developers being unhappy that they have to maintain legacy code written for the old language/platform," though, especially with any sort of employee turnover.
  - peterhunt 5y ago
    It doesn't have to be this way. If you're going to go all-in on the monolith approach like FB and Instagram did for their product code (at least when I was there), you need to have a central team whose entire job is scaling the codebase. This means actively refactoring other teams' legacy code, making sure that tests are fast, etc.
    - skrtskrt 5y ago
      Yeah I have seen this successfully done at a smallish (200 person) company. There was a "platform" team which was essentially an "un-suck the monolith team". Refactor slowly over time, turn a ball of mud into cleanly defined domain boundaries, parallelize the tests, etc. etc.
- kevingadd 5y ago
  I think while performance is not a proper justification, if you get other stuff in the bargain it can be really worth it. Rust, for example - the justification to switch is usually not just speed but safety and the safety enabling more optimizations, parallelism etc. If all you care about is speed you stay in C/C++. Sometimes the choice is also made when you already have to pay a transitional cost - at a previous employer we had to rewrite a bunch of code to move from an old version of PHP to a newer one, and a strong argument was made that at that point we should just adopt a better language since we already had to pay that cost. I think eventually a lot of stuff transitioned to Haskell as part of that process, even if other stuff stayed in PHP.
- halfmatthalfcat 5y ago
  Did Twitter actually contribute to Scala core though? They definitely created and contributed to various Scala libraries but they didn't fork/augment the Scala compiler like Facebook did with HipHop.
  - spullara 5y ago
    They did contribute to the JVM but directly, not by forking it.
    - pjmlp 5y ago
      And are nowadays one of the reference users of GraalVM in production.
      - spullara 5y ago
        Oh right! I forgot they moved to that.
- neya 5y ago
  I disagree, this logic doesn't work always unless you have investor money lying around. For a bootstrapped business, it could mean living or dying. As an example, a client of mine was paying close to $5000 per month in server costs simply because of the scale of traffic they had on their site. By re-writing Wordpress in Elixir, I was able to bring their costs down to $1000 odd per month. It's actually cheaper than what any of their competitors are paying for, in servers as well. This $4000 in savings actually allowed them to hire an additional full time developer to maintain their site. So, your logic only works for certain use cases, not all.
  - leesalminen 5y ago
    How much did you charge to rewrite WordPress in elixir though?
    - code-is-code 5y ago
      And how big was the risk to have the invested time wasted? How long did you block new feature developement?
      - neya 5y ago
        Safe to say they would've gone out of business if they kept burning that kind of money for the remainder of the year. This was around covid era, they didn't have a solid revenue stream from the traffic they got.
    - neya 5y ago
      I'm obviously not going to reveal actual numbers, but it's safe to they spent less than what they would have spent being on Wordpress for the remainder of the year. Time to completion is 3 months. No new "feature" development was blocked. Remember - this is a content website (Wordpress is mostly used for content AFAIK and less for building features). The moment it was done, we just switched nameservers. It had no impact on client's workflow or day to day life.
      - 5y ago
- spullara 5y ago
  If Twitter's port to the JVM didn't result in 10x fewer servers with 10x lower latency vs an identical Ruby implementation we wouldn't have done it. Sadly not much progress has been made on the RubyVM though Twitter did try with improvements to GC and other changes. We even tried JRuby as part of the evaluation.
  - tekknolagi 5y ago
    Shopify is working on this! They have YJIT, led by maximecb
  - nerdponx 5y ago
    Is TruffleRuby promising at all?
    - jashmatthews 5y ago
      Enormously. The latest Railsbench results on TruffleRuby are ~3x CRuby.
      - spullara 5y ago
        Nice. 10 years later but glad that Ruby programmers might have an option that lets them program the way they want without having to choose a different language / platform for performance.
        ksec 5y ago
        It is still no where near the level that Twitter would have stayed with Ruby though. Although I believe TruffleRuby being 3x faster than current CRuby is only the beginning. JVM got tons of improvement in the past 10 years as well it is uncanny. Although Dr Chris Seaton mentioned it is not his goal to have Shopify Rails running on top of TurffleRuby. Not sure if he meant it was short term or long term. I think Dr Stefan Marr will be joining him soon, and Dr Maxime Chevalier-Boisvert is leading a team working on YJIT in Shopify. ( Did Shopify just hire all known Dr. working on Ruby performance )
- cageface 5y ago
  "Language X is slow and we could save Y switching to Z" is always a false promise. The thing that keeps software interesting for me is that there are almost no absolute rules. So I'll agree that complete rewrites are usually a mistake and performance problems are often not in the language. But I can't agree that rewrites are always a mistake. esbuild is a recent good example of how much difference switching to a faster language can make.
- mgraczyk 5y ago
  Idk, I worked at IG for a year and wrote a lot of backend code in the Python webserver stack. I believe they would benefit massively by aggressively migrating services to the FB Hack stack. A ton of work goes into making sure the Python code isn't slow, and there are complicated C++ services built as workarounds for the slowness of python.
  - vpql 5y ago
    Heresy! This is a Python thread, please summon the required positivity!
- nemothekid 5y ago
  I'm not sure many companies have the luxury of doing what FB did. I can't imagine a world where choosing to rewrite your app, or choosing to rewrite the entire language, where rewriting the entire language would be the cheaper solution. Twitter did this with their Scala craze, but Twitter struggled to scale it's Ruby app. Twitter didn't have nearly as deep pockets to write a performant alternative Ruby VM.
  - deleted 5y ago
    [deleted]
  - jfoutz 5y ago
    Joel Spolsky did it, city something?
    - chucksmash 5y ago
      I believe the product was called "CityDesk" and the in-house language that compiled to several different target languages was called Wasabi. Edit: I was wrong, Fog Creek did have a "CityDesk" product at one point, but Wasabi was for on-prem FogBugz[1]. [1]: https://www.joelonsoftware.com/2006/09/01/wasabi/ https://www.joelonsoftware.com/2006/09/01/wasabi/
- flakiness 5y ago
  Yeah, it kind of reminded me the "python strict mode" they talked about before [1]. No designed to be cool but to be useful. [1] https://instagram-engineering.com/python-at-scale-strict-modules-c0bb9245c834 https://instagram-engineering.com/python-at-scale-strict-mod...
  - czardoz 5y ago
    Yes, Static Python especially relies heavily on strict modules, since they enable us to perform module-local analysis, which enables some cool optimizations.
  - bjoli 5y ago
    This is something that people have known for a long time: a lot of money has been spent on making compilers make fast code. For many intents and purposes we know what kind of reasoning we want to be able to do to produce fast code. Strict modules is probably one of the sinples yet efficient optimizations you can do: being able to reason about where and what something is without having to do an expensive lookup every time it is used is brain dead simple to understand. Yet many of the languages that evolved as "scripting languages" threw much of that knowledge out. In the middle of the 90s we knew how to make fast dynamic languages. Maybe not as crazy dynamic as something like ruby, but not far off. Or at least, they had the sense to codify it as a separate thing (CLOS in common lisp comes to mind). Something like SBCL is more dynamic than you need(TM), yet produces code that is often at least an order of magnitude faster than python.
- joe_the_user 5y ago
  One factor for Twitter was that Ruby was simply orders of magnitude worse for Twitter usage than I think Php or Python was for Facebook or instagram. Or at least one never heard of these providing horror stories where Twitter definitely had horror stories. In one of the few times I've been "close to the source", I heard Twitter's head of operations complaining the electricity had become their primary expense from starting X many Ruby instances per second. It was one instance per connection or something close, because Ruby was unstable and because Ruby never gave memory back to the system.
- drunkenmagician 5y ago
  Not sure I can agree with that. Sinking resources in to re-engineering a language infrastructure, mostly because the language / infra is not actually a good choice for the target problem is an approach only very large engineering teams can tackle. As has been already been started (further down the thread). Breaking down your code / monolith into manageable components and re targeting for performance in the key spots is almost always the right approach. Unless of course you have the engineering resources and hubris of someone like FB
- deleted 5y ago
  [deleted]
- guenthert 5y ago
  Rewriting in full for the sake of performance gain is indeed a questionable proposal. Rewriting hot spots being the more sensible approach. A better reason for rewriting however is to migrate to a more maintainable language with focus on readability from one which focus on expressiveness (and hence favored for rapid development) as the project matures.
qotgalaxy 5y ago
Instagram is prematurely releasing this only because they heard that Tinder was about to release their own CPython fork.
EmilStenstrom 5y ago
tldr: Static Python plus Cinder JIT achieves 7x the performance of stock CPython
- miohtama 5y ago
  This is meaningful for some big web workloads. By cranking up your Python codebase performance you can avoid expensive rewrites in harder to maintain languages.
- nerdponx 5y ago
  I wonder why Cinder and not building off of PyPy. Isn't there also a GraalVM implementation?
  - _carljm 5y ago
    We tried PyPy without much success (and we really tried, six month project just to get it to run our codebase.) Our workload is very heavy on the CPython C API, which makes life difficult for PyPy or any other alternative Python implementation. PyPy is great for a lot of uses, just not a great fit for us.
    - neolog 5y ago
      Have you looked into the new hpy API? https://hpyproject.org/ https://hpyproject.org/
    - mattip 5y ago
      Sorry to hear that. What were the sticking points?
      - _carljm 5y ago
        It was a few years ago and I didn’t work on it directly, so my knowledge is a bit fuzzy, and I think the person who did work on it is no longer at the company. As best I recall the slog to get it working was mostly about C extensions, but I don’t know details. And then once it was working the memory use was worse and it didn’t show perf improvements. This is speculation but I would guess the design of the pypy jit isn’t a great fit for a prefork workload, where the hot paths don’t get hot until postfork, but it’s pretty inefficient to do identical jit compilation in every worker instead of once in the parent process.
varispeed 5y ago
> We don't have the capacity to support Cinder as an independent open-source project But they had capacity to appropriate CPython? Make billions out of open source software and then throw back a rag? "Here look we made this! How cool we are. See if you can find something for yourself and we might have _a conversation_" Man this is some next level entitlement. If they are making so much money out of it, the should have some decency and contribute stuff back to CPython and not just throwing things like this. This is offensive and disrespectful!
- tick_tock_tick 5y ago
  CPython explicitly doesn't want these "improvements" CPython has explicitly declined performance boosting pull requests if they add too much complexity to the code base. The two project have very different goals.
- mgraczyk 5y ago
  Instagram upstreams a lot of changes to cpython, and most of the IG backend runs on services outside of python/Django. I think you have the entitlement characterization backward though. It is entitled to think people should do work for free so that you can use the output of their work. IG is saying "we built this stuff, we're making it open source". They are not asking for or expecting anything from the open source community or you, but you are asking for a lot (millions of dollars worth of work) from them.
  - whoknowswhat11 5y ago
    Exactly - totally pathetic. Seriously - what are you contributing in terms of engineering budget? Nothing? Then stop bothering contributors to open source.
  - kbenson 5y ago
    > I think you have the entitlement characterization backward though. It is entitled to think people should do work for free so that you can use the output of their work. That depends on the license, though. It's not entitlement to think that someone that builds on something open must release it back if the license requires it. That's one of the main differences between the BSD license and some GPL variants.
deadmutex 5y ago
I wonder if they are in touch with kmod or tried pyston: https://blog.pyston.org/ https://blog.pyston.org/.
- _carljm 5y ago
  Haven't tried Pyston, its revival as an active project happened well after Cinder was in production.
- kingmaker 5y ago
  We’ve chatted with kmod a few times and let him know we were open sourcing Cinder. Hopefully the projects can learn from each other. As _carljm mentioned, Pyston was restarted way after Cinder was in production and we had already implemented a significant amount of the core JIT functionality.
ram_rar 5y ago
There is a lot of value in leveraging existing programming languages than rewriting in something else. My team rewrote large chunks of code from python -> Go and it wasn't a pleasant experience. We were able to justify it, since it _inadvertently_ reduced our infra cost. But, if we could make python a lot faster at compiler level, it doesn't break the dev experience and lengthen dev on-boarding time. This helps the team productivity as a whole. I hope the CPython community is able to leverage few things from Cinder. This would benefit the entire python community.
- adsharma 5y ago
  Did you rewrite it by hand or use a transpiler?
  - ram_rar 5y ago
    Mostly from scratch. I am yet to see a Python -> Go transpiler which generates human readable/maintainable code.
    - adsharma 5y ago
      Try http://github.com/adsharma/py2many http://github.com/adsharma/py2many tests/expected/*.go has generally human readable code.
- pm90 5y ago
  If you don’t mind sharing, what was it about the experience that made it unpleasant?
- antpls 5y ago
  Why didn't you profile the Python code and only rewrited the hot loops?
  - masklinn 5y ago
    Not op but two reasons would be: 1. The computations are distributed enough that there is no one hot loop 2. Go is shit at integrating with other software without IPC, ipc could be infeasible or have too much overhead, it also complicates building / bundling / shipping a lot
savant_penguin 5y ago
Really freaking cool Did anyone find the benchmarks
- _carljm 5y ago
  There'll be a talk on Cinder at (virtual) PyCon in a couple weeks, there are detailed benchmark numbers in the talk. We've talked about getting them into the repo too, might happen.
- _carljm 5y ago
  Benchmarks are now in a PERF file in the repo.
deleted 5y ago
[deleted]
smg 5y ago
Was Cinder influenced by hhvm (Facebook's vm for php/hack)? A project that maintains a list of different JIT implementations for programming languages and compares them would be a great way to see what are the different approaches to implementing JITs and which language features make it hard to implement performant JITs. As an aside it is great that the Cinder team is specifically calling out that Cinder is not intended to be used outside of FB. Many people have been burned by lack of community around hhvm.
- _carljm 5y ago
  Definitely influenced. There are people on our team who also worked on hhvm.
- kungito 5y ago
  I'm afraid there is very little documentation/text on modern production JITters. When I tried finding any text for my MSc I had little success. Does anyone have a suggestion about e.g. .NET 3-tier jitting or similar?
  - chrisseaton 5y ago
    > When I tried finding any text for my MSc I had little success. Yes you basically need to sit down with an expert to learn this stuff. It's famously under-documented and extremely hard to learn how it's done in practice on your own.
  - czardoz 5y ago
    Here's some good documentation about v8's JIT: https://github.com/thlorenz/v8-perf/blob/master/compiler.md https://github.com/thlorenz/v8-perf/blob/master/compiler.md Note: Never worked on v8, just liked the information here.
  - sanxiyn 5y ago
    I wrote down "Survey of tiered compilation in JIT implementations" when I researched this: https://github.com/sanxiyn/blog/blob/master/posts/2020-01-03.md https://github.com/sanxiyn/blog/blob/master/posts/2020-01-03...
boomer918 5y ago
This is awesome, a lot of knowledge shared in that code, and lots to learn from I'm sure.
dukeofdoom 5y ago
My experience with Django was that the larger the project got, the more I wanted to bypass it and just work directly with the database. Instead of mixing in python code in html. I just wanted to output quoted html in python. I got into weird issues with the parser. Like I wanted to output a dynamic style sheet. And it wasn't updating properly, probably because of nested includes. Its also much nicer to reason about a function outputting a block of html code that represents some object like a toolbar. Than to have python code in the html doing doing for loops with html code inside.
- DangitBobby 5y ago
  I've heard criticisms of django, but I've never heard someone say that they preferred to produce their HTML without templates used as one before.
  - dukeofdoom 5y ago
    Its probably because I needed a page to act differently if it was embedded or not, desktop or phone, and if the user was logged in or not. So in essence 2x2x2=8 different behaviours. Even things like links would need to change inside the page. Easier, and shorter to write a function that changes the output. Then to battle the templating language, or write and support 8 different views.
    - viraptor 5y ago
      I don't get it... templates sound like what you want here. You want links to change based on some conditions? Why not use {{ generate_link|safe }} on one generic template?
      - dukeofdoom 5y ago
        Thats what I ended up doing most of the time. Still ended up with a lot of if then code in the html template. Making it more verbose than it would have been, had I just generated the html page inside a function.
sergiomattei 5y ago
Yes, yes, yes! This is what I've been waiting for so long. Python has emphasized readability for the reference implementation vs. the practical benefits of better performance for everyone, and the community is really hurting for it. Please CPython, upstream this or something like it. There reaches a point where this whole zeal about readability becomes idealistic and hurts real-world pragmatic goals. It's hurting the language and we can do better. Python can be ahead in the performance race. We just need to get real.
- gonational 5y ago
  [deleted]
- jimbob45 5y ago
  Nim has long had the best of Python’s readability mixed with C’s performance and low-level constructs. I urge you to check it out. https://nim-lang.org/ https://nim-lang.org/
  - 1f60c 5y ago
    I think GP is talking about the readability of CPython itself.
  - tastyminerals2 5y ago
    I disagree, it was the Nim syntax that kept me away from the language back in the Uni when I tried it out. If anything, C syntax is barely as easy to read.
- StavrosK 5y ago
  That would indeed be fantastic. V8 has had so much optimization whereas CPython has had very little of it, though Pypy deserves a mention here. They've been doing fantastic work, but I guess there's always the curse of the reference implementation being the popular one.
- rtpg 5y ago
  So much stuff just from the readme would introduce breaking changes to the Python ecosystem. Part of the reason Instagram can get away with this is they likely have almost complete control of their dependencies and the like. But the changes they decide to make would not just work for everything. That being said I think we could get a lot of neat PEPs from this. EDIT: this is just my opinion , but in a world where we have type annotations, JITs feel like a massive step back. Stuff like mypyc could get us way further into high performance stuff (and no black box hand waving for perf things).
airstrike 5y ago
Never realized Carl Meyer was at Instagram, but this explains a lot now! His work on Django has always been superb
- tekknolagi 5y ago
  I thoroughly enjoy working with Carl.
- _carljm 5y ago
  Thank you, that’s very kind of you both. To be clear, this is the work of a team, and many others have contributed a lot more than I have to it.
TheRealPomax 5y ago
> We've made Cinder publicly available in order to facilitate conversation about potentially upstreaming some of this work to CPython and to reduce duplication of effort among people working on CPython performance. Nice. Looking forward to seeing that happen over the next few months.
- kubb 5y ago
  You mean years.
  - TheRealPomax 5y ago
    No, I don't. I'm talking about when we start seeing the changes, not when they're done.
pjfin123 5y ago
What's the difference between this and PyPy? They're both JIT/performance oriented Python interpreters? Good to see more work on this front! It's seemed crazy that Python can be this popular but still not be nearly as fast as it could be.
- jreese 5y ago
  Cinder is a fork of CPython, and maintains compatibility with C extensions written for CPython. Which includes a lot of important/popular third-party packages these days.
pjfin123 5y ago
For anyone curious about the detailed history of Python interpreters https://www.youtube.com/watch?v=NdJ9BxgRpOY https://www.youtube.com/watch?v=NdJ9BxgRpOY
throwawaaarrgh 5y ago
Too bad it's destined to be abandonware. Just think of all that money and engineering time that could have been spent working on CPython upstream, rather than a fork that'll die when the company gets bored with it. Aside from faster CPython, I'd love to see a PIP that standardizes module naming convention & inheriting+extending existing modules, rather than completely reinventing the wheel and making a totally new module just to add a single feature that some other module lacked. That plus the new regex extensions and Python will be almost as useful as Perl. :-)
- minedwiz 5y ago
  The readme does say that the goal here is to either get this work upstreamed to CPython or to inspire/inform improvements to it, not to replace/supplement it.
  - throwawaaarrgh 5y ago
    But it's probably not going to go upstream. And probably very little of it will go into other projects. Every time one of these forks gets dropped on the internet, nobody does anything with it, because it goes out of date shortly after it's dropped. Probably a couple specific optimizations will get picked up from it and put upstream. Versus if development had started and stayed in upstream, you wouldn't have to force people to pick through a huge patchset looking for things they personally want upstreamed.
    - nuclear_eclipse 5y ago
      We have multiple CPython core developers at Facebook, and the Cinder team (among others) are heavily involved in discussions around CPython's direction. There are many pieces of Cinder that would never get accepted upstream, or would take a decade of PEPs and incremental improvements to get there. Cinder is both a proof of concept that these ideas can work in CPython, and something that we can use today, rather than waiting a decade to realize these benefits from a pure open source perspective.
brian_herman 5y ago
I got it to work with the following but errored out on openssl so I couldn't install pip. It builds fine with docker fedora 32. Thank you. Platform Fedora 32, Docker on Windows 10, Version 2004 Steps to Reproduce Install docker from docker.com docker run -t -i fedora:32 bash git clone https://github.com/facebookincubator/cinder.git yum install zlib-devel openssl-devel ./configure make make altinstall This error is occurs when you do ./configure --enable-optimizations this version of python complies it either way. >> Objects/accu.o Parser/listnode.c: In function ‘list1node’: Parser/listnode.c:66:1: error: ‘/cinder/Parser/listnode.gcda’ profile count data file not found [-Werror=missing-profile] 66 | } | ^ If you get this error it is because you used --enable-optimizations. github.com/brianherman https://www.linkedin.com/in/brian-herman-092919208/ https://www.linkedin.com/in/brian-herman-092919208/ Edit: Removing github flavored markdown Edit: Answering own question Edit: More formatting
- wheybags 5y ago
  Pet peeve: people who put Werror in the default build flags. Leave it off by default, turn it on in your ci!
  - nitrogen 5y ago
    For me, I'd rather get a bug report about a failure during compilation, than something more obscure and harder to find at runtime.
    - simfoo 5y ago
      The parent proposed gating for warnings in the CI, not (by default) on the developer build. I completely agree with this, as a dev I want to see warnings but I don't always want to have everything perfect from the start. The famous example here is -Wunused-arg. I DON'T want to fight this during development, when I often have prototyping stages where functions remain unimplemented intentionally. I very much want to avoid merging this into master though, so CI gating with -Werror it is.
brian_herman 5y ago
How to get it working with Docker on Windows 10. https://gist.github.com/brianherman/63b65cdd92675f4a83cec1c483f293c9 https://gist.github.com/brianherman/63b65cdd92675f4a83cec1c4...
lend000 5y ago
Are there performance comparisons with stock CPython and PyPy anywhere?
- _carljm 5y ago
  There are now benchmark comparisons with stock CPython in a PERF file in the repo.
RAULSILVADOS 5y ago
NFTTY it's really changing the game in the crypto space knowing how hoy the nft world it's right no , this proyect Will melt serious faces in this bulll run ,thanks TCL to put it out for us #thecryptolifestyle #nft #Poolz #tcl #mr100x https://www.youtube.com/watch?v=N8zcfalo https://www.youtube.com/watch?v=N8zcfalo
IceWreck 5y ago
Benchmarks: https://github.com/facebookincubator/cinder/blob/cinder/3.8/PERF https://github.com/facebookincubator/cinder/blob/cinder/3.8/... There are some perf hits, as well as all the improvements.
- rkimb 5y ago
  How does this compare to other interpreters?
- _carljm 5y ago
  It’s worth noting that we’ve never targeted perf on any of these benchmarks, so these are just the results that fell out from optimizing for our production workload, which is a totally different beast. Most of the hits are due to the default “JIT everything” behavior which is really bad if there are a lot of rarely called functions, but would be easy to fix. This is discussed in the README.
fcanesin 5y ago
Why didn’t they simply invested in PyPy? Something that would continue and be maintained because of the passion and conviction around the project.
- lacker 5y ago
  I think this section of the readme speaks to that: Is this supported? Short answer: no. We've made Cinder publicly available in order to facilitate conversation about potentially upstreaming some of this work to CPython and to reduce duplication of effort among people working on CPython performance. Cinder is not polished or documented for anyone else's use. We don't have the capacity to support Cinder as an independent open-source project, nor any desire for it to become an alternative to CPython. Facebook is basically just open-sourcing their changes to CPython, in order to make it easier to work with core CPython developers. For a company of Facebook's size, migrating to PyPy would probably be have been a much bigger endeavor than simply forking CPython. Since the goal of this open-sourcing is to improve CPython rather than to attract people to a competitor, I would interpret this work as "Facebook investing in CPython" rather than "Facebook investing in a CPython alternative".
csmpltn 5y ago
> "If a call to an async function is immediately awaited, we immediately execute the called function up to its first await. If the called function reaches a return without needing to await, we will be able to return that value directly without ever even creating a coroutine object or deferring to the event loop. This is a significant (~5%) CPU optimization in our async-heavy workload." Step 1: "Who needs Threads? Just async/await all the things!" Step 2: Build complex logic to untangle your nonsensical use of async/await and force things to run synchronously in the backend without the user knowing. See? A 5% CPU optimization right there!
- toxik 5y ago
  Asynchronous execution is not a replacement for parallel execution. That is not the value proposition of asynchronous execution models.
- pcwalton 5y ago
  > Step 2: Build complex logic to untangle your nonsensical use of async/await and force things to run synchronously in the backend without the user knowing. See? A 5% CPU optimization right there! Addressing your snark, I assume this optimization is targeted at places where you have a single "await" expression in the source that might call different functions at runtime, only some of which need to await.
  - _carljm 5y ago
    Or there are just different code paths in the called function (eg a “cache hit” code path and a “cache miss” one), and only some of the code paths need to await.
- nly 5y ago
  This exact optimization has been implemented in C++ coroutines via the 'initial_suspend' coroutine traits hook. ...and those are stackless coroutines under AOT compiled conditions, so it only saves a malloc.
karlding 5y ago
Looking through the repo, one change of note that doesn't seem to be documented in the README [0] is that this has an optimization to reduce Copy on Writes in forked workers (under the Py_IMMORTAL_INSTANCES preprocessor guard) by not dirtying pages via refcount changes. Here's the BPO issue [1] that tracks the upstreaming attempt. [0] https://github.com/facebookincubator/cinder/blob/9de726349eb95542e59018553b16622933440da3/README.rst https://github.com/facebookincubator/cinder/blob/9de726349eb... [1] https://bugs.python.org/issue40255 https://bugs.python.org/issue40255
- kingmaker 5y ago
  Good call out! I’ll make sure it ends up in the readme.
- _carljm 5y ago
  It’s also likely that this change, which is a big win for our prefork production environment, drags down our benchmark results. It makes every incref and decref more expensive.
Animats 5y ago
Why not PyPy, which is a real compiler, not an interpreter?
- sanxiyn 5y ago
  An answer from a Cinder developer here: https://news.ycombinator.com/item?id=27043486 https://news.ycombinator.com/item?id=27043486
smcameron 5y ago
Cinder is already something else: https://libcinder.org/ https://libcinder.org/ Is it that hard to google a name before choosing?
- actinium226 5y ago
  Pretty sure they were calling it cinder inside of instragram long before open sourcing it.
- pwrplus1 5y ago
  Came here to say that. They did the same thing with an iOS app called Paper. The ethics are strong with this one.
- _carljm 5y ago
  We've been calling it Cinder internally for years, long before we thought about opening it up. If we wanted to make it a big branded thing, we'd work on finding a unique name for that, but we aren't trying to do that, we're just opening the code to make it easier to share work with upstream and other people working on CPython performance. We hope libcinder keeps the mindshare around the "cinder" name.
wiz21c 5y ago
FT readme : > Cinder is not polished or documented for anyone else's use. We don't have the capacity to support Cinder as an independent open-source project I'm quite surprised by that. I understand that FB makes tons of money so I can't believe they can't have the capacity... Maybe they don't have the will (which I perfectly understand, it's just the wording) Anyway, it's great they share.
- kingmaker 5y ago
  > We don't have the capacity to support Cinder as an independent open-source project Our team is smaller than you might think! You also left off the last half of that sentence, which is quite important: "nor any desire for it to become an alternative to CPython." We'd love to see the ideas and implementations in Cinder end up in upstream CPython in some form or another. It'll make our lives easier in that we won't have to maintain as much forked code, and hopefully everyone else will benefit from a faster CPython.
  - wiz21c 5y ago
    > Our team is smaller than you might think! I thought so but the other half of my brain was screaming for "FB has so much power, how come they don't invest more in open source". But well, that's economics at play I guess. I'm an idealist after all... Thx for replying !
- nly 5y ago
  Instagram were heavily using Python (and Django) when acquired by Facebook, whereas Facebook proper were a PHP shop. The most interesting aspect really is that both companies have run in to the limits of performance with dynamically typed languages and had to implement their own language runtimes, AOT compilers, and JIT's. Static typing is the best way forward at scale.
  - aeyes 5y ago
    Do we have proof that they had performance problems? At least in Python you can write a C extension for a problematic code path which would probably take much less time. I think what they are looking for is to reduce the amount of machines they need. Which in the end is more money in their pockets. Making the website faster for the end user is a welcome side effect.
BiteCode_dev 5y ago
>Is this supported? >Short answer: no. That's refreshingly honest
pjmlp 5y ago
Regarding the Static Python subset, Microsoft also has one flavour of it. https://makecode.com/python https://makecode.com/python
- _carljm 5y ago
  Static Python is a bit different in that it’s not a subset language. It makes a few semantic changes, most notably that classes are slotified, so you can’t tack random extra attributes into instances. If you’re writing typed Python you probably don’t do much of this anyway since all type checkers will complain about it. But the approach is “normal dynamic Python with optimizations where allowed by types,” not subset.
  - rlamy 5y ago
    How similar is it to RPython?
    - czardoz 5y ago
      Unlike RPython, Static Python in cinder is not really a subset of Python, it can compile everything (although it will throw compile time errors if it sees mismatched types). If it cannot determine type information, it just assumes the type could be anything, and falls back to slower CPython behavior.
jarpineh 5y ago
I wonder if Instagram being big Django site has driven Cinder development? What sort of impact Cinder has on their Django workloads? Regular models, views, ORM stuff can be fairly generic. Did they need to change Django to better benefit from Cinder? Edit: intriguing… https://github.com/facebookincubator/cinder/blob/f60897df9f6e31ff216ee4b2fc2037ced8747cd2/Tools/scripts/perf/prod_collect.py#L82 https://github.com/facebookincubator/cinder/blob/f60897df9f6...
- minusf 5y ago
  this podcast goes more into the details about django at instagram. https://djangochat.com/episodes/django-instagram-carl-meyer-lvSQm3aL https://djangochat.com/episodes/django-instagram-carl-meyer-... tldr: very old version with tons of custom stuff.
joelthelion 5y ago
This is cool, but at their scale I wonder if they wouldn't be better off rewriting most of their codebase, or at least a performance-critical subset, in something like Rust or Go. It may seem like a lot of work, but making Python faster isn't exactly trivial either. Also, I love Python, but using it for huge projects like this sounds dubious at best. I think the benefits of static typing really show when projects grow large.
stuaxo 5y ago
A bit annoying they called it cinder when there is already libcinder.
MichaelMoser123 5y ago
in the source there are quite a lot of references to a GIL (global interpreter lock). Does Cinder offer any improvements in terms of locking/GIL ? I didn't see any mention in the documentation. https://github.com/facebookincubator/cinder/search?q=gil https://github.com/facebookincubator/cinder/search?q=gil correction: the documentation does mention the lock: https://github.com/facebookincubator/cinder/blob/f60897df9f6e31ff216ee4b2fc2037ced8747cd2/Doc/c-api/init.rst#id83 https://github.com/facebookincubator/cinder/blob/f60897df9f6...
- gsnedders 5y ago
  > correction: the documentation does mention the lock: https://github.com/facebookincubator/cinder/blob/f60897df9f6 https://github.com/facebookincubator/cinder/blob/f60897df9f6... That's unchanged from the upstream CPython repo.
  - MichaelMoser123 5y ago
    today i learned that jython and ironpyton don't have a global lock like that. interesting.
AbuAssar 5y ago
We need some benchmarks to validate these bold claims. Edit: found them at https://github.com/facebookincubator/cinder/blob/cinder/3.8/PERF https://github.com/facebookincubator/cinder/blob/cinder/3.8/...
mangecoeur 5y ago
In many ways nice that they open their performance work, some of which could be upstreamed. In some ways a bit cheeky that they took the 'dump it and see' approach rather than offering to work to upstream it, since it's kinda outsourcing the work of maintaining the performance improvements to the Python core devs rather than offering to put some of Facebook's considerable resources towards doing it themselves.
- terminalcommand 5y ago
  What more could we expect from them to do? They are a for-profit company. Opensourcing their entire interpreter stack with years of proprietary development is a bold move. It shows that they care about open source and want to give back to the community. They could've kept these developments behind closed doors and nobody would mind it. They also offered to provide support if anyone wanted to contact them. That is top-class behaviour from instagram, I personally couldn't expect more.
  - black_puppydog 5y ago
    Not saying what they did was not good; it's certainly better than keeping things closed, but to answer these two parts: > What more could we expect from them to do? > That is top-class behaviour from instagram To be considered top-class, I would expect them to go the extra mile and contribute to upstream. As many other companies have done over the past and continue to do.
    - _carljm 5y ago
      We have contributed quite a few parts of cinder upstream already and will continue to do so. I’m not sure where you got the idea that this is just a dump and we expect core devs to do all the work of upstreaming.
      - black_puppydog 5y ago
        I'm very happy to hear that _carljm. I was just replying to my GP's "they took the 'dump it and see' approach" and my Parent's "What more could we expect from them to do?" If you're going that mile, then my hat is off to you and the team, that's how it's supposed to be done! :)
ful_vio 5y ago
Does this use PEP 523?
timkofu 5y ago
> Python can be ahead in the performance race. We just need to get real. Completely agree, and Instagram did it, and they have been reaping the benefits for it. Now they've open-sourced the runtime, so we can all benefit now too.

I love how Facebook and Instagram never went the route of "full rewrite" for their apps as they scaled. I my experience "Language X is slow and we could save Y switching to Z" is always a false promise. You can pick parts of the system that are costing a lot and port them to other language/frameworks to capture the bulk of savings while keeping your developers happy working on familiar code. Or if you'e big enough like Facebook, you can go see why X is slow and if it is possible to improve at the compiler level. Never disrupt the developers flow (Twitter did this with their Scala craze back in the day)

re 5y ago
> keeping your developers happy working on familiar code That turns pretty quickly into "developers being unhappy that they have to maintain legacy code written for the old language/platform," though, especially with any sort of employee turnover.
- peterhunt 5y ago
  It doesn't have to be this way. If you're going to go all-in on the monolith approach like FB and Instagram did for their product code (at least when I was there), you need to have a central team whose entire job is scaling the codebase. This means actively refactoring other teams' legacy code, making sure that tests are fast, etc.
  - skrtskrt 5y ago
    Yeah I have seen this successfully done at a smallish (200 person) company. There was a "platform" team which was essentially an "un-suck the monolith team". Refactor slowly over time, turn a ball of mud into cleanly defined domain boundaries, parallelize the tests, etc. etc.
kevingadd 5y ago
I think while performance is not a proper justification, if you get other stuff in the bargain it can be really worth it. Rust, for example - the justification to switch is usually not just speed but safety and the safety enabling more optimizations, parallelism etc. If all you care about is speed you stay in C/C++. Sometimes the choice is also made when you already have to pay a transitional cost - at a previous employer we had to rewrite a bunch of code to move from an old version of PHP to a newer one, and a strong argument was made that at that point we should just adopt a better language since we already had to pay that cost. I think eventually a lot of stuff transitioned to Haskell as part of that process, even if other stuff stayed in PHP.
halfmatthalfcat 5y ago
Did Twitter actually contribute to Scala core though? They definitely created and contributed to various Scala libraries but they didn't fork/augment the Scala compiler like Facebook did with HipHop.
- spullara 5y ago
  They did contribute to the JVM but directly, not by forking it.
  - pjmlp 5y ago
    And are nowadays one of the reference users of GraalVM in production.
    - spullara 5y ago
      Oh right! I forgot they moved to that.
neya 5y ago
I disagree, this logic doesn't work always unless you have investor money lying around. For a bootstrapped business, it could mean living or dying. As an example, a client of mine was paying close to $5000 per month in server costs simply because of the scale of traffic they had on their site. By re-writing Wordpress in Elixir, I was able to bring their costs down to $1000 odd per month. It's actually cheaper than what any of their competitors are paying for, in servers as well. This $4000 in savings actually allowed them to hire an additional full time developer to maintain their site. So, your logic only works for certain use cases, not all.
- leesalminen 5y ago
  How much did you charge to rewrite WordPress in elixir though?
  - code-is-code 5y ago
    And how big was the risk to have the invested time wasted? How long did you block new feature developement?
    - neya 5y ago
      Safe to say they would've gone out of business if they kept burning that kind of money for the remainder of the year. This was around covid era, they didn't have a solid revenue stream from the traffic they got.
  - neya 5y ago
    I'm obviously not going to reveal actual numbers, but it's safe to they spent less than what they would have spent being on Wordpress for the remainder of the year. Time to completion is 3 months. No new "feature" development was blocked. Remember - this is a content website (Wordpress is mostly used for content AFAIK and less for building features). The moment it was done, we just switched nameservers. It had no impact on client's workflow or day to day life.
    - 5y ago
spullara 5y ago
If Twitter's port to the JVM didn't result in 10x fewer servers with 10x lower latency vs an identical Ruby implementation we wouldn't have done it. Sadly not much progress has been made on the RubyVM though Twitter did try with improvements to GC and other changes. We even tried JRuby as part of the evaluation.
- tekknolagi 5y ago
  Shopify is working on this! They have YJIT, led by maximecb
- nerdponx 5y ago
  Is TruffleRuby promising at all?
  - jashmatthews 5y ago
    Enormously. The latest Railsbench results on TruffleRuby are ~3x CRuby.
    - spullara 5y ago
      Nice. 10 years later but glad that Ruby programmers might have an option that lets them program the way they want without having to choose a different language / platform for performance.
      - ksec 5y ago
        It is still no where near the level that Twitter would have stayed with Ruby though. Although I believe TruffleRuby being 3x faster than current CRuby is only the beginning. JVM got tons of improvement in the past 10 years as well it is uncanny. Although Dr Chris Seaton mentioned it is not his goal to have Shopify Rails running on top of TurffleRuby. Not sure if he meant it was short term or long term. I think Dr Stefan Marr will be joining him soon, and Dr Maxime Chevalier-Boisvert is leading a team working on YJIT in Shopify. ( Did Shopify just hire all known Dr. working on Ruby performance )
cageface 5y ago
"Language X is slow and we could save Y switching to Z" is always a false promise. The thing that keeps software interesting for me is that there are almost no absolute rules. So I'll agree that complete rewrites are usually a mistake and performance problems are often not in the language. But I can't agree that rewrites are always a mistake. esbuild is a recent good example of how much difference switching to a faster language can make.
mgraczyk 5y ago
Idk, I worked at IG for a year and wrote a lot of backend code in the Python webserver stack. I believe they would benefit massively by aggressively migrating services to the FB Hack stack. A ton of work goes into making sure the Python code isn't slow, and there are complicated C++ services built as workarounds for the slowness of python.
- vpql 5y ago
  Heresy! This is a Python thread, please summon the required positivity!
nemothekid 5y ago
I'm not sure many companies have the luxury of doing what FB did. I can't imagine a world where choosing to rewrite your app, or choosing to rewrite the entire language, where rewriting the entire language would be the cheaper solution. Twitter did this with their Scala craze, but Twitter struggled to scale it's Ruby app. Twitter didn't have nearly as deep pockets to write a performant alternative Ruby VM.
- deleted 5y ago
  [deleted]
- jfoutz 5y ago
  Joel Spolsky did it, city something?
  - chucksmash 5y ago
    I believe the product was called "CityDesk" and the in-house language that compiled to several different target languages was called Wasabi. Edit: I was wrong, Fog Creek did have a "CityDesk" product at one point, but Wasabi was for on-prem FogBugz[1]. [1]: https://www.joelonsoftware.com/2006/09/01/wasabi/ https://www.joelonsoftware.com/2006/09/01/wasabi/
flakiness 5y ago
Yeah, it kind of reminded me the "python strict mode" they talked about before [1]. No designed to be cool but to be useful. [1] https://instagram-engineering.com/python-at-scale-strict-modules-c0bb9245c834 https://instagram-engineering.com/python-at-scale-strict-mod...
- czardoz 5y ago
  Yes, Static Python especially relies heavily on strict modules, since they enable us to perform module-local analysis, which enables some cool optimizations.
- bjoli 5y ago
  This is something that people have known for a long time: a lot of money has been spent on making compilers make fast code. For many intents and purposes we know what kind of reasoning we want to be able to do to produce fast code. Strict modules is probably one of the sinples yet efficient optimizations you can do: being able to reason about where and what something is without having to do an expensive lookup every time it is used is brain dead simple to understand. Yet many of the languages that evolved as "scripting languages" threw much of that knowledge out. In the middle of the 90s we knew how to make fast dynamic languages. Maybe not as crazy dynamic as something like ruby, but not far off. Or at least, they had the sense to codify it as a separate thing (CLOS in common lisp comes to mind). Something like SBCL is more dynamic than you need(TM), yet produces code that is often at least an order of magnitude faster than python.
joe_the_user 5y ago
One factor for Twitter was that Ruby was simply orders of magnitude worse for Twitter usage than I think Php or Python was for Facebook or instagram. Or at least one never heard of these providing horror stories where Twitter definitely had horror stories. In one of the few times I've been "close to the source", I heard Twitter's head of operations complaining the electricity had become their primary expense from starting X many Ruby instances per second. It was one instance per connection or something close, because Ruby was unstable and because Ruby never gave memory back to the system.
drunkenmagician 5y ago
Not sure I can agree with that. Sinking resources in to re-engineering a language infrastructure, mostly because the language / infra is not actually a good choice for the target problem is an approach only very large engineering teams can tackle. As has been already been started (further down the thread). Breaking down your code / monolith into manageable components and re targeting for performance in the key spots is almost always the right approach. Unless of course you have the engineering resources and hubris of someone like FB
deleted 5y ago
[deleted]
guenthert 5y ago
Rewriting in full for the sake of performance gain is indeed a questionable proposal. Rewriting hot spots being the more sensible approach. A better reason for rewriting however is to migrate to a more maintainable language with focus on readability from one which focus on expressiveness (and hence favored for rapid development) as the project matures.

qotgalaxy 5y ago

Instagram is prematurely releasing this only because they heard that Tinder was about to release their own CPython fork.

EmilStenstrom 5y ago

tldr: Static Python plus Cinder JIT achieves 7x the performance of stock CPython

miohtama 5y ago
This is meaningful for some big web workloads. By cranking up your Python codebase performance you can avoid expensive rewrites in harder to maintain languages.
nerdponx 5y ago
I wonder why Cinder and not building off of PyPy. Isn't there also a GraalVM implementation?
- _carljm 5y ago
  We tried PyPy without much success (and we really tried, six month project just to get it to run our codebase.) Our workload is very heavy on the CPython C API, which makes life difficult for PyPy or any other alternative Python implementation. PyPy is great for a lot of uses, just not a great fit for us.
  - neolog 5y ago
    Have you looked into the new hpy API? https://hpyproject.org/ https://hpyproject.org/
  - mattip 5y ago
    Sorry to hear that. What were the sticking points?
    - _carljm 5y ago
      It was a few years ago and I didn’t work on it directly, so my knowledge is a bit fuzzy, and I think the person who did work on it is no longer at the company. As best I recall the slog to get it working was mostly about C extensions, but I don’t know details. And then once it was working the memory use was worse and it didn’t show perf improvements. This is speculation but I would guess the design of the pypy jit isn’t a great fit for a prefork workload, where the hot paths don’t get hot until postfork, but it’s pretty inefficient to do identical jit compilation in every worker instead of once in the parent process.

varispeed 5y ago

> We don't have the capacity to support Cinder as an independent open-source project But they had capacity to appropriate CPython? Make billions out of open source software and then throw back a rag? "Here look we made this! How cool we are. See if you can find something for yourself and we might have _a conversation_" Man this is some next level entitlement. If they are making so much money out of it, the should have some decency and contribute stuff back to CPython and not just throwing things like this. This is offensive and disrespectful!

tick_tock_tick 5y ago
CPython explicitly doesn't want these "improvements" CPython has explicitly declined performance boosting pull requests if they add too much complexity to the code base. The two project have very different goals.
mgraczyk 5y ago
Instagram upstreams a lot of changes to cpython, and most of the IG backend runs on services outside of python/Django. I think you have the entitlement characterization backward though. It is entitled to think people should do work for free so that you can use the output of their work. IG is saying "we built this stuff, we're making it open source". They are not asking for or expecting anything from the open source community or you, but you are asking for a lot (millions of dollars worth of work) from them.
- whoknowswhat11 5y ago
  Exactly - totally pathetic. Seriously - what are you contributing in terms of engineering budget? Nothing? Then stop bothering contributors to open source.
- kbenson 5y ago
  > I think you have the entitlement characterization backward though. It is entitled to think people should do work for free so that you can use the output of their work. That depends on the license, though. It's not entitlement to think that someone that builds on something open must release it back if the license requires it. That's one of the main differences between the BSD license and some GPL variants.

deadmutex 5y ago

I wonder if they are in touch with kmod or tried pyston: https://blog.pyston.org/ https://blog.pyston.org/.

_carljm 5y ago
Haven't tried Pyston, its revival as an active project happened well after Cinder was in production.
kingmaker 5y ago
We’ve chatted with kmod a few times and let him know we were open sourcing Cinder. Hopefully the projects can learn from each other. As _carljm mentioned, Pyston was restarted way after Cinder was in production and we had already implemented a significant amount of the core JIT functionality.

ram_rar 5y ago

There is a lot of value in leveraging existing programming languages than rewriting in something else. My team rewrote large chunks of code from python -> Go and it wasn't a pleasant experience. We were able to justify it, since it _inadvertently_ reduced our infra cost. But, if we could make python a lot faster at compiler level, it doesn't break the dev experience and lengthen dev on-boarding time. This helps the team productivity as a whole. I hope the CPython community is able to leverage few things from Cinder. This would benefit the entire python community.

adsharma 5y ago
Did you rewrite it by hand or use a transpiler?
- ram_rar 5y ago
  Mostly from scratch. I am yet to see a Python -> Go transpiler which generates human readable/maintainable code.
  - adsharma 5y ago
    Try http://github.com/adsharma/py2many http://github.com/adsharma/py2many tests/expected/*.go has generally human readable code.
pm90 5y ago
If you don’t mind sharing, what was it about the experience that made it unpleasant?
antpls 5y ago
Why didn't you profile the Python code and only rewrited the hot loops?
- masklinn 5y ago
  Not op but two reasons would be: 1. The computations are distributed enough that there is no one hot loop 2. Go is shit at integrating with other software without IPC, ipc could be infeasible or have too much overhead, it also complicates building / bundling / shipping a lot

savant_penguin 5y ago

Really freaking cool Did anyone find the benchmarks

_carljm 5y ago
There'll be a talk on Cinder at (virtual) PyCon in a couple weeks, there are detailed benchmark numbers in the talk. We've talked about getting them into the repo too, might happen.
_carljm 5y ago
Benchmarks are now in a PERF file in the repo.

deleted 5y ago

[deleted]

smg 5y ago

Was Cinder influenced by hhvm (Facebook's vm for php/hack)? A project that maintains a list of different JIT implementations for programming languages and compares them would be a great way to see what are the different approaches to implementing JITs and which language features make it hard to implement performant JITs. As an aside it is great that the Cinder team is specifically calling out that Cinder is not intended to be used outside of FB. Many people have been burned by lack of community around hhvm.

_carljm 5y ago
Definitely influenced. There are people on our team who also worked on hhvm.
kungito 5y ago
I'm afraid there is very little documentation/text on modern production JITters. When I tried finding any text for my MSc I had little success. Does anyone have a suggestion about e.g. .NET 3-tier jitting or similar?
- chrisseaton 5y ago
  > When I tried finding any text for my MSc I had little success. Yes you basically need to sit down with an expert to learn this stuff. It's famously under-documented and extremely hard to learn how it's done in practice on your own.
- czardoz 5y ago
  Here's some good documentation about v8's JIT: https://github.com/thlorenz/v8-perf/blob/master/compiler.md https://github.com/thlorenz/v8-perf/blob/master/compiler.md Note: Never worked on v8, just liked the information here.
- sanxiyn 5y ago
  I wrote down "Survey of tiered compilation in JIT implementations" when I researched this: https://github.com/sanxiyn/blog/blob/master/posts/2020-01-03.md https://github.com/sanxiyn/blog/blob/master/posts/2020-01-03...

boomer918 5y ago

This is awesome, a lot of knowledge shared in that code, and lots to learn from I'm sure.

dukeofdoom 5y ago

My experience with Django was that the larger the project got, the more I wanted to bypass it and just work directly with the database. Instead of mixing in python code in html. I just wanted to output quoted html in python. I got into weird issues with the parser. Like I wanted to output a dynamic style sheet. And it wasn't updating properly, probably because of nested includes. Its also much nicer to reason about a function outputting a block of html code that represents some object like a toolbar. Than to have python code in the html doing doing for loops with html code inside.

DangitBobby 5y ago
I've heard criticisms of django, but I've never heard someone say that they preferred to produce their HTML without templates used as one before.
- dukeofdoom 5y ago
  Its probably because I needed a page to act differently if it was embedded or not, desktop or phone, and if the user was logged in or not. So in essence 2x2x2=8 different behaviours. Even things like links would need to change inside the page. Easier, and shorter to write a function that changes the output. Then to battle the templating language, or write and support 8 different views.
  - viraptor 5y ago
    I don't get it... templates sound like what you want here. You want links to change based on some conditions? Why not use {{ generate_link|safe }} on one generic template?
    - dukeofdoom 5y ago
      Thats what I ended up doing most of the time. Still ended up with a lot of if then code in the html template. Making it more verbose than it would have been, had I just generated the html page inside a function.

sergiomattei 5y ago

Yes, yes, yes! This is what I've been waiting for so long. Python has emphasized readability for the reference implementation vs. the practical benefits of better performance for everyone, and the community is really hurting for it. Please CPython, upstream this or something like it. There reaches a point where this whole zeal about readability becomes idealistic and hurts real-world pragmatic goals. It's hurting the language and we can do better. Python can be ahead in the performance race. We just need to get real.

gonational 5y ago
[deleted]
jimbob45 5y ago
Nim has long had the best of Python’s readability mixed with C’s performance and low-level constructs. I urge you to check it out. https://nim-lang.org/ https://nim-lang.org/
- 1f60c 5y ago
  I think GP is talking about the readability of CPython itself.
- tastyminerals2 5y ago
  I disagree, it was the Nim syntax that kept me away from the language back in the Uni when I tried it out. If anything, C syntax is barely as easy to read.
StavrosK 5y ago
That would indeed be fantastic. V8 has had so much optimization whereas CPython has had very little of it, though Pypy deserves a mention here. They've been doing fantastic work, but I guess there's always the curse of the reference implementation being the popular one.
rtpg 5y ago
So much stuff just from the readme would introduce breaking changes to the Python ecosystem. Part of the reason Instagram can get away with this is they likely have almost complete control of their dependencies and the like. But the changes they decide to make would not just work for everything. That being said I think we could get a lot of neat PEPs from this. EDIT: this is just my opinion , but in a world where we have type annotations, JITs feel like a massive step back. Stuff like mypyc could get us way further into high performance stuff (and no black box hand waving for perf things).

airstrike 5y ago

Never realized Carl Meyer was at Instagram, but this explains a lot now! His work on Django has always been superb

tekknolagi 5y ago
I thoroughly enjoy working with Carl.
_carljm 5y ago
Thank you, that’s very kind of you both. To be clear, this is the work of a team, and many others have contributed a lot more than I have to it.

TheRealPomax 5y ago

> We've made Cinder publicly available in order to facilitate conversation about potentially upstreaming some of this work to CPython and to reduce duplication of effort among people working on CPython performance. Nice. Looking forward to seeing that happen over the next few months.

kubb 5y ago
You mean years.
- TheRealPomax 5y ago
  No, I don't. I'm talking about when we start seeing the changes, not when they're done.

pjfin123 5y ago

What's the difference between this and PyPy? They're both JIT/performance oriented Python interpreters? Good to see more work on this front! It's seemed crazy that Python can be this popular but still not be nearly as fast as it could be.

jreese 5y ago
Cinder is a fork of CPython, and maintains compatibility with C extensions written for CPython. Which includes a lot of important/popular third-party packages these days.

pjfin123 5y ago

For anyone curious about the detailed history of Python interpreters https://www.youtube.com/watch?v=NdJ9BxgRpOY https://www.youtube.com/watch?v=NdJ9BxgRpOY

throwawaaarrgh 5y ago

Too bad it's destined to be abandonware. Just think of all that money and engineering time that could have been spent working on CPython upstream, rather than a fork that'll die when the company gets bored with it. Aside from faster CPython, I'd love to see a PIP that standardizes module naming convention & inheriting+extending existing modules, rather than completely reinventing the wheel and making a totally new module just to add a single feature that some other module lacked. That plus the new regex extensions and Python will be almost as useful as Perl. :-)

minedwiz 5y ago
The readme does say that the goal here is to either get this work upstreamed to CPython or to inspire/inform improvements to it, not to replace/supplement it.
- throwawaaarrgh 5y ago
  But it's probably not going to go upstream. And probably very little of it will go into other projects. Every time one of these forks gets dropped on the internet, nobody does anything with it, because it goes out of date shortly after it's dropped. Probably a couple specific optimizations will get picked up from it and put upstream. Versus if development had started and stayed in upstream, you wouldn't have to force people to pick through a huge patchset looking for things they personally want upstreamed.
  - nuclear_eclipse 5y ago
    We have multiple CPython core developers at Facebook, and the Cinder team (among others) are heavily involved in discussions around CPython's direction. There are many pieces of Cinder that would never get accepted upstream, or would take a decade of PEPs and incremental improvements to get there. Cinder is both a proof of concept that these ideas can work in CPython, and something that we can use today, rather than waiting a decade to realize these benefits from a pure open source perspective.

brian_herman 5y ago

I got it to work with the following but errored out on openssl so I couldn't install pip. It builds fine with docker fedora 32. Thank you. Platform Fedora 32, Docker on Windows 10, Version 2004 Steps to Reproduce Install docker from docker.com docker run -t -i fedora:32 bash git clone https://github.com/facebookincubator/cinder.git yum install zlib-devel openssl-devel ./configure make make altinstall This error is occurs when you do ./configure --enable-optimizations this version of python complies it either way. >> Objects/accu.o Parser/listnode.c: In function ‘list1node’: Parser/listnode.c:66:1: error: ‘/cinder/Parser/listnode.gcda’ profile count data file not found [-Werror=missing-profile] 66 | } | ^ If you get this error it is because you used --enable-optimizations. github.com/brianherman https://www.linkedin.com/in/brian-herman-092919208/ https://www.linkedin.com/in/brian-herman-092919208/ Edit: Removing github flavored markdown Edit: Answering own question Edit: More formatting

wheybags 5y ago
Pet peeve: people who put Werror in the default build flags. Leave it off by default, turn it on in your ci!
- nitrogen 5y ago
  For me, I'd rather get a bug report about a failure during compilation, than something more obscure and harder to find at runtime.
  - simfoo 5y ago
    The parent proposed gating for warnings in the CI, not (by default) on the developer build. I completely agree with this, as a dev I want to see warnings but I don't always want to have everything perfect from the start. The famous example here is -Wunused-arg. I DON'T want to fight this during development, when I often have prototyping stages where functions remain unimplemented intentionally. I very much want to avoid merging this into master though, so CI gating with -Werror it is.

brian_herman 5y ago

How to get it working with Docker on Windows 10. https://gist.github.com/brianherman/63b65cdd92675f4a83cec1c483f293c9 https://gist.github.com/brianherman/63b65cdd92675f4a83cec1c4...

lend000 5y ago

Are there performance comparisons with stock CPython and PyPy anywhere?

_carljm 5y ago
There are now benchmark comparisons with stock CPython in a PERF file in the repo.

RAULSILVADOS 5y ago

NFTTY it's really changing the game in the crypto space knowing how hoy the nft world it's right no , this proyect Will melt serious faces in this bulll run ,thanks TCL to put it out for us #thecryptolifestyle #nft #Poolz #tcl #mr100x https://www.youtube.com/watch?v=N8zcfalo https://www.youtube.com/watch?v=N8zcfalo

IceWreck 5y ago

Benchmarks: https://github.com/facebookincubator/cinder/blob/cinder/3.8/PERF https://github.com/facebookincubator/cinder/blob/cinder/3.8/... There are some perf hits, as well as all the improvements.

rkimb 5y ago
How does this compare to other interpreters?
_carljm 5y ago
It’s worth noting that we’ve never targeted perf on any of these benchmarks, so these are just the results that fell out from optimizing for our production workload, which is a totally different beast. Most of the hits are due to the default “JIT everything” behavior which is really bad if there are a lot of rarely called functions, but would be easy to fix. This is discussed in the README.

fcanesin 5y ago

Why didn’t they simply invested in PyPy? Something that would continue and be maintained because of the passion and conviction around the project.

lacker 5y ago
I think this section of the readme speaks to that: Is this supported? Short answer: no. We've made Cinder publicly available in order to facilitate conversation about potentially upstreaming some of this work to CPython and to reduce duplication of effort among people working on CPython performance. Cinder is not polished or documented for anyone else's use. We don't have the capacity to support Cinder as an independent open-source project, nor any desire for it to become an alternative to CPython. Facebook is basically just open-sourcing their changes to CPython, in order to make it easier to work with core CPython developers. For a company of Facebook's size, migrating to PyPy would probably be have been a much bigger endeavor than simply forking CPython. Since the goal of this open-sourcing is to improve CPython rather than to attract people to a competitor, I would interpret this work as "Facebook investing in CPython" rather than "Facebook investing in a CPython alternative".

csmpltn 5y ago

> "If a call to an async function is immediately awaited, we immediately execute the called function up to its first await. If the called function reaches a return without needing to await, we will be able to return that value directly without ever even creating a coroutine object or deferring to the event loop. This is a significant (~5%) CPU optimization in our async-heavy workload." Step 1: "Who needs Threads? Just async/await all the things!" Step 2: Build complex logic to untangle your nonsensical use of async/await and force things to run synchronously in the backend without the user knowing. See? A 5% CPU optimization right there!

toxik 5y ago
Asynchronous execution is not a replacement for parallel execution. That is not the value proposition of asynchronous execution models.
pcwalton 5y ago
> Step 2: Build complex logic to untangle your nonsensical use of async/await and force things to run synchronously in the backend without the user knowing. See? A 5% CPU optimization right there! Addressing your snark, I assume this optimization is targeted at places where you have a single "await" expression in the source that might call different functions at runtime, only some of which need to await.
- _carljm 5y ago
  Or there are just different code paths in the called function (eg a “cache hit” code path and a “cache miss” one), and only some of the code paths need to await.
nly 5y ago
This exact optimization has been implemented in C++ coroutines via the 'initial_suspend' coroutine traits hook. ...and those are stackless coroutines under AOT compiled conditions, so it only saves a malloc.

karlding 5y ago

Looking through the repo, one change of note that doesn't seem to be documented in the README [0] is that this has an optimization to reduce Copy on Writes in forked workers (under the Py_IMMORTAL_INSTANCES preprocessor guard) by not dirtying pages via refcount changes. Here's the BPO issue [1] that tracks the upstreaming attempt. [0] https://github.com/facebookincubator/cinder/blob/9de726349eb95542e59018553b16622933440da3/README.rst https://github.com/facebookincubator/cinder/blob/9de726349eb... [1] https://bugs.python.org/issue40255 https://bugs.python.org/issue40255

kingmaker 5y ago
Good call out! I’ll make sure it ends up in the readme.
_carljm 5y ago
It’s also likely that this change, which is a big win for our prefork production environment, drags down our benchmark results. It makes every incref and decref more expensive.

Animats 5y ago

Why not PyPy, which is a real compiler, not an interpreter?

sanxiyn 5y ago
An answer from a Cinder developer here: https://news.ycombinator.com/item?id=27043486 https://news.ycombinator.com/item?id=27043486

smcameron 5y ago

Cinder is already something else: https://libcinder.org/ https://libcinder.org/ Is it that hard to google a name before choosing?

actinium226 5y ago
Pretty sure they were calling it cinder inside of instragram long before open sourcing it.
pwrplus1 5y ago
Came here to say that. They did the same thing with an iOS app called Paper. The ethics are strong with this one.
_carljm 5y ago
We've been calling it Cinder internally for years, long before we thought about opening it up. If we wanted to make it a big branded thing, we'd work on finding a unique name for that, but we aren't trying to do that, we're just opening the code to make it easier to share work with upstream and other people working on CPython performance. We hope libcinder keeps the mindshare around the "cinder" name.

wiz21c 5y ago

FT readme : > Cinder is not polished or documented for anyone else's use. We don't have the capacity to support Cinder as an independent open-source project I'm quite surprised by that. I understand that FB makes tons of money so I can't believe they can't have the capacity... Maybe they don't have the will (which I perfectly understand, it's just the wording) Anyway, it's great they share.

kingmaker 5y ago
> We don't have the capacity to support Cinder as an independent open-source project Our team is smaller than you might think! You also left off the last half of that sentence, which is quite important: "nor any desire for it to become an alternative to CPython." We'd love to see the ideas and implementations in Cinder end up in upstream CPython in some form or another. It'll make our lives easier in that we won't have to maintain as much forked code, and hopefully everyone else will benefit from a faster CPython.
- wiz21c 5y ago
  > Our team is smaller than you might think! I thought so but the other half of my brain was screaming for "FB has so much power, how come they don't invest more in open source". But well, that's economics at play I guess. I'm an idealist after all... Thx for replying !
nly 5y ago
Instagram were heavily using Python (and Django) when acquired by Facebook, whereas Facebook proper were a PHP shop. The most interesting aspect really is that both companies have run in to the limits of performance with dynamically typed languages and had to implement their own language runtimes, AOT compilers, and JIT's. Static typing is the best way forward at scale.
- aeyes 5y ago
  Do we have proof that they had performance problems? At least in Python you can write a C extension for a problematic code path which would probably take much less time. I think what they are looking for is to reduce the amount of machines they need. Which in the end is more money in their pockets. Making the website faster for the end user is a welcome side effect.

BiteCode_dev 5y ago

>Is this supported? >Short answer: no. That's refreshingly honest

pjmlp 5y ago

Regarding the Static Python subset, Microsoft also has one flavour of it. https://makecode.com/python https://makecode.com/python

_carljm 5y ago
Static Python is a bit different in that it’s not a subset language. It makes a few semantic changes, most notably that classes are slotified, so you can’t tack random extra attributes into instances. If you’re writing typed Python you probably don’t do much of this anyway since all type checkers will complain about it. But the approach is “normal dynamic Python with optimizations where allowed by types,” not subset.
- rlamy 5y ago
  How similar is it to RPython?
  - czardoz 5y ago
    Unlike RPython, Static Python in cinder is not really a subset of Python, it can compile everything (although it will throw compile time errors if it sees mismatched types). If it cannot determine type information, it just assumes the type could be anything, and falls back to slower CPython behavior.

jarpineh 5y ago

I wonder if Instagram being big Django site has driven Cinder development? What sort of impact Cinder has on their Django workloads? Regular models, views, ORM stuff can be fairly generic. Did they need to change Django to better benefit from Cinder? Edit: intriguing… https://github.com/facebookincubator/cinder/blob/f60897df9f6e31ff216ee4b2fc2037ced8747cd2/Tools/scripts/perf/prod_collect.py#L82 https://github.com/facebookincubator/cinder/blob/f60897df9f6...

minusf 5y ago
this podcast goes more into the details about django at instagram. https://djangochat.com/episodes/django-instagram-carl-meyer-lvSQm3aL https://djangochat.com/episodes/django-instagram-carl-meyer-... tldr: very old version with tons of custom stuff.

joelthelion 5y ago

This is cool, but at their scale I wonder if they wouldn't be better off rewriting most of their codebase, or at least a performance-critical subset, in something like Rust or Go. It may seem like a lot of work, but making Python faster isn't exactly trivial either. Also, I love Python, but using it for huge projects like this sounds dubious at best. I think the benefits of static typing really show when projects grow large.

stuaxo 5y ago

A bit annoying they called it cinder when there is already libcinder.

MichaelMoser123 5y ago

in the source there are quite a lot of references to a GIL (global interpreter lock). Does Cinder offer any improvements in terms of locking/GIL ? I didn't see any mention in the documentation. https://github.com/facebookincubator/cinder/search?q=gil https://github.com/facebookincubator/cinder/search?q=gil correction: the documentation does mention the lock: https://github.com/facebookincubator/cinder/blob/f60897df9f6e31ff216ee4b2fc2037ced8747cd2/Doc/c-api/init.rst#id83 https://github.com/facebookincubator/cinder/blob/f60897df9f6...

gsnedders 5y ago
> correction: the documentation does mention the lock: https://github.com/facebookincubator/cinder/blob/f60897df9f6 https://github.com/facebookincubator/cinder/blob/f60897df9f6... That's unchanged from the upstream CPython repo.
- MichaelMoser123 5y ago
  today i learned that jython and ironpyton don't have a global lock like that. interesting.

AbuAssar 5y ago

We need some benchmarks to validate these bold claims. Edit: found them at https://github.com/facebookincubator/cinder/blob/cinder/3.8/PERF https://github.com/facebookincubator/cinder/blob/cinder/3.8/...

mangecoeur 5y ago

In many ways nice that they open their performance work, some of which could be upstreamed. In some ways a bit cheeky that they took the 'dump it and see' approach rather than offering to work to upstream it, since it's kinda outsourcing the work of maintaining the performance improvements to the Python core devs rather than offering to put some of Facebook's considerable resources towards doing it themselves.

terminalcommand 5y ago
What more could we expect from them to do? They are a for-profit company. Opensourcing their entire interpreter stack with years of proprietary development is a bold move. It shows that they care about open source and want to give back to the community. They could've kept these developments behind closed doors and nobody would mind it. They also offered to provide support if anyone wanted to contact them. That is top-class behaviour from instagram, I personally couldn't expect more.
- black_puppydog 5y ago
  Not saying what they did was not good; it's certainly better than keeping things closed, but to answer these two parts: > What more could we expect from them to do? > That is top-class behaviour from instagram To be considered top-class, I would expect them to go the extra mile and contribute to upstream. As many other companies have done over the past and continue to do.
  - _carljm 5y ago
    We have contributed quite a few parts of cinder upstream already and will continue to do so. I’m not sure where you got the idea that this is just a dump and we expect core devs to do all the work of upstreaming.
    - black_puppydog 5y ago
      I'm very happy to hear that _carljm. I was just replying to my GP's "they took the 'dump it and see' approach" and my Parent's "What more could we expect from them to do?" If you're going that mile, then my hat is off to you and the team, that's how it's supposed to be done! :)

ful_vio 5y ago

Does this use PEP 523?

timkofu 5y ago

> Python can be ahead in the performance race. We just need to get real. Completely agree, and Instagram did it, and they have been reaping the benefits for it. Now they've open-sourced the runtime, so we can all benefit now too.