7 ms·
Shopify strategy aka the story of YJIT If I cannot refactor my services, I shall refactor Ruby instead.
by tuananh 2y ago
Shopify strategy aka the story of YJIT
If I cannot refactor my services, I shall refactor Ruby instead.
- FBISurveillance 2y agoSince when contributing back to the community is considered a bad faith move?
- t-writescode 2y agoThat's certainly not what I get out of what they said. Shopify has introduced a bunch of very nice improvements to the usability of the Ruby language and their introductions have been seen in a very positive light. Also, I'm pretty sure both Shopify for Ruby and Facebook for their custom PHP stuff are both considered good moves.
- pjmlp 2y agoThat has been the story of every dynamic language since forever, thankfully the whole AI focus has made JITs finally matter in CPython world as well. Personally I have learnt this lesson back in 2000's, in the age of AOLServer, Vignette, and our own Safelayer product. All based on Apache, IIS and Tcl. We were early adopters of .NET, when it was only available to MSFT Partners and never again, using scripting languages without compilers, for full blown applications. Those learnings are the foundations of OutSystems, same ideas, built with a powerful runtime, with the hindsight of our experiences.
- sonudfgh 2y ago[dead]
- rapind 2y ago> Personally I have learnt this lesson back in 2000's, in the age of AOLServer, Vignette, and our own Safelayer product. All based on Apache, IIS and Tcl. Woah, your mention of “Vignette” just brought back a flood of memories I think my subconscious may have blocked out to save my sanity.
- Chetan496 2y agoHey, I have worked on the Outsystems platform. Developed some applications. Do you work at Outsystems?
- pjmlp 2y agoNo, I worked with the founders at a previous startup, Intervento, which became part of an EasyPhone acquisition, which got later renamed into Altitude Software alongside other acquisitions. They eventually left and founded OutSystems with what we learned since the Intervento days, OutSystems is of the greatest startup stories in the Portuguese industry. This was all during dotcom wave from the 2000's, instead I left to CERN.
- rat87 2y agoWhat's a scripting language? Also I'm not sure for TCL (https://news.ycombinator.com/item?id=24390937 https://news.ycombinator.com/item?id=24390937 claims it's had a bytecode compiler since around 2000) but the main python and Ruby implementations have compilers (compile to bytecode then interpret the bytecode). Apparently ruby got an optional (has to be enabled) jit compiler recently and python has an experimental jit in the last release (3.13).
- igouy 2y ago"... the distinguishing feature of interpreted languages is not that they are not compiled, but that any eventual compiler is part of the language runtime and that, therefore, it is possible (and easy) to execute code generated on the fly." p57 https://www.lua.org/pil/#1ed https://www.lua.org/pil/#1ed
- pjmlp 2y agohttps://en.m.wikipedia.org/wiki/Scripting_language https://en.m.wikipedia.org/wiki/Scripting_language
- zahlman 2y ago>thankfully the whole AI focus has made JITs finally matter in CPython world as well. Isn't most of the work in Python AI projects done in C or C++ extensions anyway?
- pjmlp 2y agoYes, but not everyone loves to have dual stack development, I surely didn't, back in the Tcl days, eventually we ask ourselves for how long.
- nerdponx 2y agoThat's not how it works in Python. The C/C++ is shipped in the form of well-established libraries like Numpy and PyTorch. Very few end users ever interact with the C/C++ parts, except for specialists with special requirements, and library contributors themselves.
- pjmlp 2y agoIt is definitely how it works in Python. As if there is nothing else to chose from regarding Python performance issues and libraries used by folks. Not everything is fashionable AI.
- zahlman 2y agoCan you name specific "un-fashionable" AI projects that are dependent on Python code for things that have any significant performance impact, which are seeing significant benefits from Python JIT implementations?
- pjmlp 2y agoI guess you will have to ask Microsoft, Facebook, NVidia and Intel why they are bothering then.
- 2y ago
- maccard 2y agoHHVM has raised its head.
- pjmlp 2y agoWhich happens to have a JIT compiler, and contributed to standard PHP having one as well.
- nerdponx 2y ago> AI The push for Python performance and JIT compilation has little to do with AI and more to do with Python's explosion in adoption for backend server applications in the 2010s, as well as the dedication of smaller projects like PyPy that existed largely because it was possible to make them exist. The ML/AI boom helped spread Python even farther and wider, yes, but none of the core language performance improvements are all that relevant for ML or AI. As another commenter pointed out, the performance bottlenecks in AI specifically have essentially to do with the CPython runtime performance. The only exception is in the pre-processing of very large text corpora, and that alone has hardly been a blip on the radar of the people working on CPython performance. Moreover, most of the "Python performance" projects that do sit closer to machine learning use cases (Cython-Numpy integration, Numba, Nuitka) are more or less orthogonal to the more recent push for Python interpreter performance. Cython itself and MypyC are mainly relevant because they are intended to be general-ish purpose performance boosters for CPython, and in doing so helped fill the need for greater performance in "hot and loopy" code such as network protocols, linters, and iterators. Cython also acted as a convenient glue layer for ad-hoc C library binding. But neither project is all that closely related to AI or to the various JIT compilers that have arisen over the years.
- pjmlp 2y agoNot at all, given Facebook and Microsoft involvement into making CPython folks finally accept a JIT has to be part of the story, coupled by NVidia and Intel work on GPU JIT DSLs for Python.
- neonsunset 2y agoDuring their black friday / cyber monday load peak, Shopify averaged between ~0.85 and ~1.94 back-to-back RPS per CPU core. Take from that what you will. Reference: https://x.com/ShopifyEng/status/1863953413559472291 https://x.com/ShopifyEng/status/1863953413559472291
- kstrauser 2y agoOuch. I had no idea it was that much of a resource hog.
- cristianbica 2y agoObviously you meant for the whole infrastructure: ruby / rails workers, Mysql, Kafka, whatever other stuff their app needs (redis, memcache, etc), loadbalancers, infrastructure monitoring, etc.
- neonsunset 2y agoThis is correct! I thought this was clear but I guess not...
- ksec 2y agoIt is not because this is the first time I heard about back to back RPS. Which when come to think of it isn't too bad of a metric from a business POV. We can also infer that into how much saving YJIT provides. At this point Shopify is likely already getting a return of investment from YJIT.
- PhilippGille 2y agoYou seem to imply that everything they run is Ruby, but they're talking about 2.4 million CPU cores on their K8s cluster, where maybe other stuff runs as well, like their Kafka clusters [1] and Airflow [2]? [1] https://shopify.engineering/running-apache-kafka-on-kubernetes-at-shopify https://shopify.engineering/running-apache-kafka-on-kubernet... [2] https://shopify.engineering/lessons-learned-apache-airflow-scale https://shopify.engineering/lessons-learned-apache-airflow-s...
- htunnicliff 2y agoFor a stranger to the Ruby ecosystem, what are the benefits of YJIT?
- Malp 2y agoJust-in-time compilation of Ruby allowing you to elide a lot of the overhead of dynamic language features + executing optimized machine code instead of running in the VM / bytecode interpreter. For example, doing some loop unrolling for a piece of code with a known & small-enough fixed-size iteration. As another example, doing away with some dynamic dispatch / method lookup for a call site, or inlining methods - especially handy given Ruby's first class support for dynamic code generation, execution, redefinition (monkey patching). From https://railsatscale.com/2023-12-04-ruby-3-3-s-yjit-faster-while-using-less-memory/ https://railsatscale.com/2023-12-04-ruby-3-3-s-yjit-faster-w..., > In particular, YJIT is now able to better handle calls with splats as well as optional parameters, it’s able to compile exception handlers, and it can handle megamorphic call sites and instance variable accesses without falling back to the interpreter. > We’ve also implemented specialized inlined primitives for certain core method calls such as Integer#!=, String#!=, Kernel#block_given?, Kernel#is_a?, Kernel#instance_of?, Module#===, and more. It also inlines trivial Ruby methods that only return a constant value such as #blank? and specialized #present? from Rails. These can now be used without needing to perform expensive method calls in most cases.
- weaksauce 2y agoit makes ruby code faster than c ruby code so they are moving toward rewriting a lot of the core ruby stuff in ruby to take advantage of it. run time performance enhancing makes the language much faster.
- dismalaf 2y agoSame as the benefits of JIT compilers for any dynamic language; makes a lot of things faster without changing your code, by turning hot paths into natively compiled code.
- janice1999 2y agoClassic story. Didn't Dropbox do the same for Python? ANd Facebook for PHP (and then forked it)?