3 ms·

How difficult is lazy parsing for Ruby? How much parsing do you need to do just to find where the method body ends?

• chrisseaton 9y ago
  You basically have to do all the parsing, but you can delay creating the actual AST and other data structures like byte code, which for us is the really expensive bit.
  • tomstuart 9y ago
    Makes sense, thanks. So you end up redoing most of this (parsing) work later when you want the AST, but a) you might not have to redo it at all if the method's never actually called, and b) it's not the expensive bit anyway?
    • chrisseaton 9y ago
      Yes. But if you have source code that you know you will likely be requiring such as the standard library, you can do the initial parsing while compiling the Ruby VM, so you don't end up doing the parse twice at runtime. Long term what we hope to do is to provide a build of the Ruby VM that includes the version of Rails you are using pre-parsed. And then longer term we'd like to actually fully parse and initialise Rails (run the top level of the files which are loaded) during compilation, and freeze the heap and store it in the Ruby VM executable. When you run this special Ruby/Rails VM the Rails code is simply mmapped into your address space with all objects initialised and ready to go. Obviously it'll require some tweaking to delay doing this like starting the web server so that doesn't get run during compile time.
      • burke 9y ago
        We've talked about trying to implement this strategy (load everything, dump/restore the heap) with MRI, but our thought experiment was on the scale of a fully-booted application. In that context, it gets difficult to determine how to proceed when an application source file has changed, since re-loading isn't safe in ruby. It's a really interesting idea to pre-load the heap with just a set of libraries, which wouldn't be subject to as much change.