3 ms·

Macro's are compile time, there is no runtime codegen. The problem was overly-frequent inlining generating enormous expressions, causing a lot JIT phase and sl

by dtech 10mo ago

Macro's are compile time, there is no runtime codegen. The problem was overly-frequent inlining generating enormous expressions, causing a lot JIT phase and slow execution.

munchler 10mo ago
Thank you for the clarification. If I understand correctly, these large expressions are created at compile-time, but the impact isn't felt until JIT occurs in the runtime environment. In that scenario, shouldn't the JIT just run once at startup, though? I'm still not quite understanding how JIT can take so much time in a production environment.
- hunterpayne 10mo ago
  Because the jit will let the unoptimized code run a few (hundred) times to take measurements to know what needs to be optimized and how it needs to be optimized. This is a good solution and makes hotspot very effective. The problem is that it happens randomly a few minutes/seconds into the operation of the service. So you randomly have a big pause with the performance hit everytime you run the service. The upside is that this only happens once. But you have to plan for a big performance hit to requests which are unlucky enough to be called at the wrong time.
  - pretzellogician 10mo ago
    And this can generally be avoided as well, by doing "warmup" when starting your service (effectively, mock some calls), but before accepting requests.
    - hunterpayne 10mo ago
      Of course, but then you have to actually do this. It is just another complexity to add. Also, I was answering a question about the hows and whys of the jit. I wasn't saying it was impossible to work around.
  - munchler 10mo ago
    Ah, that’s interesting. I wasn’t aware that JIT-ing will do that sort of performance analysis first. Thank you for the explanation.
    - still_grokking 10mo ago
      It won't in general. Doing so is a feature of high-end VM runtimes like the state of the art JVMs or JS runtimes.
gavinray 10mo ago
That's not true, Spark's entire query engine relies on use of runtime codegen via macros/quasi quotes Look up the architecture of Catalyst + Tungsten https://www.databricks.com/glossary/catalyst-optimizer https://www.databricks.com/glossary/catalyst-optimizer
- lmm 10mo ago
  Catalyst uses runtime codegen, sure, but the OP wasn't using that.
- still_grokking 10mo ago
  Scala 2's macros are compile time constructs. There is no runtime code generation / staging. Scala 3's macros support staged compilation, so you can have macros which create code in later stages at runtime. https://docs.scala-lang.org/scala3/reference/metaprogramming/staging.html https://docs.scala-lang.org/scala3/reference/metaprogramming...

by dtech 10mo ago

Macro's are compile time, there is no runtime codegen. The problem was overly-frequent inlining generating enormous expressions, causing a lot JIT phase and slow execution.

munchler 10mo ago

Thank you for the clarification. If I understand correctly, these large expressions are created at compile-time, but the impact isn't felt until JIT occurs in the runtime environment. In that scenario, shouldn't the JIT just run once at startup, though? I'm still not quite understanding how JIT can take so much time in a production environment.

hunterpayne 10mo ago
Because the jit will let the unoptimized code run a few (hundred) times to take measurements to know what needs to be optimized and how it needs to be optimized. This is a good solution and makes hotspot very effective. The problem is that it happens randomly a few minutes/seconds into the operation of the service. So you randomly have a big pause with the performance hit everytime you run the service. The upside is that this only happens once. But you have to plan for a big performance hit to requests which are unlucky enough to be called at the wrong time.
- pretzellogician 10mo ago
  And this can generally be avoided as well, by doing "warmup" when starting your service (effectively, mock some calls), but before accepting requests.
  - hunterpayne 10mo ago
    Of course, but then you have to actually do this. It is just another complexity to add. Also, I was answering a question about the hows and whys of the jit. I wasn't saying it was impossible to work around.
- munchler 10mo ago
  Ah, that’s interesting. I wasn’t aware that JIT-ing will do that sort of performance analysis first. Thank you for the explanation.
  - still_grokking 10mo ago
    It won't in general. Doing so is a feature of high-end VM runtimes like the state of the art JVMs or JS runtimes.

gavinray 10mo ago

That's not true, Spark's entire query engine relies on use of runtime codegen via macros/quasi quotes Look up the architecture of Catalyst + Tungsten https://www.databricks.com/glossary/catalyst-optimizer https://www.databricks.com/glossary/catalyst-optimizer

lmm 10mo ago
Catalyst uses runtime codegen, sure, but the OP wasn't using that.
still_grokking 10mo ago
Scala 2's macros are compile time constructs. There is no runtime code generation / staging. Scala 3's macros support staged compilation, so you can have macros which create code in later stages at runtime. https://docs.scala-lang.org/scala3/reference/metaprogramming/staging.html https://docs.scala-lang.org/scala3/reference/metaprogramming...