4 ms·

> Instead of having your program managed by a running side-by-side loader program, what if the compiler would “simply” update the memory of the running process?

> Instead of having your program managed by a running side-by-side loader program, what if the compiler would “simply” update the memory of the running process? You will most inevitably think I have gone completely crazy This isn't crazy - this is how just-in-time compilers work.

stingraycharles 5y ago
It is a bit crazy if you do it for hot code reloading, though, as the logic has now changed. In the case of JIT, it may swap the bytecode but the flow of execution remains the same. Hot code reloading usually requires a different approach.
- saagarjha 5y ago
  Yep, a JIT will typically use on-stack replacement to keep execution coherent. Hot code reloading has to deal with the semantics of the code changing out from under it, which is substantially more complicated to deal with (and many implementations do not really handle this very well).
- metadat 5y ago
  In the case of JIT, it may swap the Bytecode but the flow of execution remains the same. Are you able to elaborate on this further? 1a. Does this mean the process gets restarted during reload? 1b. If no restart, how are active threads handled? What if a thread should no longer exist post-reload?
  - saagarjha 5y ago
    JITs don’t need to reload code, they run the same code and tier it up to an optimized implementation when it gets hot. No execution context is lost, at least semantically.
    - chrisseaton 5y ago
      > JITs don’t need to reload code If I've inlined a method, and someone redefines that method in a language that allows that, then I'm going to need to reload that code.
      - tedunangst 5y ago
        Yes, but if a called function redefines its own parent, it still returns to the old code, not the new code.
        moonchild 5y ago
        That is true, but JITs nevertheless perform on-stack replacement for optimization purposes.
      - kaba0 5y ago
        I don’t know much about the internals of a JIT compiler, but wouldn’t it just (recursively) deoptimize the methods which include the inlined method and do a normal method call instead which will now go to the updated implementation?
- fulafel 5y ago
  JITs also support code reloading. Eg normal Clojure(Script) workflow is based on reloading both on JVM and JS platforms. But functions are not altered mid-execution, rather your top level invocation or main loop picks up the new versions of functions from the updated namespaces after reload. This seems more reliable in face of arg function argument changes.
deleted 5y ago
[deleted]
ianthehenry 5y ago
A typical JIT compiler runs in the same process as the executable as part of its runtime -- it's a program modifying itself. This is describing a compiler modifying the memory of a separate running process out from underneath it. Which does seem a little crazy, and the post doesn't go into details about how the compiler knows when it's reasonable to rewrite instructions -- it seems like you would need some runtime mechanism in the child to coordinate this, or you'd run the risk of... well, literally anything could happen, with the wrong patch at the wrong time. I feel like I might be missing something. The more I think about this the crazier it seems. :) (And if you had this inter-process channel for coordinating changes, then the child process could also just patch itself based on messages from the compiler, removing the need for special cross-process memory-writing privileges. LiveReload for native code...)
- estebank 5y ago
  > it seems like you would need some runtime mechanism in the child to coordinate this I had been thinking about what mechanism you could put in place to coordinate, but you made me realize that if your language provides some kind of event loop, then that would be the right place to have the "please stop while I push this old function under the rug" functionality. And languages that have async/await have to have such a runtime, which can make the whole thing completely transparent to the programmer. But you still need the inter-process comms to let the runtime know it should wait for a bit, so your last sentence still applies: at that point have the process rewrite itself.
- torginus 5y ago
  >how the compiler knows when it's reasonable to rewrite instructions JIT languages tend to be also GC languages, which tend to have a very good idea of when it's safe to stop a thread, and what the values in the registers and stack mean at a particular point in time.

stingraycharles 5y ago

It is a bit crazy if you do it for hot code reloading, though, as the logic has now changed. In the case of JIT, it may swap the bytecode but the flow of execution remains the same. Hot code reloading usually requires a different approach.

saagarjha 5y ago
Yep, a JIT will typically use on-stack replacement to keep execution coherent. Hot code reloading has to deal with the semantics of the code changing out from under it, which is substantially more complicated to deal with (and many implementations do not really handle this very well).
metadat 5y ago
In the case of JIT, it may swap the Bytecode but the flow of execution remains the same. Are you able to elaborate on this further? 1a. Does this mean the process gets restarted during reload? 1b. If no restart, how are active threads handled? What if a thread should no longer exist post-reload?
- saagarjha 5y ago
  JITs don’t need to reload code, they run the same code and tier it up to an optimized implementation when it gets hot. No execution context is lost, at least semantically.
  - chrisseaton 5y ago
    > JITs don’t need to reload code If I've inlined a method, and someone redefines that method in a language that allows that, then I'm going to need to reload that code.
    - tedunangst 5y ago
      Yes, but if a called function redefines its own parent, it still returns to the old code, not the new code.
      - moonchild 5y ago
        That is true, but JITs nevertheless perform on-stack replacement for optimization purposes.
    - kaba0 5y ago
      I don’t know much about the internals of a JIT compiler, but wouldn’t it just (recursively) deoptimize the methods which include the inlined method and do a normal method call instead which will now go to the updated implementation?
fulafel 5y ago
JITs also support code reloading. Eg normal Clojure(Script) workflow is based on reloading both on JVM and JS platforms. But functions are not altered mid-execution, rather your top level invocation or main loop picks up the new versions of functions from the updated namespaces after reload. This seems more reliable in face of arg function argument changes.

deleted 5y ago

[deleted]

ianthehenry 5y ago

A typical JIT compiler runs in the same process as the executable as part of its runtime -- it's a program modifying itself. This is describing a compiler modifying the memory of a separate running process out from underneath it. Which does seem a little crazy, and the post doesn't go into details about how the compiler knows when it's reasonable to rewrite instructions -- it seems like you would need some runtime mechanism in the child to coordinate this, or you'd run the risk of... well, literally anything could happen, with the wrong patch at the wrong time. I feel like I might be missing something. The more I think about this the crazier it seems. :) (And if you had this inter-process channel for coordinating changes, then the child process could also just patch itself based on messages from the compiler, removing the need for special cross-process memory-writing privileges. LiveReload for native code...)

estebank 5y ago
> it seems like you would need some runtime mechanism in the child to coordinate this I had been thinking about what mechanism you could put in place to coordinate, but you made me realize that if your language provides some kind of event loop, then that would be the right place to have the "please stop while I push this old function under the rug" functionality. And languages that have async/await have to have such a runtime, which can make the whole thing completely transparent to the programmer. But you still need the inter-process comms to let the runtime know it should wait for a bit, so your last sentence still applies: at that point have the process rewrite itself.
torginus 5y ago
>how the compiler knows when it's reasonable to rewrite instructions JIT languages tend to be also GC languages, which tend to have a very good idea of when it's safe to stop a thread, and what the values in the registers and stack mean at a particular point in time.