3 ms·
I don't think it's fair to say Julia's problems haven't changed at all. While compilation latency is still an ongoing issue, it has consistently and noticeably
by lhn 6y ago
I don't think it's fair to say Julia's problems haven't changed at all. While compilation latency is still an ongoing issue, it has consistently and noticeably improved over the years. Package caching is much better, you can save compilation results you depend on in your workflow with PackageCompiler, etc. There are now incredible tools (e.g., SnoopCompile.jl) for package developers to inspect closely where the compiler might have difficulty and fix the issues.
The major source of improvements in 1.6 is eliminating method invalidations. Julia's flexibility makes it vulnerable to invalidating already compiled code as new packages are loaded and new methods are defined. This triggers a disastrous cascade of recompiling a bunch of things, and is the main conceptual reason why nothing lower than type-inferred code is cached. If your method will be recompiled anyway, then what good is it to save the native code in the first place? Now that invalidations can be efficiently diagnosed and patched, there is definitely interest into caching lower levels of code in the compilation process, potentially even native machine code.
All of these progress however requires labor and care. I'd say the Julia community has spent an admirable amount of efforts into its latency issue, but there's a limit to how fast you can address these problems through open-source development without backing from major tech companies. Imagine the improvements to the Julia compiler had Google chose Julia for its S4TF project, for instance.