3 ms·

@ViralBShah and @StefanKarpinski I've had similar problems and so have many of my colleagues in silicon valley. No matter how you spin it and whatever benchmar

@ViralBShah and @StefanKarpinski I've had similar problems and so have many of my colleagues in silicon valley. No matter how you spin it and whatever benchmarks you show - Julia is a very slow fast language. :) The UX of Julia needs major work - everything is just slow. If you've used Python for 10 years and it is like running through molasses. Benchmarks are meaningless for the most part. This is what bothers me about Julia's marketing - it claims to be fast, but in reality, precompilation time alone is a no go unless you're doing high-compute large batch processing. Julia cannot be come a general purpose language unless you fix these issues. > These "conflations" sound suspiciously like excuses. I think you're the one conflating total end-to-end time spent between given a task and going home early to see my kids if I used Python.

ddragon 6y ago
While I hope they'll keep the pace of improving the JIT lag (perhaps even adding a fast optional interpreter or AoT compilation of parts of code), the benchmarks are fine considering compilation is a one time paid ticket, while runtime is paid by usage. It's not just high compute, if I set up a website in Julia on a docker after the "cold start" (which can be handled before using packagecompiler if I don't want any client to feel the slow JIT compilation, as I obviously should) then it should run at the benchmarked runtime speed even if each request is very short (as the compilation price was paid already). That applies to most services in general, and even during development Julia programmers eventually learn not to cold start all the times (keeping the session open until you stop programming, so you again only pay the cost once). Of course, any short task that must cold start like a bash script isn't convenient at all (and packagecompiler will generate a fat binary that runs fast, but it's still inconveniently fat if it's for something simple). And Julia's first impression is really important, which the "time to first plot" seems to affect the most (as you need to go deeper on the language to adapt to it's unusual workflow), so I agree that it's a UX priority.
ViralBShah 6y ago
Nobody is spinning anything here. Julia's compilation time is a known quantity. See my comment in https://news.ycombinator.com/item?id=24750559 https://news.ycombinator.com/item?id=24750559 about the different phases of Julia's compilation and execution (from a user's perspective). Obviously, if you are not working on a problem that doesn't need Julia's speed (and makes it worth paying the compilation cost), and you are more comfortable with a different tool, you should use that. You may in fact be better off using Python, or shell scripting or even Excel. Precompilation time in Julia is akin to `make` in a project with C code. You only compile the library once and use it repeatedly, until you update it. Is it a dealbreaker? Perhaps it is for you (assuming you are using precompilation in the right context). It is not for many who work with Julia day in and day out, and it is something that we continue to improve.
- fermienrico 6y ago
  Thanks for the insight. I think still for most engineering development - its mostly the one-off launch time that predominates. This is exactly why I don't use C++ for general purpose use. There is something to be said about how easy Python makes development look - despite of the package management warts. Btw, is there a reason why packages can't be precompiled and even distributed? I am sure you guys have thought about that.
  - cbkeller 6y ago
    > precompiled and even distributed There is work on this. This can be done with https://julialang.github.io/PackageCompiler.jl/dev/ https://julialang.github.io/PackageCompiler.jl/dev/, but so far only to a rather large binary / “bundle” IIUC, In principle it should be possible to do much better if you knew for sure at compile time which methods you would need to dispatch to for the data you ultimately want to run on.
    - ViralBShah 6y ago
      Yes, you can use PackageCompiler for your collection of packages. We are also exploring doing this in the background when you add more packages. The reason why we can't distribute precompiled .ji files at a package level is because of the way the package resolver works. It depends on the exact versions of dependencies of every package - and even for the same version of an end-user package, there can be slightly different versions of the dependencies installed depending on constraints imposed by other packages. One major improvement coming in 1.6 is multi-threaded precompilation, and it will leverage all the cores you have.
  - deleted 6y ago
    [deleted]

ddragon 6y ago

While I hope they'll keep the pace of improving the JIT lag (perhaps even adding a fast optional interpreter or AoT compilation of parts of code), the benchmarks are fine considering compilation is a one time paid ticket, while runtime is paid by usage. It's not just high compute, if I set up a website in Julia on a docker after the "cold start" (which can be handled before using packagecompiler if I don't want any client to feel the slow JIT compilation, as I obviously should) then it should run at the benchmarked runtime speed even if each request is very short (as the compilation price was paid already). That applies to most services in general, and even during development Julia programmers eventually learn not to cold start all the times (keeping the session open until you stop programming, so you again only pay the cost once). Of course, any short task that must cold start like a bash script isn't convenient at all (and packagecompiler will generate a fat binary that runs fast, but it's still inconveniently fat if it's for something simple). And Julia's first impression is really important, which the "time to first plot" seems to affect the most (as you need to go deeper on the language to adapt to it's unusual workflow), so I agree that it's a UX priority.

ViralBShah 6y ago

Nobody is spinning anything here. Julia's compilation time is a known quantity. See my comment in https://news.ycombinator.com/item?id=24750559 https://news.ycombinator.com/item?id=24750559 about the different phases of Julia's compilation and execution (from a user's perspective). Obviously, if you are not working on a problem that doesn't need Julia's speed (and makes it worth paying the compilation cost), and you are more comfortable with a different tool, you should use that. You may in fact be better off using Python, or shell scripting or even Excel. Precompilation time in Julia is akin to `make` in a project with C code. You only compile the library once and use it repeatedly, until you update it. Is it a dealbreaker? Perhaps it is for you (assuming you are using precompilation in the right context). It is not for many who work with Julia day in and day out, and it is something that we continue to improve.

fermienrico 6y ago
Thanks for the insight. I think still for most engineering development - its mostly the one-off launch time that predominates. This is exactly why I don't use C++ for general purpose use. There is something to be said about how easy Python makes development look - despite of the package management warts. Btw, is there a reason why packages can't be precompiled and even distributed? I am sure you guys have thought about that.
- cbkeller 6y ago
  > precompiled and even distributed There is work on this. This can be done with https://julialang.github.io/PackageCompiler.jl/dev/ https://julialang.github.io/PackageCompiler.jl/dev/, but so far only to a rather large binary / “bundle” IIUC, In principle it should be possible to do much better if you knew for sure at compile time which methods you would need to dispatch to for the data you ultimately want to run on.
  - ViralBShah 6y ago
    Yes, you can use PackageCompiler for your collection of packages. We are also exploring doing this in the background when you add more packages. The reason why we can't distribute precompiled .ji files at a package level is because of the way the package resolver works. It depends on the exact versions of dependencies of every package - and even for the same version of an end-user package, there can be slightly different versions of the dependencies installed depending on constraints imposed by other packages. One major improvement coming in 1.6 is multi-threaded precompilation, and it will leverage all the cores you have.
- deleted 6y ago
  [deleted]