6 ms·

After all the Julia hype on HN I thought I'd try it out. I was surprised to discover how slow it actually is to use. Here's a quick demo of booting up the Jul

by evmar 13y ago

After all the Julia hype on HN I thought I'd try it out. I was surprised to discover how slow it actually is to use. Here's a quick demo of booting up the Julia repl with two libraries loaded and then immediately exiting: $ time julia -e 'using Distributions; using Gadfly; exit()' real 0m22.778s With behavior like that it's not going to replace R as my go-to tool for graphically exploring data. I'm sure they'll improve this eventually. Much like with Rust, my hopes are high. But also much like with Rust, I wish the hype came after the promises were met. PS: this article claims that Julia 1.0 was released in 2012, but as best as I can tell the most recent code you can get calls itself 0.3. The above was with Version 0.3.0-prerelease+1174 (2014-01-23 16:58 UTC).

gelstudios 13y ago
Julia's JIT makes startup a little slow.
- thinkpad20 13y ago
  I'm not really familiar with the intricacies of JIT, but wouldn't it be possible to pre-compile just the startup portion of the code, thereby dramatically increasing startup time? This part of the code would seem to almost always run in exactly the same way each time (with respect to each individual package being loaded, the startups of each being also pre-compiled), so there doesn't seem to be any value gained from JIT-compiling this process there.
  - wtallis 13y ago
    Yes, Lisp systems have been doing exactly this for decades.
  - ihnorton 13y ago
    Yes, this is done now for the standard library, which has reduced time-to-REPL from ~3s to ~.5s. The same idea will eventually be extended to packages.
    - thinkpad20 13y ago
      Sweet, sounds good. I look forward to seeing this. Or maybe contributing to it :)
vanderZwan 13y ago
It's a Wired article, please don't confuse it with the Julia community itself[1] - in my experience the latter is very enthusiastic, very ambitious, but also very willing to admit the current limitations and that it's in a very early open beta stage. [1] Upon rereading, I don't think you are, but just making sure.
- evmar 13y ago
  Thanks for charitably interpreting my maybe-ambiguous comments! My general experience (seen also in e.g. Haskell) is that the core communities are usually quite reasonable, and it falls on journalists or newbies to create hype, which is what produces confusing articles like this one.
idunning 13y ago
Startup times are not fantastic with packages. This hasn't been a big area of focus yet, but there is a roadmap for getting this down. Its definitely a typo. Stable release is currently 0.2 (released near end of 2013) and upcoming version is 0.3 (a month or so away).
- carterschonwald 13y ago
  Unless julia switches to an ahead of time compilation model, startup times will fundamentally stay slow. LLVM can be used as a jit, but it's really tuned for ahead of time compilation and while the llvm devs want to support jit, their focus will always favor ahead of time compilation.
  - KenoFischer 13y ago
    There's no need to switch Julia to an AOT model, it suffices to supplement it with one, which is exactly what happens for the Julia standard library on master. It's only a question to extend that capability to packages, which is ongoing work.
    - thinkpad20 13y ago
      Couldn't the language simply cache the bytecode compiled when a package runs, and only reload if the timestamp on the compiled code is older than that on the source? I believe this is what python does with its .pyc files. Then it could go a step further and cache the binary generated by compiling the bytecode to machine code... seems like there's a lot of things that could be done to speed up package loading.
      - KenoFischer 13y ago
        This is basically what I meant. Julia doesn't really have a bytecode. There is a lower, type-inferred form of the AST as well as the llvm IR representation. However, instruction selection itself takes quite a bit of time, so if we're caching the IR, we might as well cache the generated machine code (the challenges are basically the same). As I said, it's work-in-progress (it's not that it's hard or anything, but it's a bit of work do it properly, which isn't done yet).
mturmon 13y ago
I don't think startup time, on its own, is a valuable metric for most use cases.
datawander 13y ago
Seriously? Startup time as a deal-breaker for using this? What about other, much more important and relevant metrics? If you have to spend 23 seconds but save, let's suppose, thirty minutes with faster development and runtime, isn't the tradeoff worth it? I'm not saying that's a guaranteed with Julia as I can't speak to that, but I think of all the things to care about, startup time would be a relatively trivial concern. Also, "one programming language to rule them all" is quite a fanciful stretch considering people having been trying to write that since the first programming language.
RivieraKid 13y ago
Downvoted because: 1) Julia's performance is great. What you're measuring is a precompilation time. 2) I hate that the top comment on HN is usually negative. Is that some stupid rule or something?
- claudius 13y ago
  @1) Is it? [0] seems to indicate that it is usually about twice as slow as C and sometimes fares much worse. Of course, that article is over a year old by now so maybe there are improvements, but if I can get a 2x performance gain ‘just’ by writing in C/C++, I will certainly do so. [0] http://justindomke.wordpress.com/2012/09/17/julia-matlab-and-c/#results http://justindomke.wordpress.com/2012/09/17/julia-matlab-and...
  - pessimizer 13y ago
    Are you making a case here that Matlab and R should never be used?
  - svantana 13y ago
    Note that one of the benchmarks take 5 ms in c -- if that's all you're trying to compute, why care about speed at all? And of course JIT will be a huge penalty on that time scale. There are a bunch of benchmarks on the front page of http://julialang.org/ http://julialang.org/ that show only a small penalty for using julia over c.
  - thinkpad20 13y ago
    There's almost always a performance tradeoff when choosing to use some other language than C/C++; how much of a performance cost is the key question. If it's consistently within a factor of 2 of C performance, that's actually quite good, considering all of the high-level benefits you get out of it.
  - RivieraKid 13y ago
    Compared to Python or MATLAB? Yes. 2x slower than C is amazing for a high-level and dynamic language. But of course, if performance is critical, C/C++ may be the better choice.
reeses 13y ago
Where is this "use" part of which you speak? Aggressive JIT languages are historically slow at startup. The typical tradeoff (similar to the server/client Java VM flags) is to amortize the optimizations over the length of the application if startup time is a factor. If you start Julia (or Mathematica, Excel, or whatever) up every time you need to 'graphically explore data', you're doing it wrong. Unless you fubar your environment (not that I ever do this on a regular basis), there isn't a great reason to restart your exploratory environment.
- rdtsc 13y ago
  > Where is this "use" part of which you speak Maybe you don't want to use the old repl, there there is something else going on there and you don't want to mess it up. Maybe I want to show someone an example, or try something out quickly. I use and spawn ipython windows all the time. Sometimes I have 4 open at the same time. > Aggressive JIT languages are historically slow at startup. Explaining the reason for the slow start-up doesn't make the slow startups faster.
  - deleted 13y ago
    [deleted]
  - coldtea 13y ago
    >> Aggressive JIT languages are historically slow at startup. Explaining the reason for the slow start-up doesn't make the slow startups faster. No, but insisting that you want startups faster, when you've been explained the technical reasons why they are slow, doesn't make them faster either. To me it sounds like: "This ball shouldn't fall to earth". "It fell because of gravity". "Explaining the reason it fell, doesn't make it go upwards".
    - rdtsc 13y ago
      > No, but insisting that you want startups faster, when you've been explained the technical reasons why they are slow, doesn't make them faster either. Oh please. "We've explained the technical reasons to you" excuse. As a customer the response to that is "ok, good luck with that, I am not using your product". > "This ball shouldn't fall to earth" / "It fell because of gravity" / "Explaining the reason it fell, doesn't make it go upwards". No the answer to that is, screw this planet, I'll go fly into space and find another planet. Which is the response people have with software they don't like. It is more like: "Oh your database is slow" and getting the response of "Well of course it is,the locks at level 3 in our API have not been optimized because John is on vacation still. Geez how dare you not accept that as a valid answer?" See you've equated software features hard laws of physics. So escaping from it is "ridiculous". To be more serious, having an interpreter JIT a module only if you use, on-demand, or save the trace of it, across execution maybe he hard but doesn't require changing the fine structure constant.
Derbasti 13y ago
Note that one of Julia's main competitor, Matlab, is laughably bad at this, too. Of course Julia has a slow startup because it is JIT-compiled and therefore will run fast. This analogy does not extend to Matlab.
StefanKarpinski 13y ago
You literally picked the single package that loads the most other packages. Clearly this needs work, but just running vanilla julia without loading packages is not bad at all these days: $ time julia -e 'println("Hello, world.")' Hello, world. real 0m0.209s user 0m0.299s sys 0m0.104s Julia now pre-compiles the jitted code for it's base system, and the plan is to allow the same thing for packages. As others have noted, starting up Matlab is extremely slow and starting up a full R environment isn't all that fast either, nor is starting a JVM all that speedy. Startup time alone doesn't prevent a system from being useful.
- evmar 13y ago
  Thanks for replying! I didn't anticipate that my comment would generate so much controversy. :( I was only relating my own experience in trying out the tool, and comparing it to the tool I currently use frequently despite not liking its semantics very much: R. When I had encountered the slowness I was complaining about above, I poked around online and saw that you all were certainly aware of it and had plans to improve it. I think I even downloaded a 0.2 binary and then tried out trunk to see if it had already improved. All I had meant to say was (like Rust) that I anticipate I'll need to wait for a more complete version.
- delian66 13y ago
  >>Startup time alone doesn't prevent a system from being useful. Well, it does not completely ... If I run a simulation that will take 10 hours, 2 seconds is not much at all. However, it will severely reduce its usefullness in many other cases (quick&dirty scripting, testing)... Just for the record, on an i5 system: $ time julia --version julia version 0.3.0-prerelease real 0m2.156s user 0m2.119s sys 0m0.030s $time julia -e 'println("Hello, world.")' Hello, world. real 0m2.119s user 0m2.074s sys 0m0.036s In comparison, starting an equivalent ipython script takes <0.1s on the same machine.

by evmar 13y ago

gelstudios 13y ago

Julia's JIT makes startup a little slow.

thinkpad20 13y ago
I'm not really familiar with the intricacies of JIT, but wouldn't it be possible to pre-compile just the startup portion of the code, thereby dramatically increasing startup time? This part of the code would seem to almost always run in exactly the same way each time (with respect to each individual package being loaded, the startups of each being also pre-compiled), so there doesn't seem to be any value gained from JIT-compiling this process there.
- wtallis 13y ago
  Yes, Lisp systems have been doing exactly this for decades.
- ihnorton 13y ago
  Yes, this is done now for the standard library, which has reduced time-to-REPL from ~3s to ~.5s. The same idea will eventually be extended to packages.
  - thinkpad20 13y ago
    Sweet, sounds good. I look forward to seeing this. Or maybe contributing to it :)

vanderZwan 13y ago

It's a Wired article, please don't confuse it with the Julia community itself[1] - in my experience the latter is very enthusiastic, very ambitious, but also very willing to admit the current limitations and that it's in a very early open beta stage. [1] Upon rereading, I don't think you are, but just making sure.

evmar 13y ago
Thanks for charitably interpreting my maybe-ambiguous comments! My general experience (seen also in e.g. Haskell) is that the core communities are usually quite reasonable, and it falls on journalists or newbies to create hype, which is what produces confusing articles like this one.

idunning 13y ago

Startup times are not fantastic with packages. This hasn't been a big area of focus yet, but there is a roadmap for getting this down. Its definitely a typo. Stable release is currently 0.2 (released near end of 2013) and upcoming version is 0.3 (a month or so away).

carterschonwald 13y ago
Unless julia switches to an ahead of time compilation model, startup times will fundamentally stay slow. LLVM can be used as a jit, but it's really tuned for ahead of time compilation and while the llvm devs want to support jit, their focus will always favor ahead of time compilation.
- KenoFischer 13y ago
  There's no need to switch Julia to an AOT model, it suffices to supplement it with one, which is exactly what happens for the Julia standard library on master. It's only a question to extend that capability to packages, which is ongoing work.
  - thinkpad20 13y ago
    Couldn't the language simply cache the bytecode compiled when a package runs, and only reload if the timestamp on the compiled code is older than that on the source? I believe this is what python does with its .pyc files. Then it could go a step further and cache the binary generated by compiling the bytecode to machine code... seems like there's a lot of things that could be done to speed up package loading.
    - KenoFischer 13y ago
      This is basically what I meant. Julia doesn't really have a bytecode. There is a lower, type-inferred form of the AST as well as the llvm IR representation. However, instruction selection itself takes quite a bit of time, so if we're caching the IR, we might as well cache the generated machine code (the challenges are basically the same). As I said, it's work-in-progress (it's not that it's hard or anything, but it's a bit of work do it properly, which isn't done yet).

mturmon 13y ago

I don't think startup time, on its own, is a valuable metric for most use cases.

datawander 13y ago

Seriously? Startup time as a deal-breaker for using this? What about other, much more important and relevant metrics? If you have to spend 23 seconds but save, let's suppose, thirty minutes with faster development and runtime, isn't the tradeoff worth it? I'm not saying that's a guaranteed with Julia as I can't speak to that, but I think of all the things to care about, startup time would be a relatively trivial concern. Also, "one programming language to rule them all" is quite a fanciful stretch considering people having been trying to write that since the first programming language.

RivieraKid 13y ago

Downvoted because: 1) Julia's performance is great. What you're measuring is a precompilation time. 2) I hate that the top comment on HN is usually negative. Is that some stupid rule or something?

claudius 13y ago
@1) Is it? [0] seems to indicate that it is usually about twice as slow as C and sometimes fares much worse. Of course, that article is over a year old by now so maybe there are improvements, but if I can get a 2x performance gain ‘just’ by writing in C/C++, I will certainly do so. [0] http://justindomke.wordpress.com/2012/09/17/julia-matlab-and-c/#results http://justindomke.wordpress.com/2012/09/17/julia-matlab-and...
- pessimizer 13y ago
  Are you making a case here that Matlab and R should never be used?
- svantana 13y ago
  Note that one of the benchmarks take 5 ms in c -- if that's all you're trying to compute, why care about speed at all? And of course JIT will be a huge penalty on that time scale. There are a bunch of benchmarks on the front page of http://julialang.org/ http://julialang.org/ that show only a small penalty for using julia over c.
- thinkpad20 13y ago
  There's almost always a performance tradeoff when choosing to use some other language than C/C++; how much of a performance cost is the key question. If it's consistently within a factor of 2 of C performance, that's actually quite good, considering all of the high-level benefits you get out of it.
- RivieraKid 13y ago
  Compared to Python or MATLAB? Yes. 2x slower than C is amazing for a high-level and dynamic language. But of course, if performance is critical, C/C++ may be the better choice.

reeses 13y ago

Where is this "use" part of which you speak? Aggressive JIT languages are historically slow at startup. The typical tradeoff (similar to the server/client Java VM flags) is to amortize the optimizations over the length of the application if startup time is a factor. If you start Julia (or Mathematica, Excel, or whatever) up every time you need to 'graphically explore data', you're doing it wrong. Unless you fubar your environment (not that I ever do this on a regular basis), there isn't a great reason to restart your exploratory environment.

rdtsc 13y ago
> Where is this "use" part of which you speak Maybe you don't want to use the old repl, there there is something else going on there and you don't want to mess it up. Maybe I want to show someone an example, or try something out quickly. I use and spawn ipython windows all the time. Sometimes I have 4 open at the same time. > Aggressive JIT languages are historically slow at startup. Explaining the reason for the slow start-up doesn't make the slow startups faster.
- deleted 13y ago
  [deleted]
- coldtea 13y ago
  >> Aggressive JIT languages are historically slow at startup. Explaining the reason for the slow start-up doesn't make the slow startups faster. No, but insisting that you want startups faster, when you've been explained the technical reasons why they are slow, doesn't make them faster either. To me it sounds like: "This ball shouldn't fall to earth". "It fell because of gravity". "Explaining the reason it fell, doesn't make it go upwards".
  - rdtsc 13y ago
    > No, but insisting that you want startups faster, when you've been explained the technical reasons why they are slow, doesn't make them faster either. Oh please. "We've explained the technical reasons to you" excuse. As a customer the response to that is "ok, good luck with that, I am not using your product". > "This ball shouldn't fall to earth" / "It fell because of gravity" / "Explaining the reason it fell, doesn't make it go upwards". No the answer to that is, screw this planet, I'll go fly into space and find another planet. Which is the response people have with software they don't like. It is more like: "Oh your database is slow" and getting the response of "Well of course it is,the locks at level 3 in our API have not been optimized because John is on vacation still. Geez how dare you not accept that as a valid answer?" See you've equated software features hard laws of physics. So escaping from it is "ridiculous". To be more serious, having an interpreter JIT a module only if you use, on-demand, or save the trace of it, across execution maybe he hard but doesn't require changing the fine structure constant.

Derbasti 13y ago

Note that one of Julia's main competitor, Matlab, is laughably bad at this, too. Of course Julia has a slow startup because it is JIT-compiled and therefore will run fast. This analogy does not extend to Matlab.

StefanKarpinski 13y ago

You literally picked the single package that loads the most other packages. Clearly this needs work, but just running vanilla julia without loading packages is not bad at all these days: $ time julia -e 'println("Hello, world.")' Hello, world. real 0m0.209s user 0m0.299s sys 0m0.104s Julia now pre-compiles the jitted code for it's base system, and the plan is to allow the same thing for packages. As others have noted, starting up Matlab is extremely slow and starting up a full R environment isn't all that fast either, nor is starting a JVM all that speedy. Startup time alone doesn't prevent a system from being useful.

evmar 13y ago
Thanks for replying! I didn't anticipate that my comment would generate so much controversy. :( I was only relating my own experience in trying out the tool, and comparing it to the tool I currently use frequently despite not liking its semantics very much: R. When I had encountered the slowness I was complaining about above, I poked around online and saw that you all were certainly aware of it and had plans to improve it. I think I even downloaded a 0.2 binary and then tried out trunk to see if it had already improved. All I had meant to say was (like Rust) that I anticipate I'll need to wait for a more complete version.
delian66 13y ago
>>Startup time alone doesn't prevent a system from being useful. Well, it does not completely ... If I run a simulation that will take 10 hours, 2 seconds is not much at all. However, it will severely reduce its usefullness in many other cases (quick&dirty scripting, testing)... Just for the record, on an i5 system: $ time julia --version julia version 0.3.0-prerelease real 0m2.156s user 0m2.119s sys 0m0.030s $time julia -e 'println("Hello, world.")' Hello, world. real 0m2.119s user 0m2.074s sys 0m0.036s In comparison, starting an equivalent ipython script takes <0.1s on the same machine.