4 ms·

I think it's disappointing that Julia is not meant for CLI scripting. IMO it has failed to live up to its manifesto (https://julialang.org/blog/2012/02/why-we-c

I think it's disappointing that Julia is not meant for CLI scripting. IMO it has failed to live up to its manifesto (https://julialang.org/blog/2012/02/why-we-created-julia https://julialang.org/blog/2012/02/why-we-created-julia) of being a general-purpose language that can "have it all". My own bioinformatics work involves parsing massive amounts of text data, and Julia is excellent for this -- it is very fast, yet high-level and easy to write. However, bash pipelines are also a huge part of bioinformatics, and Julia scripts are not very good for this due to the long startup time, which is a shame.

elcritch 7y ago
> I think it's disappointing that Julia is not meant for CLI scripting. IMO it has failed to live up to its manifesto (https://julialang.org/blog/2012/02/why-we-created-julia https://julialang.org/blog/2012/02/why-we-created-julia) of being a general-purpose language that can "have it all". I think it’s still early in the languages development to say that. There isn’t any fundamental reasons Julia couldn’t be tuned to be better at CLI scripting. Currently it’s what, 400ms to startup, which isn’t too terrible but could be made faster. I could see using the Julia debugger as an interpreter for CLI scripts. Or if you run a script a lot it’s possible to have Julia compile an executable. Personally I use it in mainly via notebooks or a repl.
- dnautics 7y ago
  400ms to startup? > time julia -e "exit()" real 0m0.164s user 0m0.106s sys 0m0.047s This is on a relatively old laptop: Intel(R) Core(TM) i5-5200U CPU @ 2.20GHz according to my lscpu
  - manwe150 7y ago
    I helped work on getting that number from ~400ms to ~150ms maybe 18 month ago for the v1.0 release. FWIW, a big part of it was shedding a couple excess C libraries with bad load times (usually now lazy loading them when required instead). The next big jump will take more internal effort, but I don’t think there’s any serious showstopper. The bigger short-term interest though has been towards reducing the latency of loading external (user) code/libraries. For an example of what I mean by the latter, python seems to be pretty fast initially, but then seems to take a huge hit from just trying to get numpy loaded. $ time python -c '0' real0m0.024s user0m0.016s sys0m0.008s $ time python -c 'import numpy' real0m0.165s user0m1.508s sys0m2.312s $ time ./julia -e 0 real0m0.215s user0m0.240s sys0m0.144s
    - elcritch 7y ago
      Excellent! I was mainly going off the numbers given by another comment. Well ~150ms seems within useable CLI times for me. One note, I’d reckon to have a fair comparison to the Numpy timings the Julia example would need to use an array type to do something.
    - owl57 7y ago
      I believe you compared first (uncached) startup of Python against second startup of Julia: $ time python -c '0' python -c '0' 0,03s user 0,01s system 7% cpu 0,467 total $ time python -c '0' python -c '0' 0,03s user 0,00s system 98% cpu 0,030 total $ time python -c 'import numpy' python -c 'import numpy' 0,17s user 0,05s system 9% cpu 2,401 total $ time python -c 'import numpy' python -c 'import numpy' 0,11s user 0,01s system 99% cpu 0,118 total $ time julia -e 0 julia -e 0 0,25s user 0,27s system 17% cpu 2,868 total $ time julia -e 0 julia -e 0 0,09s user 0,05s system 92% cpu 0,155 total The impact of actually calling something from numpy is also negligible in Python but not in Julia: $ time python -c 'import numpy; numpy.random.rand(10,10)' python -c 'import numpy; numpy.random.rand(10,10)' 0,10s user 0,01s system 99% cpu 0,116 total $ time julia -e 'rand(10,10)' julia -e 'rand(10,10)' 0,35s user 0,23s system 209% cpu 0,277 total $ time julia -e 'rand(10,10)' julia -e 'rand(10,10)' 0,36s user 0,22s system 209% cpu 0,278 total
      - manwe150 7y ago
        We got approximately the same numbers (your clock speed is likely to be much higher). User/system time is quasi-bogus, since it’s a high core count system (although still a bit concerning). I accounted for possible cache effect by running each a number of times and reporting the last. I’m not really trying to make an absolute time comparison here, just pointing out that if 100ms is unacceptable, numpy would just miss that bar too. Once you’re past the bar of “this needs to be kept running”, I don’t think a constant factor of 100ms vs 1s makes much difference in QoL, and now we’re just comparing apples and oranges. A constant factor gain on the rest of the time can make a huge difference on the rate of results per second. But I actually hope both will improve!
- mrtweetyhack 7y ago
  Julia was started in 2009 and launched in 2012. It's 2019. It would be easy for them change their manifesto which seems, at the moment, to be entirely disingenuous.
- snackematician 7y ago
  By "slow startup time" I also include the time to import libraries which is really the bigger problem. For example loading the DataFrames library takes quite a long time, and I've heard the libraries for plotting take even longer to import.
leiroigh 7y ago
Are startup times really your problem? Consider https://github.com/JuliaLang/julia/issues/30044 https://github.com/JuliaLang/julia/issues/30044 You can manually enable buffering of the pipe endpoints. Otherwise julia will be super-duper-slow when part of a pipe.

elcritch 7y ago

> I think it's disappointing that Julia is not meant for CLI scripting. IMO it has failed to live up to its manifesto (https://julialang.org/blog/2012/02/why-we-created-julia https://julialang.org/blog/2012/02/why-we-created-julia) of being a general-purpose language that can "have it all". I think it’s still early in the languages development to say that. There isn’t any fundamental reasons Julia couldn’t be tuned to be better at CLI scripting. Currently it’s what, 400ms to startup, which isn’t too terrible but could be made faster. I could see using the Julia debugger as an interpreter for CLI scripts. Or if you run a script a lot it’s possible to have Julia compile an executable. Personally I use it in mainly via notebooks or a repl.

dnautics 7y ago
400ms to startup? > time julia -e "exit()" real 0m0.164s user 0m0.106s sys 0m0.047s This is on a relatively old laptop: Intel(R) Core(TM) i5-5200U CPU @ 2.20GHz according to my lscpu
- manwe150 7y ago
  I helped work on getting that number from ~400ms to ~150ms maybe 18 month ago for the v1.0 release. FWIW, a big part of it was shedding a couple excess C libraries with bad load times (usually now lazy loading them when required instead). The next big jump will take more internal effort, but I don’t think there’s any serious showstopper. The bigger short-term interest though has been towards reducing the latency of loading external (user) code/libraries. For an example of what I mean by the latter, python seems to be pretty fast initially, but then seems to take a huge hit from just trying to get numpy loaded. $ time python -c '0' real0m0.024s user0m0.016s sys0m0.008s $ time python -c 'import numpy' real0m0.165s user0m1.508s sys0m2.312s $ time ./julia -e 0 real0m0.215s user0m0.240s sys0m0.144s
  - elcritch 7y ago
    Excellent! I was mainly going off the numbers given by another comment. Well ~150ms seems within useable CLI times for me. One note, I’d reckon to have a fair comparison to the Numpy timings the Julia example would need to use an array type to do something.
  - owl57 7y ago
    I believe you compared first (uncached) startup of Python against second startup of Julia: $ time python -c '0' python -c '0' 0,03s user 0,01s system 7% cpu 0,467 total $ time python -c '0' python -c '0' 0,03s user 0,00s system 98% cpu 0,030 total $ time python -c 'import numpy' python -c 'import numpy' 0,17s user 0,05s system 9% cpu 2,401 total $ time python -c 'import numpy' python -c 'import numpy' 0,11s user 0,01s system 99% cpu 0,118 total $ time julia -e 0 julia -e 0 0,25s user 0,27s system 17% cpu 2,868 total $ time julia -e 0 julia -e 0 0,09s user 0,05s system 92% cpu 0,155 total The impact of actually calling something from numpy is also negligible in Python but not in Julia: $ time python -c 'import numpy; numpy.random.rand(10,10)' python -c 'import numpy; numpy.random.rand(10,10)' 0,10s user 0,01s system 99% cpu 0,116 total $ time julia -e 'rand(10,10)' julia -e 'rand(10,10)' 0,35s user 0,23s system 209% cpu 0,277 total $ time julia -e 'rand(10,10)' julia -e 'rand(10,10)' 0,36s user 0,22s system 209% cpu 0,278 total
    - manwe150 7y ago
      We got approximately the same numbers (your clock speed is likely to be much higher). User/system time is quasi-bogus, since it’s a high core count system (although still a bit concerning). I accounted for possible cache effect by running each a number of times and reporting the last. I’m not really trying to make an absolute time comparison here, just pointing out that if 100ms is unacceptable, numpy would just miss that bar too. Once you’re past the bar of “this needs to be kept running”, I don’t think a constant factor of 100ms vs 1s makes much difference in QoL, and now we’re just comparing apples and oranges. A constant factor gain on the rest of the time can make a huge difference on the rate of results per second. But I actually hope both will improve!
mrtweetyhack 7y ago
Julia was started in 2009 and launched in 2012. It's 2019. It would be easy for them change their manifesto which seems, at the moment, to be entirely disingenuous.
snackematician 7y ago
By "slow startup time" I also include the time to import libraries which is really the bigger problem. For example loading the DataFrames library takes quite a long time, and I've heard the libraries for plotting take even longer to import.

leiroigh 7y ago

Are startup times really your problem? Consider https://github.com/JuliaLang/julia/issues/30044 https://github.com/JuliaLang/julia/issues/30044 You can manually enable buffering of the pipe endpoints. Otherwise julia will be super-duper-slow when part of a pipe.