5 ms·

> I think it requires a jvm mindset, warmup once and iterate inside rather than outside. I do not have this mindset then. I prefer tools who are mindset obliv

> I think it requires a jvm mindset, warmup once and iterate inside rather than outside. I do not have this mindset then. I prefer tools who are mindset oblivious. They are really useful! For example, imagine I have a collection of a few hundred images with their projection matrices (in text files). I want to crop them and apply a simple imagemagick operation (which is not available from inside julia). The elementary solution is to run a shell loop to apply the crop, and call julia to perform a simple adaptation of each projection matrix. This is impossible today: most of the running time of such loop is spent on julia initialization. Half a second to do nothing is simply unacceptable in a serious scripting language.

pkofod 8y ago
Why can you not call you imagemagick operation from julia? Can't you just `run(``)` it?
- enriquto 8y ago
  This is a part of a quite large system that does a lot of things. I do not want to wrap everything inside julia, that would force me to rewrite everything in julia. I just need to invert a few 4x4 linear systems from inside a shell script. I agree that this is not a very representative use case, and maybe it is irrelevant for most people. Yet, if I try to evangelize julia in my lab this problem blocks it, and it is better to stick to octave.
agumonkey 8y ago
I guessed that it wasn't your habits but if you step outside that box a bit you'll realize that lifting up or down things inside loops is the most natural thing to do. And for you example (which may not be your real use or workflow) you could have two loops, independent or coupled through a queue so that the julia process is only started once.
- enriquto 8y ago
  Of course! Once I realize that after a few seconds only a dozen images have been processed I cut the loop, remember that the julia repl is dog slow, and then, rewrite the task in a different way. But I would prefer not to have to do that. Moreover, for more complicated examples, there may be data dependencies that make the loop commutation non trivial. The slow startup time may be a minor inconvenience, I agree. But nonetheless it seems to be a case of sloppy engineering, as other scripting languages do not have this egregious problem. It sets a bad tone, and makes me wonder if there are other hidden monsters inside the interpreter, that may be solved by "you are holding it wrong" like this one.
  - xiphias2 8y ago
    You have to use it inside a REPL, like R or Matlab. Julia using LLVM and compiling on-the-fly is not sloppy design, it's a design tradeoff (and a great one, if you really need speed). Julia is not a scripting language, it's a language for mathematical analysis. Also going back to your example the solution would be to use ImageMagick.jl from Julia, and extend the library (4 extra lines with ccall) if it's not available, and send a pull request to help the community.
    - enriquto 8y ago
      > Julia is not a scripting language, it's a language for mathematical analysis. Ok, this clarifies the matter a lot. So julia is not intended to be a general-purpose programming language. Are you involved in julia development? (Besides, I strongly dislike the design tradeoff of not being a good unix citizen.)
      - agumonkey 8y ago
        Not Scripting != Not general purpose
      - ddragon 8y ago
        Julia is a general-purpose programming language. The language itself is actually really good as a scripting language, including integrating well with shell, C, Fortran, R and Python, for example: https://docs.julialang.org/en/latest/manual/running-external-programs/#Pipelines-1 https://docs.julialang.org/en/latest/manual/running-external... The only trade-off here is the restricted resources the Julia devs can afford. Focusing on the scientific computation niche, the Julia compiler was built to compile code just ahead of time, since this way it can create code as fast as compiled languages while still being dynamic like any interpreted language (such as Matlab and Python). But there is nothing in the language that prevents it from being fully interpreted (and therefore having minimal compile time overhead, but generating poorly optimized code for long running processes), or alternatively pre-compiling and caching to have both (which is being worked on: https://github.com/JuliaLang/PackageCompiler.jl https://github.com/JuliaLang/PackageCompiler.jl )
coldtea 8y ago
>I do not have this mindset then. I prefer tools who are mindset oblivious. They are really useful! Yet you picked a specialized tool "imagemagick" (made for working with images), and want to drive it by another specialized tool (the shell, made for file and process management). Since you want "mindset oblivious" tools why not write everything from scratch, in C?
- enriquto 8y ago
  > Since you want "mindset oblivious" tools why not write everything from scratch, in C? Because imagemagick, julia and other specialized programs already provide the advanced algorithms that I need. Unfortunately, julia is not a good team player.
  - krull10 8y ago
    The Julia tooling is not currently designed for repeatedly creating new REPL/interpreter instances to quickly solve one small problem. What it excels at is rapidly solving the same problem many times within one REPL/interpreter (since after the first time the code will be compiled), or for longer computations where the overhead of the initial compilation does not matter. It would be great if it could also get fast at the use case you are interested in, but this isn’t currently what it excels at.
  - coldtea 8y ago
    >Unfortunately, julia is not a good team player. Or, you know, Julia is not specialized for your use case.

pkofod 8y ago

Why can you not call you imagemagick operation from julia? Can't you just `run(``)` it?

enriquto 8y ago
This is a part of a quite large system that does a lot of things. I do not want to wrap everything inside julia, that would force me to rewrite everything in julia. I just need to invert a few 4x4 linear systems from inside a shell script. I agree that this is not a very representative use case, and maybe it is irrelevant for most people. Yet, if I try to evangelize julia in my lab this problem blocks it, and it is better to stick to octave.

agumonkey 8y ago

I guessed that it wasn't your habits but if you step outside that box a bit you'll realize that lifting up or down things inside loops is the most natural thing to do. And for you example (which may not be your real use or workflow) you could have two loops, independent or coupled through a queue so that the julia process is only started once.

enriquto 8y ago
Of course! Once I realize that after a few seconds only a dozen images have been processed I cut the loop, remember that the julia repl is dog slow, and then, rewrite the task in a different way. But I would prefer not to have to do that. Moreover, for more complicated examples, there may be data dependencies that make the loop commutation non trivial. The slow startup time may be a minor inconvenience, I agree. But nonetheless it seems to be a case of sloppy engineering, as other scripting languages do not have this egregious problem. It sets a bad tone, and makes me wonder if there are other hidden monsters inside the interpreter, that may be solved by "you are holding it wrong" like this one.
- xiphias2 8y ago
  You have to use it inside a REPL, like R or Matlab. Julia using LLVM and compiling on-the-fly is not sloppy design, it's a design tradeoff (and a great one, if you really need speed). Julia is not a scripting language, it's a language for mathematical analysis. Also going back to your example the solution would be to use ImageMagick.jl from Julia, and extend the library (4 extra lines with ccall) if it's not available, and send a pull request to help the community.
  - enriquto 8y ago
    > Julia is not a scripting language, it's a language for mathematical analysis. Ok, this clarifies the matter a lot. So julia is not intended to be a general-purpose programming language. Are you involved in julia development? (Besides, I strongly dislike the design tradeoff of not being a good unix citizen.)
    - agumonkey 8y ago
      Not Scripting != Not general purpose
    - ddragon 8y ago
      Julia is a general-purpose programming language. The language itself is actually really good as a scripting language, including integrating well with shell, C, Fortran, R and Python, for example: https://docs.julialang.org/en/latest/manual/running-external-programs/#Pipelines-1 https://docs.julialang.org/en/latest/manual/running-external... The only trade-off here is the restricted resources the Julia devs can afford. Focusing on the scientific computation niche, the Julia compiler was built to compile code just ahead of time, since this way it can create code as fast as compiled languages while still being dynamic like any interpreted language (such as Matlab and Python). But there is nothing in the language that prevents it from being fully interpreted (and therefore having minimal compile time overhead, but generating poorly optimized code for long running processes), or alternatively pre-compiling and caching to have both (which is being worked on: https://github.com/JuliaLang/PackageCompiler.jl https://github.com/JuliaLang/PackageCompiler.jl )

coldtea 8y ago

>I do not have this mindset then. I prefer tools who are mindset oblivious. They are really useful! Yet you picked a specialized tool "imagemagick" (made for working with images), and want to drive it by another specialized tool (the shell, made for file and process management). Since you want "mindset oblivious" tools why not write everything from scratch, in C?

enriquto 8y ago
> Since you want "mindset oblivious" tools why not write everything from scratch, in C? Because imagemagick, julia and other specialized programs already provide the advanced algorithms that I need. Unfortunately, julia is not a good team player.
- krull10 8y ago
  The Julia tooling is not currently designed for repeatedly creating new REPL/interpreter instances to quickly solve one small problem. What it excels at is rapidly solving the same problem many times within one REPL/interpreter (since after the first time the code will be compiled), or for longer computations where the overhead of the initial compilation does not matter. It would be great if it could also get fast at the use case you are interested in, but this isn’t currently what it excels at.
- coldtea 8y ago
  >Unfortunately, julia is not a good team player. Or, you know, Julia is not specialized for your use case.