8 ms·

Since everybody is so hyped about Julia, I took on learning it and using it in one of my pet projects. The use case seemed to be perfect for Julia: financial po

Since everybody is so hyped about Julia, I took on learning it and using it in one of my pet projects. The use case seemed to be perfect for Julia: financial portfolio optimization. But honestly, it is a royal pain in the ass to use. Mostly because tooling is so crappy (no decent IDE). For example, I've made these notes on my journey with Julia: - Juno and VSCode julia is a pain - Debugging is pain - no watch expressions (but debug> works) - Works like with matlab (workspaces, designed for the single script use-cases) - Long registry lookup times (2-5mins) when starting a new session - No hints from IDE (Juno) on what methods are available for which types - Ctrl-click navigation does not work to lookup definitations in libraries - Debugger does not show the full list of elements - there are triple dots in the middle - Filters and maps are not lazy - Arrays start at one - Precompilation times can take 1-2min - Package management is done as a part of script execution - Delay (0.5-1s) when processing commands (compilation?) - Editors (Juno) do not check types at runtime when the information is readily available - Include() includes (is this PHP all over again?) a file into the script! - Reuses same REPL between different files - DataFrames.jl: Transform does nonsense (use map instead) - DataFrames.jl: No transpose Obviously, there is a bunch of good things like freakin insane speed and vectorized method calls are awesome, but I struggle seeing myself becoming more producting than Python + Pandas. It's really a shame that language is strongly and semi-statically typed yet no tooling is making a use of that.

jackcviers3 6y ago
To be fair, most of these are ide complaints, not language complaints. Much like all development work, language designers are now required to know much more than how to write an effective static compiler, an effective language syntax, and an effective standard library. A working IDE that integrates easily with all mainstream editors, extensions for all mainstream editors, cross-platform compilation for all major environments, a secure and effective dependency manager, a secure and effective build tool, and incremental compilation are all requirements for a language to get off the ground. Honestly, that seems like a big hurdle for one person or even a dedicated team to manage effectively without major funding outside of what is normally present in research grants. Hosting and curation alone for the dependency package system is a costly problem. Are we as developers painting ourselves into a corner of stagnating pl technology with our expectations that all these things exist in order for us to call a language "good?"
- tasubotadas 6y ago
  Yes. But only computer scientists are interested only in the language itself. I am an engineer so I have to deal with practical aspects of it.
  - jackcviers3 6y ago
    I'm an engineer. I primarily care about how good the language is at expressing and executing my intent. I've been around long enough to see several tooling fads come and go. I've also been around enough to know that the barrier for delivering quality software is becoming insurmountable, and the additional tooling and infrastructure isn't resulting in projects being more successful, nor being delivered more quickly, nor being less costly than when I started. Teams are enormous now. Software projects are enormous. Deployment requires deep knowledge of multiple hosting platforms and various cloud provider apis at the enterprise level. Available libraries, increasing reliance upon testing, type safety, and better development life cycle practices have had a far larger impact than the ide's I've seen come and go. In my experience, a particular tool has a popularity lifecycle of about ten years. Vscode and lsp will probably be the same. Give me types, good compiler error output, and access to a large universe of libraries with good documentation over relying upon ide api discovery any day of the week and twice on Sundays. The rest is an ever-changing tide of requirements fashion.
    - kuzuman 6y ago
      "I am an engineer" Did you pass the Fundamentals of Engineering exam, are you a member in order of your local association of engineers?, do you have an engineer seal? If you answer no to the questions above please don't call yourself engineer
  - KenoFischer 6y ago
    Give us some more time, there's just so many things to do and the reliance issues favor getting the fundamentals right first (much easier to switch the debugger UI than the names of functions that'll get put all over the place). We certainly understand the value of good tooling, so just stay tuned and maybe try it again every once in a while to see if it's good enough for you yet. Too much to do, too little time :)
bieganek 6y ago
> Long registry lookup times (2-5mins) when starting a new session I'm not sure what you are referring to here. Yes, sometimes after you've installed new packages, the Julia VS Code Language Server has to do some re-indexing which can take some time, but they've improved this and you can still program and use the integrated REPL while this is occurring. > Ctrl-click navigation does not work to lookup definitations in libraries In VS Code, F12 (Go to Definition) does work. You can also use `@edit foo(x)` in the REPL. > Filters and maps are not lazy Use Iterators.filter or use generator expressions. > Package management is done as a part of script execution I'm also not sure what this refers to, but the package manager in Julia is one of the best things about Julia. > Editors (Juno) do not check types at runtime when the information is readily available The VS Code Julia extension has a linter that is pretty helpful. It's not perfect, but they're actively working on developing it. > DataFrames.jl: Transform does nonsense (use map instead) The new select and transform functions in DataFrames.jl are actually quite powerful and useful. > DataFrames.jl: No transpose Transpose is not a generic concept for a table. Sure, it might make sense in specific cases, but in general it doesn't make sense to transpose a table.
- setr 6y ago
  >Sure, it might make sense in specific cases, but in general it doesn't make sense to transpose a table. From my intuition, I don't see anything stopping the function from existing -- it can be applied to any arbitrary table. You probably don't want to transpose your table, except when you want to, but that's true of any function -- I'm can't imagine any scenario where transpose(table)->table would as an algorithm fail (unless I suppose if julia tables include header rows, in which case there's probably no generally correct definition)
  - dragonwriter 6y ago
    > From my intuition, I don't see anything stopping the function from existing A datatable is (or is isomorphic to and can be analyzed as) a mapping from row numbers to tuples of a given shape[0]. The transpose of table will only be table (a mapping from rows numbers to tuples of a common shape) if the tuples of the starting table were homogenous (every field of the same type.) This works with, say, matrices where all the elements are numbers; but it fails in the general case. [0] Yes, I know, Julia defines them as columns of arrays, and columnar organization is ideal for all kinds of processing tasks. For me, thinking about and explaining the the problem with transpose works easier thinking about it in row-oriented form (which is logically equivalent). In column-oriented description, a datatable is an ordered set of columns, each of which is a homogenous array, but if you try to transpose it, each of the columns of the result would be a heterogenous array unless the columns were all of the same type to start with. So, again, it fails to be a table->table function except in the case where the starting table consists of columns of identical type.
ogogmad 6y ago
- Arrays start at one Why is this such a big deal? You just change the range of your for-loops from 0<=i<n to 1<=i<=n. Are you using cyclic buffers?
- adolgert 6y ago
  Dijkstra on the matter: https://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/EWD831.html https://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/E...
  - ogogmad 6y ago
    The points he makes are so extremely minor... You could just as easily say that natural language and mathematical convention are index-1.
    - e12e 6y ago
      I never understood the fascination with preferring "the element at zero offset from the start" to "the first element" - and I don't think djiskarta makes a compelling argument.
  - goto11 6y ago
    Dijkstra has a certain tone. He can make his opinions sound almost like a mathematical proof. But when you dig into it, he just says 0-based is "nicer".
- Athas 6y ago
  It matters when doing multidimensional index arithmetic. The formulae for ranking and unranking multi-dimensional to flat indexes are neater with 0-indexing than with 1-indexing.
  - ogogmad 6y ago
    Fair enough. But is this a common enough occurrence in the life of a programmer that it's worth being a religious extremist about?
JHonaker 6y ago
> Editors (Juno) do not check types at runtime when the information is readily available This complaint is really common with people that see types and expect them to do something like TypeScript. Julia is not a statically typed language. The types are not there for a type checker to check your code for correctness. Instead the types allow Julia to dispatch your method call to the correct implementation for the type of your variable at runtime. This is Julia's secret sauce, and it's the main reason you'll hear about Julia programmers declaring about how composable Julia packages are. I can "reach into your package", and define the behaviour of your functions on my own custom types and then whenever anyone tries to call the function/method with one of my types, it'll just work. I don't need to author a pull request to your package or futz around with anything that you've wrote. Novice Julia programmers often come in thinking that the type annotations are there for ensuring correctness, but that's not it at all. In actuality, you want to be as general with your types as you can, and by default most parameters will probably be untyped. If you need certain behaviour for the function, then you should probably annotate it, but it's definitely not required. I'm guilty of this misstep as well, one of the biggest things I struggled with when I was new was using ::Array everywhere, when what I wanted was ::AbstractArray. > Include() includes (is this PHP all over again?) a file into the script! Use Modules. Don't fault Julia for you being a beginner and not reading the necessary parts of the documentation.
- systemvoltage 6y ago
  > Instead the types allow Julia to dispatch your method call to the correct implementation for the type of your variable at runtime. This is Julia's secret sauce, and it's the main reason you'll hear about Julia programmers declaring about how composable Julia packages are. I can "reach into your package", and define the behaviour of your functions on my own custom types and then whenever anyone tries to call the function/method with one of my types, it'll just work. I don't need to author a pull request to your package or futz around with anything that you've wrote. That's not a bug, that's a feature. Libraries are abstractions. I get an API and it encapsulates the behavior of the library. That is a nice thing. Being able to modify runtime behavior of internal libraries can lead to insane jumblygoo of code spathetti that would bring the finest programmers to their knees. I don't want implicit behavior at runtime. I want explicit behavior in case of non-contractual externalities (wrong user input for e.g. at runtime). I want the program to fail so it can be patched.
nalimilan 6y ago
> - DataFrames.jl: Transform does nonsense (use map instead) What is this supposed to mean? > - DataFrames.jl: No transpose This is actually going to be fixed in the next few days (https://github.com/JuliaData/DataFrames.jl/pull/2447 https://github.com/JuliaData/DataFrames.jl/pull/2447). Given the limited amount of work it required, it sounds quite exaggerated to mention it as a major limitation of the language.
huijzer 6y ago
These problems mostly seem to come from trying to use Julia in some kind of visual studio paradigm. The key is to leave the REPL running, I find. Then, with Revise.jl the speed is insane because nothing has to start up; only the changes you make will be recompiled during execution. Also take a look at Pluto.jl. Edit: By the Visual Studio paradigm, I mean a paradigm where all the stuff is made around visual interactions with the IDE. For example, showing autocomplete which works because the programmer is very constrained in OOP (foo.b completes to foo.bar) and statically typed languages with Java and C# as prime examples. I think these constraints are made around the idea that programmers are dumb and that it mostly is a distraction from how one actually wants to think about the problem at hand.
clircle 6y ago
Arrays start at 1!? What Joy! Maybe I can try this language!
kortex 6y ago
Edit: just learned about OffsetArrays.jl - ok now that's really cool. Keeping comment for context. > - Arrays start at one Oof. Big turnoff for me. For some reason, this particular context switch really grinds my gears. All languages I use (except bash, which grinds my gears) use Dijkstra-style array slices: python, go, c/++, js. I'm sure it makes it easier for Matlab converts. This, plus the mentioned weaknesses with libs like http, grpc et al, IMHO will relegate Julia to many years of just being wrapped by other languages. To avoid that, I think they should be thinking about continuing to woo more of the engineering crowd - which to their credit I think they've done pretty well so far. https://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/EWD831.html https://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/E... https://github.com/JuliaArrays/OffsetArrays.jl https://github.com/JuliaArrays/OffsetArrays.jl
- FranzFerdiNaN 6y ago
  They’re making the language mostly for statisticians, mathematicians and scientific programmers, where it makes sense to have arrays start at 1. Why should they cater to whiny engineers who can’t deal with anything they aren’t used to?

jackcviers3 6y ago

To be fair, most of these are ide complaints, not language complaints. Much like all development work, language designers are now required to know much more than how to write an effective static compiler, an effective language syntax, and an effective standard library. A working IDE that integrates easily with all mainstream editors, extensions for all mainstream editors, cross-platform compilation for all major environments, a secure and effective dependency manager, a secure and effective build tool, and incremental compilation are all requirements for a language to get off the ground. Honestly, that seems like a big hurdle for one person or even a dedicated team to manage effectively without major funding outside of what is normally present in research grants. Hosting and curation alone for the dependency package system is a costly problem. Are we as developers painting ourselves into a corner of stagnating pl technology with our expectations that all these things exist in order for us to call a language "good?"

tasubotadas 6y ago
Yes. But only computer scientists are interested only in the language itself. I am an engineer so I have to deal with practical aspects of it.
- jackcviers3 6y ago
  I'm an engineer. I primarily care about how good the language is at expressing and executing my intent. I've been around long enough to see several tooling fads come and go. I've also been around enough to know that the barrier for delivering quality software is becoming insurmountable, and the additional tooling and infrastructure isn't resulting in projects being more successful, nor being delivered more quickly, nor being less costly than when I started. Teams are enormous now. Software projects are enormous. Deployment requires deep knowledge of multiple hosting platforms and various cloud provider apis at the enterprise level. Available libraries, increasing reliance upon testing, type safety, and better development life cycle practices have had a far larger impact than the ide's I've seen come and go. In my experience, a particular tool has a popularity lifecycle of about ten years. Vscode and lsp will probably be the same. Give me types, good compiler error output, and access to a large universe of libraries with good documentation over relying upon ide api discovery any day of the week and twice on Sundays. The rest is an ever-changing tide of requirements fashion.
  - kuzuman 6y ago
    "I am an engineer" Did you pass the Fundamentals of Engineering exam, are you a member in order of your local association of engineers?, do you have an engineer seal? If you answer no to the questions above please don't call yourself engineer
- KenoFischer 6y ago
  Give us some more time, there's just so many things to do and the reliance issues favor getting the fundamentals right first (much easier to switch the debugger UI than the names of functions that'll get put all over the place). We certainly understand the value of good tooling, so just stay tuned and maybe try it again every once in a while to see if it's good enough for you yet. Too much to do, too little time :)

bieganek 6y ago

> Long registry lookup times (2-5mins) when starting a new session I'm not sure what you are referring to here. Yes, sometimes after you've installed new packages, the Julia VS Code Language Server has to do some re-indexing which can take some time, but they've improved this and you can still program and use the integrated REPL while this is occurring. > Ctrl-click navigation does not work to lookup definitations in libraries In VS Code, F12 (Go to Definition) does work. You can also use `@edit foo(x)` in the REPL. > Filters and maps are not lazy Use Iterators.filter or use generator expressions. > Package management is done as a part of script execution I'm also not sure what this refers to, but the package manager in Julia is one of the best things about Julia. > Editors (Juno) do not check types at runtime when the information is readily available The VS Code Julia extension has a linter that is pretty helpful. It's not perfect, but they're actively working on developing it. > DataFrames.jl: Transform does nonsense (use map instead) The new select and transform functions in DataFrames.jl are actually quite powerful and useful. > DataFrames.jl: No transpose Transpose is not a generic concept for a table. Sure, it might make sense in specific cases, but in general it doesn't make sense to transpose a table.

setr 6y ago
>Sure, it might make sense in specific cases, but in general it doesn't make sense to transpose a table. From my intuition, I don't see anything stopping the function from existing -- it can be applied to any arbitrary table. You probably don't want to transpose your table, except when you want to, but that's true of any function -- I'm can't imagine any scenario where transpose(table)->table would as an algorithm fail (unless I suppose if julia tables include header rows, in which case there's probably no generally correct definition)
- dragonwriter 6y ago
  > From my intuition, I don't see anything stopping the function from existing A datatable is (or is isomorphic to and can be analyzed as) a mapping from row numbers to tuples of a given shape[0]. The transpose of table will only be table (a mapping from rows numbers to tuples of a common shape) if the tuples of the starting table were homogenous (every field of the same type.) This works with, say, matrices where all the elements are numbers; but it fails in the general case. [0] Yes, I know, Julia defines them as columns of arrays, and columnar organization is ideal for all kinds of processing tasks. For me, thinking about and explaining the the problem with transpose works easier thinking about it in row-oriented form (which is logically equivalent). In column-oriented description, a datatable is an ordered set of columns, each of which is a homogenous array, but if you try to transpose it, each of the columns of the result would be a heterogenous array unless the columns were all of the same type to start with. So, again, it fails to be a table->table function except in the case where the starting table consists of columns of identical type.

ogogmad 6y ago

- Arrays start at one Why is this such a big deal? You just change the range of your for-loops from 0<=i<n to 1<=i<=n. Are you using cyclic buffers?

adolgert 6y ago
Dijkstra on the matter: https://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/EWD831.html https://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/E...
- ogogmad 6y ago
  The points he makes are so extremely minor... You could just as easily say that natural language and mathematical convention are index-1.
  - e12e 6y ago
    I never understood the fascination with preferring "the element at zero offset from the start" to "the first element" - and I don't think djiskarta makes a compelling argument.
- goto11 6y ago
  Dijkstra has a certain tone. He can make his opinions sound almost like a mathematical proof. But when you dig into it, he just says 0-based is "nicer".
Athas 6y ago
It matters when doing multidimensional index arithmetic. The formulae for ranking and unranking multi-dimensional to flat indexes are neater with 0-indexing than with 1-indexing.
- ogogmad 6y ago
  Fair enough. But is this a common enough occurrence in the life of a programmer that it's worth being a religious extremist about?

JHonaker 6y ago

> Editors (Juno) do not check types at runtime when the information is readily available This complaint is really common with people that see types and expect them to do something like TypeScript. Julia is not a statically typed language. The types are not there for a type checker to check your code for correctness. Instead the types allow Julia to dispatch your method call to the correct implementation for the type of your variable at runtime. This is Julia's secret sauce, and it's the main reason you'll hear about Julia programmers declaring about how composable Julia packages are. I can "reach into your package", and define the behaviour of your functions on my own custom types and then whenever anyone tries to call the function/method with one of my types, it'll just work. I don't need to author a pull request to your package or futz around with anything that you've wrote. Novice Julia programmers often come in thinking that the type annotations are there for ensuring correctness, but that's not it at all. In actuality, you want to be as general with your types as you can, and by default most parameters will probably be untyped. If you need certain behaviour for the function, then you should probably annotate it, but it's definitely not required. I'm guilty of this misstep as well, one of the biggest things I struggled with when I was new was using ::Array everywhere, when what I wanted was ::AbstractArray. > Include() includes (is this PHP all over again?) a file into the script! Use Modules. Don't fault Julia for you being a beginner and not reading the necessary parts of the documentation.

systemvoltage 6y ago
> Instead the types allow Julia to dispatch your method call to the correct implementation for the type of your variable at runtime. This is Julia's secret sauce, and it's the main reason you'll hear about Julia programmers declaring about how composable Julia packages are. I can "reach into your package", and define the behaviour of your functions on my own custom types and then whenever anyone tries to call the function/method with one of my types, it'll just work. I don't need to author a pull request to your package or futz around with anything that you've wrote. That's not a bug, that's a feature. Libraries are abstractions. I get an API and it encapsulates the behavior of the library. That is a nice thing. Being able to modify runtime behavior of internal libraries can lead to insane jumblygoo of code spathetti that would bring the finest programmers to their knees. I don't want implicit behavior at runtime. I want explicit behavior in case of non-contractual externalities (wrong user input for e.g. at runtime). I want the program to fail so it can be patched.

nalimilan 6y ago

> - DataFrames.jl: Transform does nonsense (use map instead) What is this supposed to mean? > - DataFrames.jl: No transpose This is actually going to be fixed in the next few days (https://github.com/JuliaData/DataFrames.jl/pull/2447 https://github.com/JuliaData/DataFrames.jl/pull/2447). Given the limited amount of work it required, it sounds quite exaggerated to mention it as a major limitation of the language.

huijzer 6y ago

These problems mostly seem to come from trying to use Julia in some kind of visual studio paradigm. The key is to leave the REPL running, I find. Then, with Revise.jl the speed is insane because nothing has to start up; only the changes you make will be recompiled during execution. Also take a look at Pluto.jl. Edit: By the Visual Studio paradigm, I mean a paradigm where all the stuff is made around visual interactions with the IDE. For example, showing autocomplete which works because the programmer is very constrained in OOP (foo.b completes to foo.bar) and statically typed languages with Java and C# as prime examples. I think these constraints are made around the idea that programmers are dumb and that it mostly is a distraction from how one actually wants to think about the problem at hand.

clircle 6y ago

Arrays start at 1!? What Joy! Maybe I can try this language!

kortex 6y ago

Edit: just learned about OffsetArrays.jl - ok now that's really cool. Keeping comment for context. > - Arrays start at one Oof. Big turnoff for me. For some reason, this particular context switch really grinds my gears. All languages I use (except bash, which grinds my gears) use Dijkstra-style array slices: python, go, c/++, js. I'm sure it makes it easier for Matlab converts. This, plus the mentioned weaknesses with libs like http, grpc et al, IMHO will relegate Julia to many years of just being wrapped by other languages. To avoid that, I think they should be thinking about continuing to woo more of the engineering crowd - which to their credit I think they've done pretty well so far. https://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/EWD831.html https://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/E... https://github.com/JuliaArrays/OffsetArrays.jl https://github.com/JuliaArrays/OffsetArrays.jl

FranzFerdiNaN 6y ago
They’re making the language mostly for statisticians, mathematicians and scientific programmers, where it makes sense to have arrays start at 1. Why should they cater to whiny engineers who can’t deal with anything they aren’t used to?