9 ms·

I don't have a link to the project unfortunately, it was really haphazardly put together (personal access tokens, other people's identifying information etc, ha

I don't have a link to the project unfortunately, it was really haphazardly put together (personal access tokens, other people's identifying information etc, have been committed to the repo), and I don't have the time to clean it up. But everything I used in the project is publicly available, and you'll just have to spend some time searching for it in various docs. I completely agree with Jakob's take in the piece you've shared btw. Personally, I don't think Jakob goes far enough. I've used Julia professionally too, and worked with large code bases (a 10000 line Julia monorepo project that just keeps growing), and the lack of interfaces, the lack static type checking, the lack of error types, etc would all be show stoppers for me to consider Julia at the start of our project if I knew then what I know now. I've even tried to run static type checkers like JET.jl on our code base and it just keeps spinning or failing. Unfortunately it is a sunk cost for us at this point and we'll just have to use Julia, and deal with these pain points, and hope and pray the language gets better over time. I can see some improvements such as minor latency improvements happening over time. But (in my opinion) I don't think language features like interfaces are likely to happen until Julia 2.0. In 2018, the core team had laid out priorities in a discourse post, and marked those as done last year (i.e. 2021). https://discourse.julialang.org/t/compiler-work-priorities/17623/122 https://discourse.julialang.org/t/compiler-work-priorities/1... In my opinion, there's a LOT more language features required to help tooling in the ecosystem improve. The fact that they marked this as done (which I just found out while typing this comment) and that there's no follow up priorities outline post (as far as I can find) is a little troubling. I think a lot of people that pick Julia as their first language really like it, but I believe it is mainly because they haven't experienced something better to compare it to. I'm not saying everyone is this way, and obviously everyone values different things in their programming language / development environment. I do think if you don't know what's out there you tend to just accept that something you are doing is the norm. And I think a lot of people that come to Julia from languages like Python, R, C++ see value in Julia for solving package development issues (compared to python), better language design (compared to R), performance without verbosity (compared to C++). And I think they weight those issues more heavily than I do. I've dealt with Python packaging issues, and still do on a daily basis but I now know my way around whatever comes up. R's syntax might be kludgy but I'm damn productive if I'm using ggplot. I'd rather use Python for general purpose work and R for plotting, than having to deal with Julia. C++ though I'd never want to touch again, and would just use rust instead. If rust wasn't an option, I think I'd end up using Julia (even though I complain about it so much). This is just were my priorities / skill level lies, and I can completely see how it would be different for others. I'm much more productive plotting in Python and R than I am in Julia, purely because of compile times, even though Julia has a much better designed plotting package than Python (R's plotting ecosystem is rather convenient even if it is counter intuitive some times). When you've used Rust, Go, Nim etc, developing in Julia feels so ... antiquated. Just as one example, when using Rust, you can write a unit test function right next to the function you are testing. And you can run just that unit test function from a terminal (e.g. `cargo test -- test_just_this_one_function`). And because you can write a quick little test function pretty much anywhere in your code and run it in the terminal, you don't really feel the need for a REPL all that much. When I'm not writing tests, I would say I'm as productive in Rust as I am in Julia (mainly because of awesome language server features in Rust). But the moment I need to write tests in Julia ... it's a big oof y'all. In Julia there's NO official way to run just one test. You HAVE to run all of them. And the tests can be scattered in various different tests files in the test folder. Try contributing to the core Julia language itself, and trying running their test suite. You'll have to go get coffee multiple times just to make one single patch. For projects that I'm working on, I literally have a Julia REPL open with Revise, where I manually include a specific test file after commenting out unnecessarily global state, and then call a the function I want to test that way. It's madness, and I _think_ this is what everyone does (?). If people are saying they prefer this, I can't help but liken it to Stockholm syndrome or something to that effect.

fault1 5y ago
I agree with almost all of what you said. Except personally, I'm much more productive in something like Julia (or R or Python) than a language like Rust for the type of development I usually want to do. I mostly write Python and C++ for my dayjob, but would have a hard time justifying Julia for production at the moment, though I love the language. Even more than Rust or Go, I think Julia could take a page from Typescript in terms of tooling because JS is so dynamic and the tooling is so nice. I do think compiler latency has gotten way better in more recent versions of Julia, but I think better ahead of time compilation, faster interpretation, and code caching are all important. I think Julia's tooling used to be better in Atom (Juno), however, Atom is dead.
- blindseer 5y ago
  I agree with you. Julia has gotten better but it’s still way slower than Python or R. And the workflow of running tests just make it painful. I think I’m familiar enough with Julia and I can write code and it’ll likely just work, but I’m not a 100% confident. With rust (and Python or Typescript with type hinting), I’m much MUCH more confident.
amkkma 5y ago
> there's a LOT more language features required to help tooling in the ecosystem improve. Aside from interfaces, what would those be?
- blindseer 5y ago
  I’ll first say that this is my opinion, and that I don’t have good suggestions for how Julia as a language should implement this, just that it should: * Better language level support for testing * Better language level support for docstrings - currently doc strings are just normal strings stored in a global struct, and are missing crucial information about the type signature of structs or functions or methods that are being documented * A way to type hint without using multiple dispatch / interfaces - I want to be able to say this argument has fields x, y, z but I don’t want to use multiple dispatch to enforce this interface. Multiple dispatch is useful with the implementation details of your code depend on the data structure. But I want better static type checking for a handful of fields or methods. Typescript is absolutely awesome at this. * Support for using @copied s.x = 1 where s is an immutable struct. Users are literally incentivized to use mutable structs instead of immutable structs because that results in shorter cleaner syntax, even though performance drops through the roof in that case. With immutable structs you’ll have to recreate the struct by calling the constructor. In most cases, the compiler will compile as this away anyway but users are going to prefer shorter cleaner syntax when writing code. * Better macro support. Currently macros have to be valid Julia code, meaning you end up with weird dsl that only kind of sort of matches the domain being modeled. In rust and nim I can literally copy almost any syntax and paste it in a macro block and it’ll just work. This is more work for macro developers and probably will make inference harder but will improve readability. * Language level support for extending Array or letting users define arrays that are as efficient as the built in ones. * Similar request for String buffers. * Rewrite parser that is currently in scheme in Julia, I suspect better tooling will fall out of this * Support for a subset of Julia that is fast to compile and run and produces small binaries. I think more generally I want Julia to be a slightly different language than it is ( or possibly ever can be ). Currently a user of Julia needs to be cognizant of too many things that could accidentally introduce type instability. Almost none of these problems are going to be encountered by someone that understands how compilers work / how computers work, which in my experience is not a lot of domain scientists but is a lot of the core team working on Julia. Domain scientists have PhDs or Masters in highly niche subjects after years of writing Python, R, or Matlab but are more often than not terrible software engineers. I can’t tell how many times in almost a decade of being in this field have I seen 10,000 line scripts in a single file without any comments written by extremely smart scientists. Guiding these folk from the get go to make better software engineering decisions is important imo. Currently, if I’m leading a project, I can’t trust a fresh graduate or junior engineer to write idiomatic type stable fast and extensible Julia code, even if they are the experts of their domain. They have to be experts of their domain AND awesome software engineers which is so rare to find. Given additional constraints about funding, writing proposals, documentation etc, it’s honestly kind of amazing that Julia’s libraries are as good as they are. In my opinion, Julia really shines when one or two people have to write code to tackle a specific problem. Code is super concise and clean and it all fits in your head and everything is fine and dandy. But when you have an engineering team that needs to work with databases, authentication, logging, deployment, scaling, machine learning, data analysis and plotting, and do most if not all of that in Julia … it’s so easy to write something that gets checked in that passes tests and code review that just destroys performance benchmarks.

fault1 5y ago

I agree with almost all of what you said. Except personally, I'm much more productive in something like Julia (or R or Python) than a language like Rust for the type of development I usually want to do. I mostly write Python and C++ for my dayjob, but would have a hard time justifying Julia for production at the moment, though I love the language. Even more than Rust or Go, I think Julia could take a page from Typescript in terms of tooling because JS is so dynamic and the tooling is so nice. I do think compiler latency has gotten way better in more recent versions of Julia, but I think better ahead of time compilation, faster interpretation, and code caching are all important. I think Julia's tooling used to be better in Atom (Juno), however, Atom is dead.

blindseer 5y ago
I agree with you. Julia has gotten better but it’s still way slower than Python or R. And the workflow of running tests just make it painful. I think I’m familiar enough with Julia and I can write code and it’ll likely just work, but I’m not a 100% confident. With rust (and Python or Typescript with type hinting), I’m much MUCH more confident.

amkkma 5y ago

> there's a LOT more language features required to help tooling in the ecosystem improve. Aside from interfaces, what would those be?

blindseer 5y ago
I’ll first say that this is my opinion, and that I don’t have good suggestions for how Julia as a language should implement this, just that it should: * Better language level support for testing * Better language level support for docstrings - currently doc strings are just normal strings stored in a global struct, and are missing crucial information about the type signature of structs or functions or methods that are being documented * A way to type hint without using multiple dispatch / interfaces - I want to be able to say this argument has fields x, y, z but I don’t want to use multiple dispatch to enforce this interface. Multiple dispatch is useful with the implementation details of your code depend on the data structure. But I want better static type checking for a handful of fields or methods. Typescript is absolutely awesome at this. * Support for using @copied s.x = 1 where s is an immutable struct. Users are literally incentivized to use mutable structs instead of immutable structs because that results in shorter cleaner syntax, even though performance drops through the roof in that case. With immutable structs you’ll have to recreate the struct by calling the constructor. In most cases, the compiler will compile as this away anyway but users are going to prefer shorter cleaner syntax when writing code. * Better macro support. Currently macros have to be valid Julia code, meaning you end up with weird dsl that only kind of sort of matches the domain being modeled. In rust and nim I can literally copy almost any syntax and paste it in a macro block and it’ll just work. This is more work for macro developers and probably will make inference harder but will improve readability. * Language level support for extending Array or letting users define arrays that are as efficient as the built in ones. * Similar request for String buffers. * Rewrite parser that is currently in scheme in Julia, I suspect better tooling will fall out of this * Support for a subset of Julia that is fast to compile and run and produces small binaries. I think more generally I want Julia to be a slightly different language than it is ( or possibly ever can be ). Currently a user of Julia needs to be cognizant of too many things that could accidentally introduce type instability. Almost none of these problems are going to be encountered by someone that understands how compilers work / how computers work, which in my experience is not a lot of domain scientists but is a lot of the core team working on Julia. Domain scientists have PhDs or Masters in highly niche subjects after years of writing Python, R, or Matlab but are more often than not terrible software engineers. I can’t tell how many times in almost a decade of being in this field have I seen 10,000 line scripts in a single file without any comments written by extremely smart scientists. Guiding these folk from the get go to make better software engineering decisions is important imo. Currently, if I’m leading a project, I can’t trust a fresh graduate or junior engineer to write idiomatic type stable fast and extensible Julia code, even if they are the experts of their domain. They have to be experts of their domain AND awesome software engineers which is so rare to find. Given additional constraints about funding, writing proposals, documentation etc, it’s honestly kind of amazing that Julia’s libraries are as good as they are. In my opinion, Julia really shines when one or two people have to write code to tackle a specific problem. Code is super concise and clean and it all fits in your head and everything is fine and dandy. But when you have an engineering team that needs to work with databases, authentication, logging, deployment, scaling, machine learning, data analysis and plotting, and do most if not all of that in Julia … it’s so easy to write something that gets checked in that passes tests and code review that just destroys performance benchmarks.