18 ms·

What's bad about Julia?

• The_rationalist 5y ago
  the language integration/interop could be solved if they migrated to GRAALVM
• adenozine 5y ago
  The things that stick out to me as unwieldy in Julia: * Not embracing functional style idioms. There's no TCO so writing recursive algorithms from old FP papers is hard. Not an issue per se, just gripe. * Not having explicit interfaces. I'm sure there's some advanced Julia-fu that I could do to introspect my way to knowing what methods I need, but the very concept of them not being explicit irks me. Again, not an issue per se, but it annoys me. * There's a complete and total disregard to any notion of a smaller Julia or a Julia more suited for general purpose. Julia seems ripe to benefit from a Ruby&mruby sorta situation, but alas. We need this gigantic blob of dependencies and toolchains and compilers and this and that, to operate this programming language. I don't get it. For the last time, it's not an issue per se, and the language has achieved incredible engagement in many demanding niches, so what do I know anyhow?
  • tines 5y ago
    There's no TCO in many languages, even proudly functional ones like Clojure, so that doesn't seem to be as make-or-break as you and I would have expected. (Clojure does have recur though, but it doesn't help for mutual recursion.)
    • sitkack 5y ago
      Clojure has no TCO so it can interop with Java. It is the last language I would bring up as evidence. https://clojure.org/about/functional_programming https://clojure.org/about/functional_programming
      • tines 5y ago
        I don't think it matters why; the fact is that it's a popular "functional" language that doesn't support (full) TCO.
      • thayne 5y ago
        fwiw, scala has some limited TCO, and it also runs on the JVM
        • e12e 5y ago
          Kotlin has explicit tail-calls via tailrec: https://kotlinlang.org/docs/functions.html#tail-recursive-functions https://kotlinlang.org/docs/functions.html#tail-recursive-fu...
          • tines 5y ago
            Looks like the equivalent of Clojure's `recur`. Unfortunately doesn't work for mutually recursive routines.
  • coldtea 5y ago
    >There's no TCO so writing recursive algorithms from old FP papers is hard. Why write "recursive algorithms from old FP papers"? As an exercize, or is there a business need?
    • neolog 5y ago
      Inherently recursive algorithms are clearer when written in recursive style.
  • amkkma 5y ago
    >There's a complete and total disregard to any notion of a smaller Julia or a Julia more suited for general purpose That's false. General purpose Julia is a stated goal of the core team, it's just that numerical domination is higher priority. Also, small binary static compilation with tree shaking is further along than you think
    • dnautics 5y ago
      yeah, if anything that's what I dislike about julia, I wish it didn't claim to be a GP programming language, and instead worked more on figuring out how to glue it to languages that aren't C. For context, I have basically stopped using julia (not for any taste-related reason except I started doing jobs that were more suited for elixir and haven't had to do numerical computation since), and elixir doesn't claim to be a GP-PL and it's perfectly fine and doing quite well.
      • oscardssmith 5y ago
        Is there anything you think that prevents Julia from being general purpose? The main one I run into is that the garbage collector needs work, but fundamentally I don't think there's much that keeps Julia from being good as a general purpose language.
        • dnautics 5y ago
          "JIT" (ELAOT) compilation and tail latencies. Python is a GP pl (not a very good one, IMO, but it is one). For a long time for some distros Linux CLI and even GUI tools were in python. Can you imagine such an ecosystem in Julia? I can't, nor would I want Julia to make the sacrifices it would need to make to fulfill such a role. There are so many things that python is "mediocre, but good enough" at, that Julia is quite frankly terrible at, because of the (good for what it does) choices that Julia made. I wouldn't write a quick CLI using Julia (versus a long-running or test runner CLI - I have done this professionally [0]). I wouldn't write a web server that is expected to take a load (a personal website is probably fine, though probably still quite painful). I wouldn't write Anything embedded. [0] Wrote a containerized storage block device performance measurement tool. Julia was useful to generate and track statistical distributions of random reads and writes with very clear and concise code. I also think Julia's concurrency story is not that great. But I'm biased against every PL since I have spent a ton of time in erlang/elixir-land which currently has the only (high level) concurrency story that makes any sense IMO. I liked when Julia's concurrency was done per-process (still lives on in Distributed.jl, iirc). I get that this has performance implications, especially around spawning and expensive communications, even with MPI, but I kind of wanted some sort of non-locality of data to be acknowledged, and maybe I would have preferred the new threading concurrency to work with the Distributed.jl abstraction. I think it would have been even more awesome if Julia GPU treated computation against gpu arrays in the same fashion as a distributed job... But as of the last time I used julia (it's been a while) that wasn't the case.
• PaulHoule 5y ago
  Often the lazy pattern for evaluating iterators has bad "mechanical sympathy" and the strategy of "evaluate step 1, write it into a list" and so forth often beats the "step 1 calls step 2 calls step 3..." pattern by an order of magnitude in performance. You just win so much by keeping the inner loop in the I-cache, having the branch predictor completely focused on one inner loop at a time, etc. Writing/reading the array burns memory bandwidth but the CPU usually likes the access pattern. True the eager approach can fail terribly if the array is huge and it's particularly obnoxious if you are running 10^6 calculations and just one of them makes a big array, but the lazy approach is one of those lower teachings that gets passed off as a higher teaching.
  • sesuximo 5y ago
    I think what you really want is a jit compiler that decides what to evaluate on a case by case basis. Julia has nearly all the inputs it needs to do this well… it just doesn’t
• tines 5y ago
  It's probably just a huge bias, but I feel that, as a programmer matures and experiences more and broadens his or her horizons regarding programming languages, he or she takes a road from dynamically typed languages to statically typed languages, often going through lisp as an intermediate step.
  • coldtea 5y ago
    That's what a programmer mid-way to maturity would do. When the programmer finally reaches enlightenment they see that static types have their own burden, and Lisp or Smalltalk or Forth or Erlang, can be the final step, not some gateway drug to static types. Heck, C and C++ ain't bad themselves.
    • andi999 5y ago
      So the answer to the question of life is c++?
      • lasagnaphil 5y ago
        Yes, that's the moment you finally accept that reality is just plain shit sometimes, but you need to deal with it and move on.
    • otabdeveloper4 5y ago
      > and Lisp or Smalltalk or Forth or Erlang, can be the final step Nah. Final boss programmers (Fabrice Bellard, Mike Pall, etc.) work in C. (Or maybe sometimes a bit of C++.)
      • coldtea 5y ago
        For systems works maybe. Most impressive application programs and games I've seen were in C++. But I've seen boss programmers do wonders in all kinds of languages. Didn't this Notch guy make 2.5 billion in a game he wrote in Java?
• mountainriver 5y ago
  Great article, thanks for writing this, I’ve had a lot of frustration with the Julia community being unwilling to see faults in the language. I really want to like Julia and I think I’m theory it has a lot to offer but due to much of what you listed I find it hard to develop in. One thing I would add is that package management is also odd to me. It seems to have a lot of overhead and behave in unexpected ways. In general I think Julia could learn a lot from Go. When I develop in Go most everything is simple. The type system is easy and just works, packaging is simple and usable, it compiles quickly. I fend writing the language seamless and productive. Unfortunately Go also isn’t good for data science due to the memory model so Julia has a lot of room to step in there.
  • arkitaip 5y ago
    > I’ve had a lot of frustration with the Julia community being unwilling to see faults in the language. How about just moving on? It's not like there is a lack of languages and tools out there.
    • jakobnissen 5y ago
      All languages have problems. Don't take my post to mean "don't use Julia" - the very first words of the post is that it's my favorite language. Also do note that the webpage is built with Julia. Not a coincidence. But even good systems need critique, otherwise there can be no improvement or growth.
      • dochtman 5y ago
        Your praise for Rust combined with the list of Julia problems left me curious what you find redeems Julia even though it has all these problems.
        • mountainriver 5y ago
          Rust is too low level for most data scientists, the borrow checker interrupts the flow of thought and it doesn’t work well in notebooks.
• jstx1 5y ago
  Adding to the list - the way it treats namespaces. "using Library1, Library2" at the top of a file makes it difficult to figure out where each function comes from. It's like "import *" in Python.
  • jakobnissen 5y ago
    Yes, I should have added that. It intersect with the type system in a complicated matter, since two functions can co-exist in the same namespace as long as their signatures are incompatible. Which of course means that, if I'm correct that the community is moving away from type signature as much as possible (they will still be needed to some extent to control dispatch), the problem of namespace collisions will increase over time.
    • Certhas 5y ago
      I think you're completely right with this observation. In practice much of Julia is duck typed. It uses types to pass information from the outermost caller to the compiler of the inner functions, not to express invariants. Human annotations get in the way of that, and thus get in the way of composability. You want to annotate Interfaces in Julia, not types. But we don't have interfaces.
    • WanderPanda 5y ago
      Aren't namespace collisions the feature that makes multiple dispatch so composable? I think this is what allows to replace the built in Arrays by GPUArrays and most of the code should still work because all the major operators and functions have GPU implementations
      • BadInformatics 5y ago
        They aren't collisions so much as overloads. e.g. GPUArrays implements the same base AbstractArray methods. I'm not a big fan of using either (in any language) because it obscures where imports come from, and AFAICT overloads are created even with a plain import.
    • 5y ago
• the__alchemist 5y ago
  Re point 1 (JIT latency): This has been done to death, but truly fixing it would make Julia part of my toolkit. For example, right now I'm experimenting with numerical diffeq solvers for chemistry. It involves experimenting with parameters, and plotting. AFAIK, this isn't possible with Julia unless I switch to a REPL or notebook workflow. This brings up a key question: What is the technical limitation behind compiling dependencies every time you run the program? Is non-REPL workflow an afterthought? I'd rather use Rust for something like this! `cargo run` works faster than Julia's JIT since it doesn't need to recompile dependencies like a plotting or numerical lib each time I change a parameter. Out there idea: What about a new language that takes advantage of Julia's strengths, like clean mathematical syntax, 1st-class array support, and being a fast scripting/numerical language, without some of its problematic design decisions? Julia's been the language I've been wanting to love for years.
  • jakobnissen 5y ago
    It's a fundamental problem of having a dynamic, compiled language. It's not possible to actually solve. Think of it similar to how C++ and Rust needs to compile its source code before running it. No-one talks about this as a problem to solve. The problem with simply compiling dependencies to static binaries is that all Julia code is allowed to redefine other Julia code. So package X can load package Y, then define a method that causes package Y to change behaviour, and thus needs to be recompiled. This is not unintentional, by the way. Having packages able to use each other's code is critical for having the ecosystem be "composable", that is, being able to make two different packages work effectively together. For example, I might take a type from package X and throw it into a function from package Y, and it works.
    • the__alchemist 5y ago
      C++ and Rust don't need to compile dependencies between program code modifications. Compiling a Rust program is faster, anecdotally, than running an equivalent Julia one. At least for the numerical and plotting uses I've tried.
    • fiddlerwoaroof 5y ago
• Certhas 5y ago
  I agree with much of what the article gripes about. To me it's not prohibitive, and obviously brilliant stuff is being built in Julia as it is right now. But I just feel that things could be that much better. That said, I do feel some of the defensiveness around these topics from years past is fading somewhat. Still, while some of the things are fixable in Julia 1.X, others likely will require a 2.0 release...
• lloydatkinson 5y ago
  Wow, that an awful font.
• cetu86 5y ago
  I have to admit never really dug into Julia. When it was new I read about it as I was looking for a replacement of python/numpy. But then I was really repulsed by the fact that array indices don't start at 0. c.f. https://www.cs.utexas.edu/users/EWD/ewd08xx/EWD831.PDF https://www.cs.utexas.edu/users/EWD/ewd08xx/EWD831.PDF
  • oscardssmith 5y ago
    As someone who felt the same way, trust me. You'll hit it twice, and then start using eachindex/ begin,end and then you'll not notice it anymore.
  • jstx1 5y ago
    In practice, indexing is the least of anyone's problems. I switch between 0- and 1-indexed languages a lot and it's never an issue, I don't even have to think about it.
    • pavon 5y ago
      I disagree. When immersed in a language it doesn't tend to be an issue, but switching between them absolutely is. Over my career I've had to deal with a ton of off-by-one bugs in code that was prototyped in Matlab then ported to C++. My hope with Julia was that the language was fast enough that you wouldn't have this divide between prototype and production, but given the overhead in using it anywhere but a REPL, it doesn't look like things are going to work out that way. On it's own, 1-based indexing certainly isn't enough to keep me from using a language though.
  • parfamz 5y ago
    This was my experience as well.
  • robot_no_419 5y ago
    Starting arrays at 1 deeply and personally offends me as well, but I honestly can't see that as being a reason to discard a programming language TBH.
  • 5y ago
• habibur 5y ago
  > Large memory consumption [...] 150 MB for hello world. My personal tests showed similar results. Good to know that I wasn't doing something wrong.
• cardanome 5y ago
  The compile time latency is the biggest the biggest show stopper for me. Yes, you can cope with it by staying in the REPL and they are actively working on it but it is very unfortunate that the "slower" competitors like Python and R feel so much more responsive while developing. Other than that, Julia is a absolutely beautiful language. Sure, it is not offering the same static safety as Rust or Haskell but I would not hold that against it. There is always some trade offs to be made regarding the dynamic vs static spectrum and the dynamic aspects serve Julia well for the typical use cases.
  • williamstein 5y ago
    The compile time latency is really bad for some libraries. I was recently testing out computing class groups using https://github.com/oscar-system/Oscar.jl https://github.com/oscar-system/Oscar.jl and was surprised to hit *several minute* latency due to LLVM compilation. For me, the most striking things are that (1) there is no indication in the REPL that calling a function is slow due to LLVM compilation, rather than the function actually running, and (2) it is difficult to avoid paying the compilation penalty every single time you restart Julia, since caching the JIT in general is a technically very, very difficult problem. I wonder if there is an easy fix for (1) that I don't know about: some way of showing a spinner or something saying "I'm compiling code right now".
• temp8964 5y ago
  Is there also something about timing? Given that R and Python are so mature and active, this really leaves not much space for Julia. Yes, Julia is faster than R and Python (just the language, not packages), but R and Python can easily integrate with C/C++ and many R and Python packages are in fact internally C/C++. This really leaves not much advantage for Julia against R and Python. If Julia had appeared earlier, it may have had better position in the data analytics programing ecosystem, and gain more attention and opportunity to grow. Now, the whole data industry is booming, but people would rather contribute to the matured R and Python for real work, instead of waiting for Julia to grow.
  • oscardssmith 5y ago
    I think Julia has a pretty good path forward since it is often faster than C++ (especially once you consider language interop overhead), and it is much more pleasant to write than c++. Since Julia and most of the packages are written in Julia (for the most part), Julia makes it really easy for users to become developers. Python and R have much bigger barriers to entry since users will usually hit a binary blob, which makes it much harder to interact with library implimentations.
  • jakobnissen 5y ago
    I don't think that's so convincing, actually. When I learned programming, Python was _just_ replacing Perl as the "common language" in bioinformatics. Until then, there were lots of people who claimed Perl would essentially never be dethroned because it had become the standard. The issue is that while _some_ users only experience the surface level of packages like PyTorch, Numpy, SciPy, Pandas, and so on, people who actually have to develop these packages must work with the C++ and C underneath. To the developers, the problem of Python's performance is front and center, and a real issue they face every day. So, the developer folks will make the switch first, as they realize it's much nicer to develop Julia packages in Julia than it is to develop Python packages in C. Over time, this will mean the development power switches to Julia - and as soon as these developers have build tools as good or better than existing tools, the majority of users will slowly migrate, too.
    • 5y ago
• snicker7 5y ago
  Referring to start-up latency, the author asserts: > But the problem is fundamentally unsolvable, because it's built into Julia on a basic design level. Citation need. Today, the Julia compiler runs in JIT mode. But there is nothing preventing it from being ahead-of-time compiled (indeed, some work is going behind the scenes to make this happen). There is also an interpreter for Julia. I don't see latency or large memory footprints as being "fundamentally unsolvable".
  • lasagnaphil 5y ago
    Yeah, though the solution for it basically seems to be "throw out all the previous code that was highly coupled with LLVM and actually write our own fast JIT compiler and interpreter from the ground up". It's possible, but probably takes an incredible amount of work and would maybe involve cloning a copy of Mike Pall (the creator of LuaJIT) to help on the project.
    • snicker7 5y ago
      I don't think you need to rebuild the compiler. Here is a proposal how how this can be done using the work to compile Julia for GPU's: https://docs.google.com/document/d/1rLpZf-HK6b9uYPDT_piUza4LUhtIrLbUftGa_YaXGKQ/edit https://docs.google.com/document/d/1rLpZf-HK6b9uYPDT_piUza4L...
      • lasagnaphil 5y ago
        My response's more about the dynamic execution side of things (latency & memory bloat of Julia). Static compilation is a whole different topic, but I'm confident that it can be solved with existing LLVM infrastructure (because what's LLVM good at? static compilation!) The hardest part may be how to modify Julia's semantics to fit that paradigm (for instance, the need to introduce "header files" in the language, and all the gnarly details coming with it)
  • otabdeveloper4 5y ago
    > I don't see latency or large memory footprints as being "fundamentally unsolvable". Dunno. Look at Python. Does "never in a million years, because switching to a better language would be easier" count as "fundamentally"?
• deleted 5y ago
  [deleted]
• galimaufry 5y ago
  One thing that I miss coming from python to Julia, is typing-as-documentation. In python I often write type annotations that are much more restrictive than they need to be. They work as checkable documentation of how a function is expected to be used (and under what circumstances do I guarantee it will work), without restricting how it can be used.
  • gugagore 5y ago
    I think this is a real conflict. In my opinion (not just mine), the only reason to write type constraints on a method definition in Julia is to control dispatch. Adding types to method arguments for the purposes of documentation is counterproductive to generic programming.
    • galimaufry 5y ago
      You're not wrong. I guess what I'd like is the ability to apply (up to) two type annotations, with the second one a subtype of the first, and use the first for dispatch and the second for documentation/testing/static analysis... ... which actually seems like it might be doable with some macros? Those are beyond my ken right now, but the goal would be @doubly_typed f(x::Number|Integer) = x f(6.0) # Same as g(x::Number) = x; g(6.0) @strictify f(6.0) # Same as g(x::Integer) = x; g(6.0) Then you would use @strictify when running tests to ensure that the stricter types in your codebase are all compatible. But you'd still need to figure out what to do about return types and the help command...
    • ChrisFoster 5y ago
      I used to think this was true (as a developer of a lot of generic Julia code and small data analysis applications). But now as a developer of larger amounts of "application style" code, I'm not so sure. In an application, you've got control of the whole stack of libraries and a fairly precise knowledge of which types will flow through the more "business logic" parts of the system. Moreover, you'd really like static type checking to discover bugs early and this is starting to be possible with the likes of the amazing JET.jl. However, in the presence of a lot of duck typing and dynamic dispatch I expect static type inference to fail in important cases. Static type checking brings so much value for larger scale application work that I'm expecting precise type constraints to become popular for this kind of non-generic code as the tooling matures.
• spekcular 5y ago
  I like Julia because of the trivial 10x performance improvements I get over writing similar code in Python/numpy. What I don't like is how Julia attempts to force you into a REPL workflow. Everything is optimized for REPL work. I just want to write a script in a text editor and run it from the command line. Please make this easy for me. (Other HN comments have said this in more detail, for example: https://news.ycombinator.com/item?id=27310747 https://news.ycombinator.com/item?id=27310747.)
  • jakobnissen 5y ago
    It's really hard to do, because having a command-line centered development experience with Julia is a complete nonstarter until compile time latency is near zero. And that's just really, really hard, and unlikely to happen, at least any time soon.
    • cbkeller 5y ago
      Hopefully eventually
  • totalperspectiv 5y ago
    That is an extremely well put comment in the link. Thanks for sharing it.
  • gugagore 5y ago
    > In programming I tend to think the whole computer and its OS and so on as the programming tool. My impression is that for Julia folks Julia is something totally separate from anything else in the computer. But I don't understand why. Is it that its felt that changing syntax on the fly is too cognitively demanding or something? This is something that one gets used to very fast. This comment really does not resonate with me. It sounds like a criticism of Smalltalk, not Julia. Not sure what it means to change "syntax on the fly", but that does sound like the sort of design that is maximally flexible, at a considerable cost to readability. Macros already allow you to embed DSLs (as long as they parse), and almost everyone agree that they should be used sparingly. > And also tends to ease the IMHO weird hung-ups on syntax. In fact, I think Julia has made some (IMHO misguided) syntactic decisions because they just want to do something differently from Python just for the sake of being different. I find this totally senseless. I am not aware of any decision that is done just for the sake of being different. If there are any, then yes that's senseless.
• CountSessine 5y ago
  I love julia and wouldn't mention this if I hadn't written a big chunk of data-analysis code with it... It's a small gripe, but one thing that really bothers me is that they gave up on libreadline for the REPL (for what seem like fairly superficial build management issues, looking at the original email threads) in favor of a custom solution, which * doesn't support vim-like or emacs-like keybindings, and * won't read my .inputrc file and my customizations It's just maddening using julia's REPL without this. It's easy to dismiss user complaints like this, but I can go back and forth between bash and into the python REPL and all of my line discipline actually makes sense, but then the julia REPL is broken and clumsy. Also, pdb.set_trace() is a lot easier and more productive than what Debugging.jl has to offer.
  • AlexCoventry 5y ago
    Have you experimented with something like rlwrap? https://github.com/hanslub42/rlwrap https://github.com/hanslub42/rlwrap
    • CountSessine 5y ago
      It interferes with the Julia REPL line discipline.
  • rrock 5y ago
    You might have some sort of dangling config issue somewhere? I use the emacs keybindings in the Julia REPL all the time. They work fine out of the box (at least here).
    • CountSessine 5y ago
      Ah good to hear. I wish I could edit my post. But vim bindings are nowhere to be found. The good news I guess is that having a clumsy REPL has really gotten me into vim-slime.
  • ChrisFoster 5y ago
    Have you tried Infiltrator.jl? It's great for breaking out into an interactive REPL from some inner scope, allowing you to examine the local variables and other program state interactively. This covers the part of the functionality of pdb.set_trace() which I care about.
• BlanketLogic 5y ago
  While all the points are valid what scares me most about Julia is what is covered under the section "The core language is unstable". Recently I tried the latest beta version and came across an issue - I just could not add packages. (`] add JSON` or ). No matter what package I tried to add, it failed with a stacktrace pointing to libuv. I tried downgrading to lower versions of Julia but no luck. Turns out it is a regression introduced by a commit dated Sep 23 2020. In libuv. As I tried to fix it, I realized Julia uses a version of libuv that is significantly diverged from libuv main branch ("123 commits ahead, 144 commits behind libuv:v1.x" as I write this comment) and the bug was in the code. Using the code from the corresponding function from libuv upstream fixed the problem. To summarize 1. There's a bug introduced close to a year ago that breaks basic functionality on a mainstream operating system. 2. Julia has its own version of, of all things, libuv, that is significantly diverged from the original. 3. The bug is in one of the changes introduced. While each of the above is defensible on its own. Taken together, they do scare me away from considering Julia for production use. I am hoping am wrong somewhere. I think it is a lovely language.
  • eigenspace 5y ago
    I mean, that's bad and scary, but you did mention that you were using a beta version of julia. Finding this sort of thing is exactly why the beta versions exist. If the beta was stable and ready for production, it wouldn't be called beta...
    • jakobnissen 5y ago
      I think you missed the point where he said the bug was present in the stable release, too.
      • eigenspace 5y ago
        Oh I see I did misunderstand. Yes, that is unfortunate. I'd guess that since this made it into a stable release, it's likely a pretty tricky bug to encounter, since the failure is so spectacular. I will say though that bugs appearing in stable releases of any software is a bad, but practically unavoidable thing. It sucks that this bites people, but it's not what I'd really call instability.
• shakow 5y ago
  This article is painfully true. Julia is full of gems, but also features so many warts everywhere that I'm not sure whether it will be able to overcome the latter – I hope it will. I would add to the author's list the subtle differences between REPL and scripts, the weird obsession of Julia for embedding slightly different and out of date versions of its dependencies (BLAS, uv, LLVM, ...) that makes it a nightmare to package, the painfully slow documentation website, the sometimes surprising function names, and wholeheartedly agree on the failure that is functional programming handling, both in syntax and implementation – which is surprising for a language stemming from MIT.
• spacedome 5y ago
  I love Julia and choose to work in it almost exclusively, but I agree with the points in the article. I've run into a lot of issues just writing numerical linear algebra type algorithms. Even core, and not quite core but maintained by core dev, libraries like Distributed.jl and IterativeSolvers.jl can feel pretty rough. For example IterativeSolvers has had strange type issues and not allowed multiple right hand sides for linear solves, for years, afaik due to some aspects of the type system and some indecision in the linalg interface. DistributedArrays still is very poorly documented and looks like it hasn't been touched in 3 years. I've run into problems when I need more explicit memory management, for example none of the BLAS/LAPACK routines have interfaces for the work arrays, so you either get reallocation or have to rewrite the ccall wrapper yourself. It can also be hard to tell where the memory allocation is happening. My most recent problem had been with Distributed and DistributedArrays, where everything is fine if you just want a basic parallel mapreduce, but has been a huge pain past that. It's not even clear to me if Distributed/DistributedArrays has been more or less abandoned in favor of MPI.jl, which for me removes most of the benefit of writing in julia, since you then have to run it through MPI. There is an MPI sort of interface for DistributedArrays but that part is not well documented and looks like more of an afterthought. My use case isn't even that complex, I just want to persistantly store some matrices across the nodes, run some linear algebra routines on them every iteration and send an update across the nodes, then collect at the end. If anyone has any idea how to do this correctly in Distributed or DistributedArrays or can point me to some examples that would be amazing because it has been taking me forever to piece it together. Not going to stop using Julia but there are many basic things even just in a scientific computing workflow that still feel like they were rushed and they can really take the wind out of your sails.
  • jakobnissen 5y ago
    Agreed, but it's funny that criticism of Julia broadly falls into two categories: 1) Julia doesn't have X. X is critical for modern programming languages, and without X, we should not even entertain the idea that Julia may be usable 2) Julia's feature X is too unstable. It's like they tried to implement too many things in Julia 1.0, and developer time stretched thin. They should have just not implemented all this stuff! I mean yes, we all would like a programming language that materializes with 1,000,000 developer hours already poured into it, great editor support out of the box, and which is somehow born with 10 years of usage. It's similar to wanting an employee who enters the work force with 10 years of industry experience. Nice, but it's not very realistic.
• neolog 5y ago
  I agree with everything in this except "You can't extend existing types with data". Data inheritance is implementation inheritance. Don't add it.
  • ivirshup 5y ago
    For sure. I particularly took issue with: > In e.g. Python, you are not going to run into types you want to subclass, but can't. In python, you can only really subclass things that were designed to be subclassed. Subclassing arrays, dataframes, or anything from xarray or dask is explicitly recommended against by those packages. In practice I only subclass the base scikit-learn transformers, Abstract Base Classes, or classes I've written.
• sharikone 5y ago
  In my opinion the most important positive thing Julia has is the Compsci community it attracted. There has never been so much enthusiasm in oftware for doing physical simulations and correct and stable arithmetics. It made a sizeable dent in the Fortran/C++/Python/Matlab environment physicists were in. That said even after 10 years Julia has still so many rough edges that I wouldn't employ it for anything outside the aforementioned use case. Just getting it to Plot something in a headless environment was a pain. Essential utilities like debuggers (!) are external dependencies. Matrix/Vector/Array confusion permeates the air. Ditto for a syntax annoyingly in the middle between Matlab, Python and Fortran. Given it is already not that young I predict it will never really attract the masses beyond its niche, but it can still kill the remaining Fortran codebases.
  • jakobnissen 5y ago
    It's a little hard to say "after 10 years" when Julia has been at 1.0 for 3 years only. And remember, that's not with Rust's niche of low-level developers, but with a niche of people who are generally not programmers, and therefore much more conservative about their languages! It's surprisingly hard to get good data about how quickly Julia grows, but it certainly doesn't look like it's plateauing yet. It's just too early to know how far it'll go and where it will end up in 10 years.
    • NationalPark 5y ago
      If anything, Julia becoming popular with academics makes me extremely skeptical about its missing guardrails. Obviously anecdotal, but I have many years of experience working with academics and statisticians and I know to be wary when they suddenly like a new technology. Usually it's because it makes it easier to swallow errors somehow, or because their friend wrote it and sent them an email about it.
      • jakobnissen 5y ago
        Ha! That's hilarious. Yeah, I've seen some real horrorshows of programming in my field. I think it's important to realize, though, that academics do have a special use-case when programming. That is, academics don't pick e.g. Python because they are bad programmers and don't know what they are doing, but because Python is a good fit for their needs. I see Julia as an excellent - near-perfect, in fact - academic programming language. That Julia excels at this use case does not mean it's bad at everything else.
• bionhoward 5y ago
  One thing I found about Julia which turned me off was the library used for distributions used 64bit floats while that used for neural Nets used less precision, so it's hard to connect the dots there. Also there wasn't a lot of great tooling for RL
  • klaff 5y ago
    If you specify a distribution using, say, Float32s, you'll get a distribution in Float32s. I'm guessing the machine learning example used Float32s so they would better take advantage of a GPU. It's almost always true in Julia that you can combine your choice of numeric type with other neat packages without the author of either one being aware of the other. Want to simulate ODEs in Float32 for speed on GPU, in Float64 for GPU, or if you have an exotic need for high accuracy use something bigger but slower, you can do it.
  • BadInformatics 5y ago
    Regarding RL, did you look at https://juliareinforcementlearning.org/ https://juliareinforcementlearning.org/? I haven't used it myself, so it would be interesting to know where it's lacking.
• Tarrosion 5y ago
  OP, thank you for this post. I use Julia daily, and it's likewise my favorite language. So I went into this post prepared to be...defensive. But several times in the post I found myself thinking "no! Wait, well, maybe, hmm, urgh, yeah that's not great." I'd like to read an epilogue or follow-up "so what" post. If Julia is your favorite language but many of these problems seem like some combination of a) unsolvable given the design priorities of the language; b) unsolvable until a hypothetical future 2.0; c) solvable but only with huge resource increases, how do you manage these problems day to day? What do you see as the moderate term future?
  • jakobnissen 5y ago
    Thank you! That's exactly what I was aiming for. That sounds like a good idea. The problem is that I just don't know enough about compilers, Julia's internals, and how the core dev team works to really forecast the future and know what's possible. How hard would it be to implement caching of native code? A fast interpreter that runs side by side with the compiler? No idea. Maybe it would still be useful to make a post explaining how I cope with the limitations of Julia. I'll think about it.
    • sharikone 5y ago
      I think this is what parent is referring to. A short guide from a power user on coping with those flaws would be invaluable for occasional users.
• spullara 5y ago
  He points out a bug: julia> open(read, "/home/jakob/Documents") # yes, a directory UInt8[] I'm not sure if this is true in other languages but it is also true in Java when you attempt to read a directory as if it were a file. I ended up adding code to test if it was a directory to avoid the issue. Does this happen at a lower level?
  • bitwize 5y ago
    It's built into the semantics of the OS. Unix directories are ordinary files, treated specially. They can be opened and read (and written to) like any file. If you have a more primitive pager than less, you can sometimes see this by running it on a directory; you will see binary garbage with file names embedded in it, like whole pieces of corn in a turd. That said, if your HLL doesn't provide a better abstraction when reading a directory with the standard file I/O primitives, it is indeed a serious bug in the language. Don't do that, language designers and standard library authors!
  • jakobnissen 5y ago
    It originally threw an error in Julia, but it turned out that checking if a file is a directory is a surprisingly expensive system call, so it was removed. It seems the issue is stuck, since most people agree the current behaviour is bad, but forcing every file read to do a heavy system call is not a great solution
• gorkish 5y ago
  So, Julia is Perl?
• ummonk 5y ago
  The issues with the type system and weak static analysis are the biggest gripes for me. The other features of Julia still make using it worthwhile, but it's rather frustrating to code compared to any language with good static type inference and a powerful type annotation system.
  • jstx1 5y ago
    That's one of those things that are never going to change. Julia's very specific type system is one of the major features and so much of the language and the ecosystem is build around it.
    • ummonk 5y ago
      Some issues are fundamental, sure, but I would guess there is a lot of room for improvement still. Abstract interfaces seem perfectly compatible with the type system, for example. I think the situation with traits could be improved somewhat as well. And there is no question that the situation for static analysis could be improved without having to overhaul Julia's type system.
      • BadInformatics 5y ago
        To your point about static analysis, https://github.com/aviatesk/JET.jl https://github.com/aviatesk/JET.jl appears far more capable than comparable systems for other dynamic languages.
• lasagnaphil 5y ago
  Julia is incredibly fascinating for me as a language to observe, although I haven't got the time/patience to try it out yet. Its language design seems incredibly progressive and yet pragmatic: they've repopularized dynamic dispatch (which makes perfect sense in a mathematics-oriented language), and has a unique mix of dynamic and static typing that seems perfect in paper. In practice, Julia seems to suffer from the initial architectural decisions that the creators has made when the language was young - the biggest which is its heavy dependence on LLVM. This is a problem since LLVM is mainly used for static compilation of code for languages like C++/Rust/Zig and not with dynamic execution - its included JIT is famous for being unreasonably slow. Even the dev seems to know this but they don't seem to have that much choice - since too much of their codebase is dependent on it. (For example: https://discourse.julialang.org/t/jeff-bezanson-remarks-on-llvm-getting-slower-and-slower/39637 https://discourse.julialang.org/t/jeff-bezanson-remarks-on-l... - LLVM codegen keeps getting slower, which in turn cancels out all the little optimizations they can do for the language) It's a language that seems so perfect on paper but suffers from its real-world implementation - and therefore captivates me in a special way. Some major architectural overhaul might be a much better move (such as a fallback bytecode interpreter independent from LLVM) - but it's definitely going to be a major effort.
  • StefanKarpinski 5y ago
    Using LLVM is not exactly an indelible architectural decision — switching to a bespoke JIT would be entirely possible, it’s just not worth it yet. Moreover, if you want to generate world-class high speed code, you really need something like LLVM. So yes, it’s a bit of a drag but it’s also a massive enabler of critical performance. That’s the real reason the project continues to use it. There are a number of approaches to reducing latency further, which are being pursued. The most promising current avenue is better tooling around system images (compiled executables with additional functionality beyond the core language pre-loaded): generating them faster, generating them incrementally (load one, add a few things and produce another), and generating them automatically for the set of packages in the current active project (not feasible until the first two are done). That will give an experience comparable to compiled languages.
    • 5y ago
• Kalanos 5y ago
  A rule of thumb is that 90% of unhappy customers don't complain. So it's likely that a lot of people are experiencing these same problems, but not voicing them. "Thank you" seems to be missing from a lot of comments. A good response might entail: what is the core team is currently prioritizing, why, and how will this feedback help guide those efforts? Also, take heed of the number of occurrences of the following statement, which I support: > "community is hostile to feedback"
  • oscardssmith 5y ago
    Short list of current priorities (in no particular order) Making it easier to interface with the compiler (this is step 1 on better ability to static compile/debug) Better garbage collection More performance Reducing the memory footprint of small strings Lowering the overhead of multithreading It's worth noting that Julia doesn't really have a core team. A lot of the development is done by a pretty large community of developers without any central management.
    • amkkma 5y ago
      They've been pretty mum on the type system stuff though. Do you have any insight there?
      • oscardssmith 5y ago
        The short answer is that changing the type system to allow multiple inheritance (traits) involves solving some really hard and open-ended problems (eg method specificity). Solving this is probably breaking, and requires a ton of work on a problem that might not even be solvable. If I had to guess it will be solved by the language that replaces Julia in 20 years.
        • deleted 5y ago
          [deleted]
        • amkkma 5y ago
          Thanks
• wbthomason 5y ago
  This is an interesting read, thanks! I'm a bit surprised by https://viralinstruction.com/posts/badjulia/#julia_cant_easily_integrate_into_other_languages https://viralinstruction.com/posts/badjulia/#julia_cant_easi... - I've been considering embedding Julia (as opposed to LuaJIT) in a project I'm working on that requires fast, mostly numerical computation. Based on https://docs.julialang.org/en/v1/manual/embedding/ https://docs.julialang.org/en/v1/manual/embedding/, this seemed fairly simple, and as though I could avoid repeatedly paying the compile time latency cost by just calling my embedded functions at least once before the performance-critical loop (to ensure that they're compiled). Is that not the case? There's definitely still the high memory overhead to consider, but (admittedly without having tried it yet), embedding Julia doesn't seem too terrible to do. Worse than e.g. compiling a shared object library or something, but not at all unreasonably hard.
  • jakobnissen 5y ago
    That might very well be true! But it's still much less pleasant than just using a compiled binary from a static language.
    • wbthomason 5y ago
      Totally fair, yep. Thanks again!
• stabbles 5y ago
  > Here is a fun challenge for anyone who thinks "it can't be that bad": Try to implement a TwoWayDict, an AbstractDict where if d[a] = b, then d[b] = a I've done the exercise, let me walk you through it. First, there is no inheritance of concrete types, only composition, so I wrote a wrapper: struct TwoWayDict{K, V, D <: AbstractDict{K,V}} <: AbstractDict{K,V} d::D end How do I know this signature? By looking at julia> supertypes(typeof(Dict("a"=>1))) (Dict{String, Int64}, AbstractDict{String, Int64}, Any) and the {K, V, D <: AbstractDict{K,V}} pattern is a well-known trick in julia. Then I tried to instantiate a TwoWayDict in the REPL, and it complains about iteration support, so I implement that: julia> d = TwoWayDict(Dict{String,String}()) ERROR: MethodError: no method matching iterate(::TwoWayDict{String, String, Dict{String, String}}) julia> import Base: iterate julia> iterate(x::TwoWayDict) = iterate(x.d) julia> iterate(x::TwoWayDict, s) = iterate(x.d, s) julia> d = TwoWayDict(Dict{String,String}()) TwoWayDict{String, String, Dict{String, String}}() How about setting a value? julia> d["a"] = "b" ERROR: MethodError: no method matching setindex!(::TwoWayDict{String, String, Dict{String, String}}, ::String, ::String) Okay, so let's implement that import Base: setindex! function setindex!(d, key, value) d.d[value] = key d.d[key] = value end julia> d["a"] = "b" "b" julia> d Error showing value of type TwoWayDict{String, String, Dict{String, String}}: ERROR: MethodError: no method matching length(::TwoWayDict{String, String, Dict{String, String}}) Hm, so length is missing, let's add that julia> import Base: length julia> length(x::TwoWayDict) = length(x.d) julia> d TwoWayDict{String, String, Dict{String, String}} with 2 entries: "b" => "a" "a" => "b" Ok great. How about retrieving a value: julia> d["a"] ERROR: MethodError: no method matching get(::TwoWayDict{String, String, Dict{String, String}}, ::String, ::Symbol) Hm, okay, 3 arguments?! Let's see how it works for Dict: julia> get(d::Dict, [tab][tab] get(h::Dict{K, V}, key, default) where {K, V} in Base at dict.jl:504 Ok so julia> import Base: get julia> get(d::TwoWayDict, x, default) = get(d.d, x, default) julia> d["a"] "b" --- It should be documented better, but without leaving the repl it's not that difficult to implement the basics, just some function forwarding really. Now if you ask: did you implement the full interface? I don't know [1], but I do like that julia doesn't force me to implement things I don't need in my code. By not being strict about interfaces, julia solves the expression problem [2], and in some cases this is very useful. E.g. if I want to create a number type and I only want to implement +, -, *, /, and potentially conversion and promotion, then I can do that, but I don't have to worry about exponentiation, trig functions, or whatever people think is encapsulated by the notion of number. [1] Well, I forgot to write a constructor to enforce the d[a] = b and d[b] = a invariant on instantiation, and there's no support for deletion by key. [2] https://en.wikipedia.org/wiki/Expression_problem https://en.wikipedia.org/wiki/Expression_problem
• lamontcg 5y ago
  I just started using Julia last week. The REPL-based workflow and the compile time latency just need to go. Stop defending it. Its a problem. It harms adoption. Fix it and move on and you'll stop hearing endless complaints about it. Sometimes the endless complaints have a point and you need to listen to it. Coming from a decade-long history with ruby I do think that dynamic, duck-typed languages are inherently flawed, but c'est la vie and Julia is miles better than Matlab. I'm not looking at Julia to be exactly like go or rust, I want automatic differentiation. Don't particularly care about the memory bloat either. Newbie documentation does need to be a whole lot better. I suspect that would probably get fixed if the compilation and REPL-centric issues got fixed and it wasn't so immediately hostile to your time on day 1. Package management also needs to get better. Rubygems gets the shape mostly right if you don't look too close at the details, cargo seems to be excellent. Languages created in the past 10 years really should have strong package management from the start and not treat it as an afterthought and not try to reinvent the wheel without understand the successful systems that have come before them. But really its about the compilation and REPL-centric issues. That should have been a requirement for 1.0, and I can't believe that a language that is nearly 10 years old could have that bad of an initial workflow for users.
  • linspace 5y ago
    All languages make compromises and Julia has decided to not compromise in execution speed. I say this not because I don't think compilation time sucks but because 10 years is not that much time for solving a complex problem. How many years in Python without proper parallelism (I don't think the answer is the multiprocessing lib)? I think Julia package system is good but since I come from Python maybe it doesn't mean that much.
    • lamontcg 5y ago
      Stop making excuses for it. It is a 10 year old language and this is a crippling day #1 newbie workflow and productivity issue. Just say "yeah, that's a horrible experience and really needs to be top priority now that they've closed on a $24M funding deal"
• WhompingWindows 5y ago
  As an R user, my issue with Julia: what niche is it filling? R is established in statistics and data analysis, Python is established in general-purpose and data analysis and ML, so why should someone learn Julia? The "main advantage" of Julia being faster than R seems irrelevant to me, R is plenty fast enough for my one-off statistical analyses. Further, I can't justify spending my attention on learning yet another language for analyzing data, given I already learned SAS, SPSS, R, STATA, Excel, and Python.
  • linspace 5y ago
    People who is tired of wrapping C++ or Fortran libraries. It's not exactly a drop in replacement but it's almost there. If speed is not relevant to you then it may not be a good choice unless you are into language learning for the sake of it
  • bsdubernerd 5y ago
    That's probably because you didn't try to write anything significant in R besides running an analysis. R is an interesting language which adapts "ok" to statistics and related fields, however it's very limited by today standards, completely REPL oriented and very, _very_ slow. In fact, the first thing one learns in R is to never loop on any set larger than a few hundred elements if you want your script to execute at decent speed. All the R speed is actually backed by fortran and C/C++ extensions. This goes from the R core to all the scientific and biosciences packages. R as a language has an extremely primitive interpreter and GC runtime. It's often much slower than python or perl. Although given the primitive GC and runtime, it's quite straightforward to write extensions for it. But *nobody* really wants to change language (whichever it is) to write extensions just to get some speed back. Especially when writing these extensions bars you from the entirety of the R ecosystem itself.
    • WhompingWindows 5y ago
      Mmm, very good points. I used R for quite a few high-level statistical analyses, I found it handy for munging data and .rmds are nice for organizing a systematic set of chunks to make the code logical. Sometimes I ran into speed issues, which is when I would reach for data.table, which is quite a huge speed boost for some of R's slowest issues. I had to learn the basics of STATA for a modeling project, because the other researcher knew STATA (these network effects of collaborators knowing different things). Turns out fixed-effects multivariate logistic regression modeling is WAY faster in STATA than R, to the point where our infra just couldn't complete the R code in any reasonable timeframe so STATA was just the better pick. My main gripe is especially in healthcare, so many workers are trained in something in their 20's, then use that for decades. There are thousands out there still using SAS every day, for instance, because it's what they and their collaborators know. I suppose Julia and Python will grow in healthcare over time, SAS should in theory go down in usage...who knows!
• kilodeca 5y ago
  - Active denial of language specification
• thayne 5y ago
  I also Julia, but agree with all these points. And I'll add one more: It is difficult to chain a value through a series of function calls. Meaning you can't do something like `a.b().c().d().e()`. You either have to store intermediate results in variables, call it like `e(d(c(b(a))))` which IMO is less readable, or do something like this: a |> x -> b(x) |> x-> c(x) |> x -> d(x) | x -> e(x) there is some hope of this getting improving in the future, but in the present, it bugs me.
  • jakobnissen 5y ago
    OP here. Try using `Transducers`. It has problems of its own, but it does address most of my issues with Julia's iterators, including that one.
  • klaff 5y ago
    Why did you write all those anonymous functions? You can just do a |> b |> c|> d |> e although there are style guides that discourage that and instead recommend using a line for each with good temp variable names.
    • thayne 5y ago
      ah, in this case that works. although it doesn't if you have additional arguments to b,c,d and e. I guess I should have crafted my example better.
      • BadInformatics 5y ago
        https://github.com/jkrumbiegel/Chain.jl https://github.com/jkrumbiegel/Chain.jl would probably work there, but you're not the only one who would like to see this in the language/stdlib itself.
• hn_throwaway_99 5y ago
  Sorry, slightly off topic, but as someone not really versed in Julia, but who sees it frequently on HN (but, honestly, nowhere else, but that could just be the type of industry I'm in), what would you say the closest "competitor" is for Julia? Or rather, if I am currently using programming language X to do Y, what would be the values for X and Y that should make me really want to consider Julia instead.
  • jaggirs 5y ago
    Python
  • jakobnissen 5y ago
    Matlab, Octave, R, Perl, Python, in order. Especially consider it if you're a scientist, engineer or mathematician - while it's general-purpose, it's really good at these domains.
    • e12e 5y ago
      Possibly also SAS, Mathematica.
  • bobbylarrybobby 5y ago
    Using python for any kind of high performance computing — array/based operations (especially operations in sequence over one array, which will occur in a single pass), distributing work over several processors, moving data to the GPU, even just wanting your for-loops to be fast — all of these are easier in Julia.
• amkkma 5y ago
  Can someone from the core team chime in on the general thinking around the type system/trait stuff? Most of the rest of the complaints are actively being addressed or have been thoroughly discussed, which leaves me unconcerned. Haven't heard much about the type system stuff in years, or it's just been "duck type and thtt works ok". Is that the consensus? Honestly, that's a pretty valid perspective also. Julia works amazingly well and there's no other contender for the same combination of speed and composability. But, I feel like things could be easier/smoother. Is there a general design plan, or maybe there's a desire/intent to do something but it's going to require iteration down the line?
  • mccoyb 5y ago
    Not a core team member -- but this comment confuses me. What exactly do you mean by "duck type and thtt" (?) Also, people are constantly asking for traits. But I don't think I've seen a compelling example where: this is something that you can only do with traits, but you can't do with multiple dispatch. Perhaps what you're arguing for is the ergonomics of multiple abstract inheritance (?) So a concrete struct can inherit from multiple abstract types (?) So maybe it's not a comment on utility, but more ergonomics.
    • neolog 5y ago
      Is there a difference between traits and multiple abstract inheritance? Those seem equivalent to me. Re duck typing: people usually respond to "what is the type of map?" with "it's duck typed". But we should be able to do better.
      • mccoyb 5y ago
        I think the difference is in name only -- but TBH, I'm a little naive on more specific PL details. That's just how I imagine the implementation of that concept would likely go in the Julia space.
        • neolog 5y ago
          I think we have a big PL-shaped hole in our community. Taine's work [1] shows how much PL folks can add to Julia. We need more of them. [1] https://github.com/thautwarm https://github.com/thautwarm
• mint2 5y ago
  Wow this is helpful, I’ve had a slowly growing interest in julia due to all the positive threads in HN, but that latency forcing a repl dev or notebook dev is a nonstarter. That’s a big limitation. And from what I’ve seen of dev done in python notebooks for data analytics, now I’m concerned about the code quality of Julia projects. Does Julia have good linters for notebooks if people are pushed to use them due to the latency?
  • mbauman 5y ago
    Folks generally don't do development in notebooks, thankfully. Instead, they generally use a Revise- or REPL- based workflow, often with VS Code.
  • mccoyb 5y ago
    New compiler improvements have vastly increased the space of linting -- please see JET.jl, for example: https://github.com/aviatesk/JET.jl https://github.com/aviatesk/JET.jl
  • ddragon 5y ago
    I don't think there is anyone developing packages on pluto/jupyter, so I wouldn't worry about that. The most common method for that should be using an editor like VSCode (which will have some linting capabilities) with an open REPL and Revise [1]. What it does is every time you save any of your files (with some known restrictions), it will automatically and incrementally update the state of your application in the REPL, allowing you to probe your code whenever you want (with tons of introspection methods, up to interactively inspecting the native code being generated), and since you never leave the session you don't face the compiling latency more than once. I end up preferring this workflow for experimenting and data science stuff since it retains the structure and tooling of an editor with the ability to interact with my application (and really miss it when I'm writing Python applications), but of course each one has a preferred workflow and it'd be nice if Julia supported more of them as well. [1] https://github.com/timholy/Revise.jl https://github.com/timholy/Revise.jl
• bobochan 5y ago
  This is probably a "me problem," but I have had a hard time moving beyond the tutorials to actually getting work done. For example, I am working on a project right now where I would typically use a dict of dicts in python and then use map functions so that I can process the keys in parallel. When coming to python from perl, ruby, and c++ I found that I already had a pretty good intuition about how this would work. I read the documentation, but could not make as much sense of it as I should. I searched online for example and found a few hits on the question, but not nearly as many as I would in other languages, and if there were answers then they were often marked "wrong," or I could not get them to work with my code. After a lot of searching and trial and error I finally got that working, then started looking into how to run this on multiple nodes and processors and got completely lost. I am sure the documentation is there and that I could do a better job figuring it out. I am also sure Julia would be a great match for the type of work that I do, and I am going to keep trying. But from what I have seen so far it feels like the pain of trying to migrate from Python to Julia is pretty difficult to justify.
  • BadInformatics 5y ago
    This is definitely not just a you problem. There are resources out there like [1] and [2], but much less in the way of curation and a dearth of end-to-end tutorials/walkthroughs. I'm not sure how best to improve the situation, but the current state of things leaves much to be desired. If you're willing, don't hesitate to post about your experience and any feedback you have on the community forums. [1] https://www.youtube.com/watch?v=dczkYlOM2sg https://www.youtube.com/watch?v=dczkYlOM2sg [2] https://github.com/juliohm/julia-distributed-computing https://github.com/juliohm/julia-distributed-computing
• macawfish 5y ago
  I really like Julia, and also get the sense that the community is a little young and unsettled in some ways. The biggest way I've seen this manifest is in terms of a total lack of decent documentation for reputable, well known projects. For example, there's this trend in the Julia community of putting little clever puns in the github package description. Okay I'm not trying to rain on anyone's parade and I appreciate humor but when I'm looking for packages that little github description area is critical. Maybe save the puns for the readme doc? Beyond that, it's frustrating to find core libraries with absolutely zero API documentation or with totally outdated/incomplete/misleading docs. I'm not all that mad about it or anything so I won't point fingers. I just find it to be out of touch and mostly just a buzzkill. In the past when considering investing my time and energy in building a project in Julia, I've found it frustrating to have to do all this guesswork to get a sense for where critical packages are at and what their limitations are. So from an optimistic angle, I think this probably reflects the fact that there are quite a few researchers writing Julia packages, which is really cool. There are some awesome cutting edge techniques implemented as Julia packages. I don't think researchers should be off the hook for maintaining crappy packages, but I do think it takes some higher level thinking to cultivate good tools and collective habits around this stuff. For whatever reason, this is evidently challenging for the Julia community. I hope that some of the culture and tooling around documentation and package maintenance gets more mature over the next few years. If anyone knows about efforts to improve documentation in the Julia ecosystem, I'd love to hear about them.
  • StefanKarpinski 5y ago
    Not to call anyone out but examples? Every time I look at packages I’m blown away by how much effort has gone into their docs. But people keep saying this kind of thing so there must be some packages they’re hitting that are under-documented.
    • tpoacher 5y ago
      I find that "online" docs are good, but "in-repl" docs are usually poor to inexistent (in packages, that is). Base Julia has improved on this a lot, though there's still room for improvement (especially in terms of useful examples or linking to related functions).
• bruce343434 5y ago
  >map, filter and split are eager, returning Array. What is "split"? Is that like reduce? I tried searching but couldn't find anything meaningful to me.
• heuroci 5y ago
  @jakobnissen you have a typo in the first section: "Among Julians, latiency is" - Sorry, I would have nudged elsewhere but could not find an alternative venue to contact you. Excellent article!
• UncleOxidant 5y ago
  I mean, I don't know if this is bad, but I do find the behavior to be surprising: julia> d = [1 2; 3 4] julia> d 2×2 Matrix{Int64}: 1 2 3 4 julia> @show( d[1], d[2], d[3], d[4] ) d[1] = 1 d[2] = 3 d[3] = 2 d[4] = 4 I guess I would have expected something like: d[1] = 1 2 d[2] = 3 4 d[3] -> array index out of bounds error but looks like you have to do: julia> d[1,:] 2-element Vector{Int64}: 1 2
  • eigenspace 5y ago
    This is known as Column major order. https://en.wikipedia.org/wiki/Row-_and_column-major_order#Programming_languages_and_libraries https://en.wikipedia.org/wiki/Row-_and_column-major_order#Pr...
  • klaff 5y ago
    https://docs.julialang.org/en/v1/manual/arrays/#Linear-indexing https://docs.julialang.org/en/v1/manual/arrays/#Linear-index...
    • UncleOxidant 5y ago
      Ok... I read that. I have to say it's very not intuitive. I like Julia overall, but this is an area where I scratch my head and say "Hmmm... why would they do that?" (performance, I'd guess, but it's not very user friendly). Is there a set of guidelines for translating Python/numpy code to Julia because indexing in Python/numpy works a lot differently (and seems more intuitive).
      • klaff 5y ago
        You've probably found this already: https://docs.julialang.org/en/v1/manual/noteworthy-differences/#Noteworthy-differences-from-Python https://docs.julialang.org/en/v1/manual/noteworthy-differenc... That's the only "official" python-julia resource I know of although I bet there are some blogs about the process.
      • klaff 5y ago
• thayne 5y ago
  The only other language I'm familiar with multiple dispatch is common lisp, and it also has a problem with unenforceable and undiscoverable interfaces. I'd be very interested to see a good solution for interfaces with multiple dispatch. Perhaps the popularity of Julia will lead to a good solution.
• Iwan-Zotow 5y ago
  Aesthetical issues: 1. Indexing from 1 2. Range is defined as [S...N], not [S...N), which means composing simple things is hard
• geokon 5y ago
  "map, filter and split are eager" why is this an issue? It's generally quite rare to leverage laziness (like with infinite lists and generators or something). And it causes errors to emerge away from the function callsite. I haven't thought about this issue much.. are there other issues with eagerness?
  • kinjba11 5y ago
    In a functional language `map` and `filter` are nice to chain together. For instance in Haskell: [1, 2, 3] & map (* 3) & filter (> 4) & sum This takes the array, maps it, filters and sums it in one go. If the array was say 1,000,000 elements and you used multiple `map`, `filter`, and other similar functions you'd soon run out memory (or use much more than needed) if they were eager and allocated a million-element array at each step as opposed to running through the array one time. The alternative is to use imperative style (for loops and such), but the functional style is often more declarative and easier to read and write (think SQL).
    • geokon 5y ago
      Okay I hadn't thought of that :) I guess even after all this time using these functions I'm still not entirely comfortable with the lazyness. I think a more explicit method in Clojure is to use `eduction` (if I'm not misunderstanding!) https://clojuredocs.org/clojure.core/eduction https://clojuredocs.org/clojure.core/eduction This allows you to be more explicit and say you want to chain these transducers together. The lazy variant makes it more difficult to reason about performance (and this being a "math" language - that seems a priority) b/c you need to read the tea leaves to see where the sequence gets realized. It really requires a gear shift in reasoning about your code b/c you need to think "okay, I'm writing this huge reduction, but I don't care about its size b/c it won't all get realized b/c I did this other thing later on". My gut feeling is it's a bit hard to keep track of everything :) But I'd be curious to hear an alternative.. I remember writing some math code and finding it was really challenging to get the lazy operations to chain together well. It was over a year ago, so the details are a bit hazy.. but I needed it to run down an array of numbers and do some operations and this lazy method ended up being incredibly slow. Unfortunately I don't remember the ultimate cause, but I'm guessing it thrashed the cache. It could also just be a limitation in Clojure internals when it comes to compositing lazy seq's - but again, I'm not quite sure.
• sivakon 5y ago
  One major gripe I have is the infamous "TTFP"(Time to First Plot) problem is more or less solved, these improvements haven't manifested in any other areas. Here is time to first read csv problem. I am on newest Intel Macbook Pro cat read_csv.jl using CSV using DataFrames df = CSV.read("hello.csv", DataFrame) println(df) cat hello.csv col1,col2,col3 1,2,3 4,5,6 time julia read_csv.jl julia read_csv.jl 10.48s user 0.37s system 102% cpu 10.609 total
• stellalo 5y ago
  Nice article! I agree with many of the points. If I know the Julia community well enough, criticism like this leads to constructive discussion and eventual improvement. One section I don’t get is that about the iteration protocol. It seems like the author is unhappy about the state being passed around, and the “implicit” trait methods one should implement. The former point appears to be a consequence of the lack-of-object-orientation (which is by design) while keeping object attributes vs state separate wherever possible (which is arguably a good thing in general); I don’t see how else this can be done in a language that only has structs and functions, honestly. Also, the fact that iterators can be themselves stateful is something that can happen in Python too (one is just “expected” not to hold state information inside Iterable types, but there’s no guarantee of that by just looking at the type). This seems hard to avoid. Maybe a trait could be used in Julia to provide this information about an iterator type? The latter is the same problem as “implicit traits” highlighted in other sections of the article, so not really anything specific to the iteration protocol.
  • jakobnissen 5y ago
    Indeed, the latter problem is the same as implicit traits. For the former, you can absolutely do it without having object-orientation or anything like that. Simply lower the iterator protocol to ``` itr = iterator(x) while (i = next(itr)) !== nothing # stuff end ``` Then `itr` can be mutable, even when `x` isn't. This is what most modern languages does, and it solves all the problems I mentioned, since the state is stored in the mutable iterator, while at the same time, `x` can be unaffected. Read more here: https://mikeinnes.github.io/2020/06/04/iterate.html https://mikeinnes.github.io/2020/06/04/iterate.html
    • stellalo 5y ago
      Well, to be honest I still don’t understand the difference: they seem to me like two different styles of structuring the same kind of logic. In fact, the linked list example in the referenced article from Mike Innes I don’t find very convincing, stated as it is: I would like to see that example concretely, to understand how the current Julia protocol would fall short there (“leak”). Same for the `zip` example in the OP here: I’m not sure the Python `itr = iterate(xs)` kind of protocol makes it any simpler. But I’m on holiday and cannot try it out right now—I owe you one here :-)
• oalae5niMiel7qu 5y ago
  >The experience was not that my program became more safe in the sense that I could ship it without sweat on my brow. No, it was that it just worked, and I could completely skip the entire debugging process that is core to the development experience of Julia, because I had gotten all the errors at compile time. >And this was for small scripts. I can only imagine the productivity boots that static analysis gives you for larger projects when you can safely refactor, because you know immediately if you do something wrong. Getting the program right the first time can't _possibly_ apply to programs larger than a small script. Type errors are only 2% of all bugs in large programs with companies behind them. When a big Rust program fails, then you have to contend with the fact that you can't even set a debugger checkpoint where an error occurs: https://github.com/rust-lang/rust/issues/54144 https://github.com/rust-lang/rust/issues/54144 Obviously people wouldn't be asking for that if the type and borrow checkers caught all the bugs at compile time. Not even close! And not only can't you debug errors, you also can't change the program at all. All you can do is terminate the program, generate a new one from modified source code, and run the new program. I hope that bug you were working on is simple enough that you can reproduce it with a unit test! LOL Rust problems! Now if only Rust had resumable exception handling, could be incrementally compiled, allowed function and type redefinitions at runtime, and wasn't designed as if we were still in the punched card era but with files instead of cards. That would give you far more of a productivity boost than a type checker.