5 ms·

Surprised by some of the negativity here! I've been extensively using julia for my graduate physics research and a lot of hobby programming for almost 3 years n

• systemvoltage 7y ago
  The negativity is warranted if you've ever run Julia or had to maintain it in production. Use it for your own Jupyter notebooks and personal analysis? Great! Need to debug some weird obscure error (which Julia does a poor job of reporting to the user, let alone which line in the stack trace) while having production pressure to get it up and running again? Julia is unquestionably, unarguably and utterly unsuitable. Do not use it in any production workflow, heed my advice as a maintainer of Julia repos and the tech debt that graduate students have created in our company (I am sorry but just stating the facts). You're still unsure? Let's switch sides, you maintain our company's production Julia repositories and I will take your spot to write beautiful code in a couple of hobby projects. I guarantee you, with absolute certainty, you will come down to your knees. You'll lose sleep. You'll hate management for letting this happen.
  • burntsushi 7y ago
    Do you have any links to other material where I could read up on some of the problems folks have faced using Julia in production? Thanks!
    • systemvoltage 7y ago
      I'd love to do a write up on the issues to guide the Julia team towards focusing on production and robustness, investing most of their efforts in a solid debugger support (Juno - totally incapable tool), and stopping all new features and "optimization" projects before getting these things sorted out. I am just too beaten, too bruised and injured to have the mental capacity to write an article. :( I am sorry if I am too harsh.
      • laughinghan 7y ago
        Harshness is not the problem, credibility is. I don't mean that you sound like you're lying, but that it's impossible for anyone to evaluate your assertion or how it applies to their use case. Anyone who reads your comment (including me) just files it away in their head as "somebody on HN made an unsubstantiated comment that Julia sucks in production in some vague way". If I were in the position of making or advising someone making the decision of whether or not to use Julia, an unsubstantiated, untestable assertion like that makes virtually no difference in my decision or advice. I hope it's clear this is not an insult, it's just that if I have no way to evaluate your assertion or its applicability, there's no way for me to incorporate it into my decision or advice. By contrast, if you could point to even one concrete example, I could file it away in my head as "someone on HN pointed out that Julia has X, Y, and Z problem". If I were in a position of making or advising a decision on Julia, I would be able to evaluate whether it applies the use case in question, whether it points to some deeper design problems with Julia, or whether I even think it's a problem at all.
        • systemvoltage 7y ago
          You're right, I see the problem. I'll do an article and post it with detailed analysis. It will take some time because repos are like Tolstoy's quote - "All good repos are alike, every broken repo is broken in its own way". There are issues ranging from startup time, upstream packages breaking, constantly changing ecosystem (given for a new language), quirky behaviors and certain problems related to HTTP package and LibPQ packages that I couldn't even debug. So, I rewrote them in Python. Until then - I concede, please take it with a grain of salt and use your own judgement.
          • dnautics 7y ago
            Thanks. I would enjoy reading this too.
          • tomkwong 7y ago
            It's quite possible that certain area of the Julia ecosystem are not not mature enough. The situation continues to improve quickly as far as I can tell. Rather than rewriting code in Python, we took an approach of using PyCall. It turns out that the overhead of calling Python is very small... at least not to an extent that I had to worry about. For my production system, we used this strategy to access Oracle and Apache Kafka. The code still looks very clean as the calls to the Python packages are just like normal Julia function calls.
  • dnautics 7y ago
    I've thought about this problem, quite a bit, and I think that productionizing Julia would require at least a few additional things above your criticisms about error codes. 1. Get rid of global dependencies. Store all of your dependencies in a project local deps directory with a project lock file. 2. Opinionated file system structure. Maybe you don't need this in one-off scripts but definitely for some sort of "project" layout 3. Force all packages in the package manager to obey these constraint. The ship may have already sailed on 3), sadly.
    • tomkwong 7y ago
      What's the issue about global dependencies that you're trying to solve? For my production system, we manage version changes by way of Manifest.toml so that we don't get any surprises. I agree that an opinionated file system structure is nice. Julia already wants you to put source code inside `src` but then underneath you are free to do whatever you want. We generally manage this via coding convention since people can never agree on a standard.
    • oxinabox 7y ago
      You are describing every Julia package. These are the rules of the package manager. 1. Every package must declare it's dependencies (and to register must declare compat bounds on them, and good devs do that always anyway) 2. Every package must have Project.toml in the base directory, source code goesin `src`, test code goes in `test`, documentation goes in. `docs` (and more structure there if using Documenter.jl) What more could one want? If a package needs more folder structure within `src` then it should be multiple packages. 3. These are the rules, done. Further you are describing every sensibly done julia application. Basically no seriously Julia developer uses the global environment for anything but dev-tools, like BenchmarkTools or ProfileView. Certainly one does not depend on the content of them for any reused code -- that is what Project.toml and Manifest.toml is for.
      • dnautics 7y ago
        > every sensibly done julia application. Can you point me to some documentation on those best practices. My girlfriend - architectural acoustics consultant - is working on a project in Julia and is having a hell of a time managing dependency shift underneath her program (also she barely knows how to use git).
  • tomkwong 7y ago
    I'm empathetic about your experience but I have been using Julia in production (financial services use case) for over 2 years and it's absolutely working great for us - developer productivity, performance, etc. We are not stopping and writing more Julia code at the moment. To be honest, you can incur technical debt with any language. Software must be designed and maintained properly if you need a long-lasting solution. Research projects are also different than production code. Proper training and involvement with the developer community could help a lot. My suggestion to everyone reading this thread - if you are new to Julia programming and need to work on a production project, do talk to Julia Computing folks. Their consultants can lead you to the right track. In addition, join the Julia Slack and Discourse community. You can almost get instant answers to anything you ask there. Lastly, here's a selfish plug to my book: Hands-on Design Patterns and Best Practices with Julia https://www.amazon.com/Hands-Design-Patterns-Julia-comprehensive/dp/183864881X https://www.amazon.com/Hands-Design-Patterns-Julia-comprehen...
    • systemvoltage 7y ago
      Thank you, I can definitively say that some of the problems arise from poor programming practices than the language itself. But some languages are better at preventing people from writing bad code - see Rust and Go (impeccable packagement, project structure, memory management, RAII, etc) and Python (strict syntax PEP8/black).
      • tomkwong 7y ago
        That's fair... although I generally found bad code that are not just bad syntax but rather bad design. Bad syntax or even project structure is easy to fix. Bad design is not. BTW, this kind of discussion is healthy. The Julia core developers are already taking notes and I'm certain they will continue improving the language or ecosystem.
      • eigenspace 7y ago
        Yes, as a grad student I'm not working on large production code-bases, but I am in contact with quite a few people who are working on large julia production code-bases and my impression is that it has worked well for most of them, hence my dismay at the sorts of comments I see here. In my experience, one of the biggest problems is that people approach julia as "python or matlab but faster", which is going to be a recipe for failure. Julia is a very different language with its own idioms and very different style and if people try to just write Python with Julia syntax I can see how problems arise. To be clear, I do definitely believe there are some big pain points in julia at scale, but depending on the usage domain and needs, my impression is that julia is an appropriate tool for at least some niches.
  • ddragon 7y ago
    I wonder if the issue here is the graduate students part more than the Julia part (although obviously the tooling needs work). Maintainability is an art that comes with experience (or proper guidance), and I had to deal in college with a ton of really scary C code from other students (like all the program in the main function with 2000 lines scary). And in the same way in a professional scenario (in a polyglot company with mostly young developers) we had to deal with unmaintainable code in multiple languages, from Python to Scala. Even short scripts that were extremely tricky to reproduce in a more sane approach sometimes. If my job was just that then I would certainly be heavily burned by it and I wouldn't want to touch the language. But the better solution would be to instead learn the language properly and create a serious guideline that everyone has to follow. If people can't use macros properly (or any of Julia's most powerful features), then every PR with a macro requires a very convincing justification and a review from a more experienced developer. If you're adding a new functionality then you have to be sure there isn't cyclic dependencies with other functionalities, and possibly making it a small independent library instead. And more importantly code is not done until it's properly documented, reviewed and unit tested.
    • qchris 7y ago
      I think that this is very insightful. At least in my experience, maintainable code isn't something that naturally arises. It comes from experience and interest in writing code that will need to be maintained. A lot of students (or academic researchers in general) don't really pay much attention to these things, which is totally understandable given their overall incentive structure. I'm working with a computer science undergrad at the moment who put multiple full versions of his code to into the Github repository. He didn't use branches, he didn't use pull requests for updates, the diff log is just a new file that is the previous one with changes, all of which sort of defeats the purpose of using source control like git. But, as you say, it's something that takes some guidance and experience to understand. I do find that it makes me lean towards tools that encourage good coding practices by design and which make untying knots easier, which it sounds like Julia might not make super easy (I don't have any production experience with it). Obviously, the practices you laid out definitely help to make sure things don't slip through the cracks, though!
  • chappi42 7y ago
    You probably cannot say which company this is? Or what production workflow? Or what you are using Julia for?
    • systemvoltage 7y ago
      Unfortunately not because of obvious reasons, but we are a Biotech AI startup, about 180+ people.
      • chappi42 7y ago
        Thank you. And: good luck!