5 ms·

I used Genie with PackageCompiler on AWS to host build a wedding website for some friends (features included personalized invites + custom photo uploads with fi

I used Genie with PackageCompiler on AWS to host build a wedding website for some friends (features included personalized invites + custom photo uploads with filters directly to the bride and groom + live mini games during the wedding). The awesome part about this was that image processing in Julia was a joy to work with. And packaging and environment issues were non existent. I've spent A LOT of time dealing with those in Python, and having things almost always just work was really nice. I had a few issues though, 0) I could not use the free tier in heroku because Julia + Genie would take too long to start up, and Heroku's free tier will kill the VM if you don't serve a response in 30 seconds. I was able to get it to work with PackageCompiler but even that was close when the VM booted up for the first time on Heroku. At the end I decided to use AWS instead. 1) None of my friends who wanted to contribute wanted to use Julia (they said if they were learning something new they'd prefer Go, Ruby or even just using C++). 2) Library support for Genie was just okay. Implementing Auth + Database support + dealing with cookies, sessions etc took waaayyy too long, when I just wanted to implement features for the website. I wanted to parse HTML from a custom calendar page and tried to use Gumbo, and it was slow and painful to work with. In the past when I did the same thing with Python, things just worked. Beautiful Soup 4 was SOOO much more productive for me than trying to do the same thing in Gumbo. 3) Development was SLOW. Julia + Genie takes forever to reload, even with live reload. And if you change one line in the HTML that you are about to generate, and you'll have to wait 30 or more seconds to see the update in a local developent environment. This is PAINFUL. Once the size of the application grows or you add dependencies, it takes longer. The error messages aren't too good either and don't get me started on the usability of language server in vim/neovim. The one other friend who actually downloaded Julia and installed every thing in order to contribute to the project instantly gave up on making any contributions just because of the development experience. And, when I deployed it to AWS, even after using PackageCompiler I wasn't getting instant server responses (I'm assuming I did something wrong here though). The final binaries that I was deploying to AWS were also huge (hundreds of megabytes). On reflection, if I had implemented the same project in Python, I would have completed this maybe 10 times faster (based on past experience), and run time would be been as fast too (except for the image processing piece). Using Python + Flask and using a subprocess to call Julia would have been a better approach to solving this problem, instead of what I choose to do. I had stretch goals for this project (more mini games) that I wasn't able to even attempt making, primarily because I was dealing with all the pain points of developing in Julia (I guess it was also my fault for trying to get all this done in a couple of weeks of my spare time). I really like Julia as a language, but using it for anything other than scientific computing (more specifically using it as a general purpose programming language) seems to me as a recipe for pain at the moment. I would not recommend deploying production applications in Julia right now unless you HAVE to use it (because of some library that's only available in Julia) or your development team is willing to put in work to write libraries from scratch + deal with slow start up. Maybe in a few years when tree shaking is implemented and more latency issues are resolved and language server gets significantly better, it'll be a competitive choice. Until then I'll be curiously watching from a distance :)

ModernMech 5y ago
Very interesting post, thanks for sharing. Do you have a link to your project code? Your experience seems to be in line with this post I read recently: https://viralinstruction.com/posts/badjulia/ https://viralinstruction.com/posts/badjulia/ It's written by a Julia fan, so it's a very lucid and measured take on the not so great side of Julia.
- blindseer 5y ago
  I don't have a link to the project unfortunately, it was really haphazardly put together (personal access tokens, other people's identifying information etc, have been committed to the repo), and I don't have the time to clean it up. But everything I used in the project is publicly available, and you'll just have to spend some time searching for it in various docs. I completely agree with Jakob's take in the piece you've shared btw. Personally, I don't think Jakob goes far enough. I've used Julia professionally too, and worked with large code bases (a 10000 line Julia monorepo project that just keeps growing), and the lack of interfaces, the lack static type checking, the lack of error types, etc would all be show stoppers for me to consider Julia at the start of our project if I knew then what I know now. I've even tried to run static type checkers like JET.jl on our code base and it just keeps spinning or failing. Unfortunately it is a sunk cost for us at this point and we'll just have to use Julia, and deal with these pain points, and hope and pray the language gets better over time. I can see some improvements such as minor latency improvements happening over time. But (in my opinion) I don't think language features like interfaces are likely to happen until Julia 2.0. In 2018, the core team had laid out priorities in a discourse post, and marked those as done last year (i.e. 2021). https://discourse.julialang.org/t/compiler-work-priorities/17623/122 https://discourse.julialang.org/t/compiler-work-priorities/1... In my opinion, there's a LOT more language features required to help tooling in the ecosystem improve. The fact that they marked this as done (which I just found out while typing this comment) and that there's no follow up priorities outline post (as far as I can find) is a little troubling. I think a lot of people that pick Julia as their first language really like it, but I believe it is mainly because they haven't experienced something better to compare it to. I'm not saying everyone is this way, and obviously everyone values different things in their programming language / development environment. I do think if you don't know what's out there you tend to just accept that something you are doing is the norm. And I think a lot of people that come to Julia from languages like Python, R, C++ see value in Julia for solving package development issues (compared to python), better language design (compared to R), performance without verbosity (compared to C++). And I think they weight those issues more heavily than I do. I've dealt with Python packaging issues, and still do on a daily basis but I now know my way around whatever comes up. R's syntax might be kludgy but I'm damn productive if I'm using ggplot. I'd rather use Python for general purpose work and R for plotting, than having to deal with Julia. C++ though I'd never want to touch again, and would just use rust instead. If rust wasn't an option, I think I'd end up using Julia (even though I complain about it so much). This is just were my priorities / skill level lies, and I can completely see how it would be different for others. I'm much more productive plotting in Python and R than I am in Julia, purely because of compile times, even though Julia has a much better designed plotting package than Python (R's plotting ecosystem is rather convenient even if it is counter intuitive some times). When you've used Rust, Go, Nim etc, developing in Julia feels so ... antiquated. Just as one example, when using Rust, you can write a unit test function right next to the function you are testing. And you can run just that unit test function from a terminal (e.g. `cargo test -- test_just_this_one_function`). And because you can write a quick little test function pretty much anywhere in your code and run it in the terminal, you don't really feel the need for a REPL all that much. When I'm not writing tests, I would say I'm as productive in Rust as I am in Julia (mainly because of awesome language server features in Rust). But the moment I need to write tests in Julia ... it's a big oof y'all. In Julia there's NO official way to run just one test. You HAVE to run all of them. And the tests can be scattered in various different tests files in the test folder. Try contributing to the core Julia language itself, and trying running their test suite. You'll have to go get coffee multiple times just to make one single patch. For projects that I'm working on, I literally have a Julia REPL open with Revise, where I manually include a specific test file after commenting out unnecessarily global state, and then call a the function I want to test that way. It's madness, and I _think_ this is what everyone does (?). If people are saying they prefer this, I can't help but liken it to Stockholm syndrome or something to that effect.

ModernMech 5y ago

Very interesting post, thanks for sharing. Do you have a link to your project code? Your experience seems to be in line with this post I read recently: https://viralinstruction.com/posts/badjulia/ https://viralinstruction.com/posts/badjulia/ It's written by a Julia fan, so it's a very lucid and measured take on the not so great side of Julia.

blindseer 5y ago
I don't have a link to the project unfortunately, it was really haphazardly put together (personal access tokens, other people's identifying information etc, have been committed to the repo), and I don't have the time to clean it up. But everything I used in the project is publicly available, and you'll just have to spend some time searching for it in various docs. I completely agree with Jakob's take in the piece you've shared btw. Personally, I don't think Jakob goes far enough. I've used Julia professionally too, and worked with large code bases (a 10000 line Julia monorepo project that just keeps growing), and the lack of interfaces, the lack static type checking, the lack of error types, etc would all be show stoppers for me to consider Julia at the start of our project if I knew then what I know now. I've even tried to run static type checkers like JET.jl on our code base and it just keeps spinning or failing. Unfortunately it is a sunk cost for us at this point and we'll just have to use Julia, and deal with these pain points, and hope and pray the language gets better over time. I can see some improvements such as minor latency improvements happening over time. But (in my opinion) I don't think language features like interfaces are likely to happen until Julia 2.0. In 2018, the core team had laid out priorities in a discourse post, and marked those as done last year (i.e. 2021). https://discourse.julialang.org/t/compiler-work-priorities/17623/122 https://discourse.julialang.org/t/compiler-work-priorities/1... In my opinion, there's a LOT more language features required to help tooling in the ecosystem improve. The fact that they marked this as done (which I just found out while typing this comment) and that there's no follow up priorities outline post (as far as I can find) is a little troubling. I think a lot of people that pick Julia as their first language really like it, but I believe it is mainly because they haven't experienced something better to compare it to. I'm not saying everyone is this way, and obviously everyone values different things in their programming language / development environment. I do think if you don't know what's out there you tend to just accept that something you are doing is the norm. And I think a lot of people that come to Julia from languages like Python, R, C++ see value in Julia for solving package development issues (compared to python), better language design (compared to R), performance without verbosity (compared to C++). And I think they weight those issues more heavily than I do. I've dealt with Python packaging issues, and still do on a daily basis but I now know my way around whatever comes up. R's syntax might be kludgy but I'm damn productive if I'm using ggplot. I'd rather use Python for general purpose work and R for plotting, than having to deal with Julia. C++ though I'd never want to touch again, and would just use rust instead. If rust wasn't an option, I think I'd end up using Julia (even though I complain about it so much). This is just were my priorities / skill level lies, and I can completely see how it would be different for others. I'm much more productive plotting in Python and R than I am in Julia, purely because of compile times, even though Julia has a much better designed plotting package than Python (R's plotting ecosystem is rather convenient even if it is counter intuitive some times). When you've used Rust, Go, Nim etc, developing in Julia feels so ... antiquated. Just as one example, when using Rust, you can write a unit test function right next to the function you are testing. And you can run just that unit test function from a terminal (e.g. `cargo test -- test_just_this_one_function`). And because you can write a quick little test function pretty much anywhere in your code and run it in the terminal, you don't really feel the need for a REPL all that much. When I'm not writing tests, I would say I'm as productive in Rust as I am in Julia (mainly because of awesome language server features in Rust). But the moment I need to write tests in Julia ... it's a big oof y'all. In Julia there's NO official way to run just one test. You HAVE to run all of them. And the tests can be scattered in various different tests files in the test folder. Try contributing to the core Julia language itself, and trying running their test suite. You'll have to go get coffee multiple times just to make one single patch. For projects that I'm working on, I literally have a Julia REPL open with Revise, where I manually include a specific test file after commenting out unnecessarily global state, and then call a the function I want to test that way. It's madness, and I _think_ this is what everyone does (?). If people are saying they prefer this, I can't help but liken it to Stockholm syndrome or something to that effect.