6 ms·

I use both R and Python in my work but when we move our models to production it's not real time, just a batch execution like once in a day. I'd like to hear fro

by amrrs 8y ago

I use both R and Python in my work but when we move our models to production it's not real time, just a batch execution like once in a day. I'd like to hear from anyone who uses Julia in their actual job/work. Is it worth learning Julia, hoping to use it in work some day?

gotfork 8y ago
I used Julia in my day-to-day work at my previous job (Intel) and in my new position (Rigetti). There's quite a few companies that are using it in production now.
- dejv 8y ago
  Glad to hear more about how are you deploying Julia in production.
- graphene 8y ago
  That's really cool! I checked Rigetti's github but there seem to be no open source Julia projects there (although, somewhat surprisingly, lots of Common Lisp!). Are you free to say what Julia is being used for at Rigetti (and Intel), and whether there are any plans to release things as open source in the future?
  - gotfork 8y ago
    At Intel I developed a few product modeling tools in Julia, including one that's now used in the introduction of every new product. The only real annoyances that I ran into were related to dependency management in Pkg2, but I think Pkg3 should solve all of that. I'm quite new at Rigetti, but I would suggest keeping an eye on our GitHub page.
ChrisFoster 8y ago
I've been using it in production since version 0.5. It's a joy for creating high performance numerical code without having to leave the comfort of a productive and interactive scientific computing environment. Perhaps surprisingly it's also a great glue language, where I think it does a better job of replacing shell scripts than python.
- deleted 8y ago
  [deleted]
- ivirshup 8y ago
  I do love Julia, but start up and import time can make scripting a little rough. Hopefully PackageCompiler.jl will solve that soon.
  - staticfloat 8y ago
    If you haven't yet, you should check out a build of 1.0. The start-up time is _greatly_ improved. Basic startup time is twice as fast as ipython for me now: $ time ipython -c '1+1' Out[1]: 2 real 0m0.493s user 0m0.401s sys 0m0.063s $ time julia -e '1+1' real 0m0.222s user 0m0.115s sys 0m0.101s Yes, python itself is still much faster. I compare against ipython just because that used to be my go-to interactive shell for technical computing.
    - ivirshup 8y ago
      That's pretty cool! I never had to much of an issue with the startup time for interactive computing though. It mainly has come up as an issue when I was writing short scripts that could be run a couple dozen times in serial. The real question is how long does `using Plots` take in v1.0?
ChrisRackauckas 8y ago
We are using Julia to develop a clinical trial simulator for drug dosage prediction and personalized medicine applications as a joint MIT, University of Maryland Baltimore, and JuliaComputing project. It will be open sourced in January, and at the same time we will be starting a clinical trial to test the methods in a real-world setting.
- elcritch 8y ago
  Definitely post a link! Are you doing stochastic modeling, and if so how do you structure the input data? I’d love to use similar techniques for other data modeling.
  - ChrisRackauckas 8y ago
    Here's a link to the talk: https://www.youtube.com/watch?v=KQ4Vtsd9XNw https://www.youtube.com/watch?v=KQ4Vtsd9XNw . We are using stochastic models at two levels: at the population level using nonlinear mixed effects models, and at the individual level using discrete stochastic (Gillespie) models and stochastic differential equations via DifferentialEquations.jl. The input data has a standard structure which we read in with tools like JuliaDB, and for output the DiffEq solutions are table structures which can directly output to JuliaDB, DataFrames, or csvs.
rogerluo 8y ago
I think sometimes people are distracted by Julia's performance, according to https://github.com/JuliaLang/Microbenchmarks https://github.com/JuliaLang/Microbenchmarks Julia is not the fastest language/compiler (maybe LuaJIT is). Julia is not only good for its performance, but also its multiple dispatch, its type system and more. Because of those features we have https://github.com/JuliaGPU/CUDAnative.jl https://github.com/JuliaGPU/CUDAnative.jl, more elegant package interface, like https://github.com/JuliaOpt/JuMP.jl https://github.com/JuliaOpt/JuMP.jl and there is more. I still believe there is no silver bullet, if you want something Python then you get another Python. But think about this: is the full Python's dynamic feature really what we need in works like simulation, HPC, etc.? Probably no. I think Julia is kinda of balance for the related field (like for scientific computing). Just like in the old days, people start using FORTRAN, MATLAB, Lisp as the most advanced tools at that time. We start using Julia now.
- Q6T46nT668w6i3m 8y ago
  In addition to the type system, what are some other features you believe are too dynamic (for reasonable performance)?
  - rogerluo 8y ago
    Yes probably just for reasonable performance, and I don't actually need to use Python's rich object model. Mostly, I just use Python to call C++ for easier interface (and C++ is for performance). And before Python 3 since there is no type annotation, refactoring is quite annoying. I'm not an PL expert, so I would like to ref a previous discuss in discourse, which is more detailed https://discourse.julialang.org/t/julia-motivation-why-werent-numpy-scipy-numba-good-enough/2236/5 https://discourse.julialang.org/t/julia-motivation-why-weren... I was just trying to say that if you want to develop something with reasonable performance, elegant interface in a short time. Probably under this situation, Julia is more suitable.
- celrod 8y ago
  Those benchmarks draw a lot of hate, because anyone coming from language X will get offended at how unoptimized code in language X is. Also true if X == Julia. For example, they never turn off boundschecks, which disables vectorization. The point is mostly to (a) show the difference between fast languages compiled to efficient assembly and (b) represent code someone new to a language may bang out to get something done (while avoiding performance pitfalls) and avoid the benchmark game. That said, I agree. I try (poorly) not to advertise speed, because people coming from languages like R will rarely fail to write type unstable code that is slow, observe JIT compilations that make Julia a little laggy, and then come away disappointed. Things like the type system and meta-programming shown off in your examples are amazing, and also not something you can reproduce in other languages by adding binary dependencies.
fundamental 8y ago
I use it pretty much every work day to explore ML/DSP ideas and to experiment around before formalizing an algorithm. It's fast to run (which is important to be able to go through enough data to make conclusions), the code produced is reasonably readable, there are a decent number of libs, and it's fairly quick to write. I'd say if you're looking for something like a MATLAB/R/Python/etc replacement Julia is a good option, though it has been a bumpy ride at times while the language has been evolving. I think I started using Julia around 2013 or so after getting fed up with the slower speed of octave, so I'm guessing that things should be a fair bit smoother from this point on if they've officially tagged 1.0.
currymj 8y ago
R and Python are still (currently) nicer for stats and ML, although there's a lot of not-yet-mature Julia libraries that are extremely cool (Turing.jl for Bayesian inference in particular, and systems for autodiff). I also think Plots.jl is the best-designed plotting system, although if you are a ggplot true believer you might not agree. But for classical scientific computing, of the sort typically done by real engineers wearing coke-bottle glasses, using MATLAB and expensive toolboxes, it can't be beat. It's really easy to work with matrices and make your code super-fast. And it steals all the good parts of MATLAB syntax but with a good, modern, sensibly designed programming language. Also, DifferentialEquations.jl is undeniably the best available piece of software for solving differential equations, in terms of both performance and ease-of-use. It feels like using alien technology or something.
chenning 8y ago
In my opinion, Julia has everything you need to be productive in a scientific computing or academic arena right now. In fact, if you're in an atmosphere where there's a lot of chaining together various command line tools Julia actually does subprocesses and pipes quite well. As a more general point though, I work in an academic environment and I see tons of languages whizzing by. I routinely see C, C++, Python, R, Matlab, csh, bash, and the list just goes on and on. I've even seen someone write a command line tool in PHP that had no justifiable reason for being written in PHP. But it works, so whatever. If it was written in Julia, maybe it would be uncommon, but also slightly more refreshing. So, really, why not Julia?

by amrrs 8y ago

gotfork 8y ago

I used Julia in my day-to-day work at my previous job (Intel) and in my new position (Rigetti). There's quite a few companies that are using it in production now.

dejv 8y ago
Glad to hear more about how are you deploying Julia in production.
graphene 8y ago
That's really cool! I checked Rigetti's github but there seem to be no open source Julia projects there (although, somewhat surprisingly, lots of Common Lisp!). Are you free to say what Julia is being used for at Rigetti (and Intel), and whether there are any plans to release things as open source in the future?
- gotfork 8y ago
  At Intel I developed a few product modeling tools in Julia, including one that's now used in the introduction of every new product. The only real annoyances that I ran into were related to dependency management in Pkg2, but I think Pkg3 should solve all of that. I'm quite new at Rigetti, but I would suggest keeping an eye on our GitHub page.

ChrisFoster 8y ago

I've been using it in production since version 0.5. It's a joy for creating high performance numerical code without having to leave the comfort of a productive and interactive scientific computing environment. Perhaps surprisingly it's also a great glue language, where I think it does a better job of replacing shell scripts than python.

deleted 8y ago
[deleted]
ivirshup 8y ago
I do love Julia, but start up and import time can make scripting a little rough. Hopefully PackageCompiler.jl will solve that soon.
- staticfloat 8y ago
  If you haven't yet, you should check out a build of 1.0. The start-up time is _greatly_ improved. Basic startup time is twice as fast as ipython for me now: $ time ipython -c '1+1' Out[1]: 2 real 0m0.493s user 0m0.401s sys 0m0.063s $ time julia -e '1+1' real 0m0.222s user 0m0.115s sys 0m0.101s Yes, python itself is still much faster. I compare against ipython just because that used to be my go-to interactive shell for technical computing.
  - ivirshup 8y ago
    That's pretty cool! I never had to much of an issue with the startup time for interactive computing though. It mainly has come up as an issue when I was writing short scripts that could be run a couple dozen times in serial. The real question is how long does `using Plots` take in v1.0?

ChrisRackauckas 8y ago

We are using Julia to develop a clinical trial simulator for drug dosage prediction and personalized medicine applications as a joint MIT, University of Maryland Baltimore, and JuliaComputing project. It will be open sourced in January, and at the same time we will be starting a clinical trial to test the methods in a real-world setting.

elcritch 8y ago
Definitely post a link! Are you doing stochastic modeling, and if so how do you structure the input data? I’d love to use similar techniques for other data modeling.
- ChrisRackauckas 8y ago
  Here's a link to the talk: https://www.youtube.com/watch?v=KQ4Vtsd9XNw https://www.youtube.com/watch?v=KQ4Vtsd9XNw . We are using stochastic models at two levels: at the population level using nonlinear mixed effects models, and at the individual level using discrete stochastic (Gillespie) models and stochastic differential equations via DifferentialEquations.jl. The input data has a standard structure which we read in with tools like JuliaDB, and for output the DiffEq solutions are table structures which can directly output to JuliaDB, DataFrames, or csvs.

rogerluo 8y ago

I think sometimes people are distracted by Julia's performance, according to https://github.com/JuliaLang/Microbenchmarks https://github.com/JuliaLang/Microbenchmarks Julia is not the fastest language/compiler (maybe LuaJIT is). Julia is not only good for its performance, but also its multiple dispatch, its type system and more. Because of those features we have https://github.com/JuliaGPU/CUDAnative.jl https://github.com/JuliaGPU/CUDAnative.jl, more elegant package interface, like https://github.com/JuliaOpt/JuMP.jl https://github.com/JuliaOpt/JuMP.jl and there is more. I still believe there is no silver bullet, if you want something Python then you get another Python. But think about this: is the full Python's dynamic feature really what we need in works like simulation, HPC, etc.? Probably no. I think Julia is kinda of balance for the related field (like for scientific computing). Just like in the old days, people start using FORTRAN, MATLAB, Lisp as the most advanced tools at that time. We start using Julia now.

Q6T46nT668w6i3m 8y ago
In addition to the type system, what are some other features you believe are too dynamic (for reasonable performance)?
- rogerluo 8y ago
  Yes probably just for reasonable performance, and I don't actually need to use Python's rich object model. Mostly, I just use Python to call C++ for easier interface (and C++ is for performance). And before Python 3 since there is no type annotation, refactoring is quite annoying. I'm not an PL expert, so I would like to ref a previous discuss in discourse, which is more detailed https://discourse.julialang.org/t/julia-motivation-why-werent-numpy-scipy-numba-good-enough/2236/5 https://discourse.julialang.org/t/julia-motivation-why-weren... I was just trying to say that if you want to develop something with reasonable performance, elegant interface in a short time. Probably under this situation, Julia is more suitable.
celrod 8y ago
Those benchmarks draw a lot of hate, because anyone coming from language X will get offended at how unoptimized code in language X is. Also true if X == Julia. For example, they never turn off boundschecks, which disables vectorization. The point is mostly to (a) show the difference between fast languages compiled to efficient assembly and (b) represent code someone new to a language may bang out to get something done (while avoiding performance pitfalls) and avoid the benchmark game. That said, I agree. I try (poorly) not to advertise speed, because people coming from languages like R will rarely fail to write type unstable code that is slow, observe JIT compilations that make Julia a little laggy, and then come away disappointed. Things like the type system and meta-programming shown off in your examples are amazing, and also not something you can reproduce in other languages by adding binary dependencies.

fundamental 8y ago

I use it pretty much every work day to explore ML/DSP ideas and to experiment around before formalizing an algorithm. It's fast to run (which is important to be able to go through enough data to make conclusions), the code produced is reasonably readable, there are a decent number of libs, and it's fairly quick to write. I'd say if you're looking for something like a MATLAB/R/Python/etc replacement Julia is a good option, though it has been a bumpy ride at times while the language has been evolving. I think I started using Julia around 2013 or so after getting fed up with the slower speed of octave, so I'm guessing that things should be a fair bit smoother from this point on if they've officially tagged 1.0.

currymj 8y ago

R and Python are still (currently) nicer for stats and ML, although there's a lot of not-yet-mature Julia libraries that are extremely cool (Turing.jl for Bayesian inference in particular, and systems for autodiff). I also think Plots.jl is the best-designed plotting system, although if you are a ggplot true believer you might not agree. But for classical scientific computing, of the sort typically done by real engineers wearing coke-bottle glasses, using MATLAB and expensive toolboxes, it can't be beat. It's really easy to work with matrices and make your code super-fast. And it steals all the good parts of MATLAB syntax but with a good, modern, sensibly designed programming language. Also, DifferentialEquations.jl is undeniably the best available piece of software for solving differential equations, in terms of both performance and ease-of-use. It feels like using alien technology or something.

chenning 8y ago

In my opinion, Julia has everything you need to be productive in a scientific computing or academic arena right now. In fact, if you're in an atmosphere where there's a lot of chaining together various command line tools Julia actually does subprocesses and pipes quite well. As a more general point though, I work in an academic environment and I see tons of languages whizzing by. I routinely see C, C++, Python, R, Matlab, csh, bash, and the list just goes on and on. I've even seen someone write a command line tool in PHP that had no justifiable reason for being written in PHP. But it works, so whatever. If it was written in Julia, maybe it would be uncommon, but also slightly more refreshing. So, really, why not Julia?