6 ms·

My Experience at JuliaCon

114 points by ViralBShah 12y ago

srean 12y ago
Would the video, slides (and corresponding source repos) be uploaded sometime ? Would love to see those. Reading between the lines, devectorize.jl and the vectorization framework will make for a heady mix. I am glad that Julia is challenging the traditional mantra of "vectorize (as in the R/Numpy/Matlab sense) the loops". It is heart warming to see that the language designers get it that this style has inherent speed limitations (time is spent filling and copying temporary vectors). If any Julia core developer is seeing this, just a shout out, (i) to say you are doing awesome work, and (ii) @ macros available in the standard library are not documented well. It would be good to have a page that describes what is available now. EDIT: Just clarifying my point. Macros as a feature are quite well described in the docs. What I miss is an index of macros that come built in with the standard library. They would help both with pedagogy and with putting them to actual use, for example removing bounds checks. If they are a little more discoverable than they are now that would be very helpful to a newcomer. For a new'ish language the documentation is otherwise quite admirable. @ViralBShah I filed an issue.
- ViralBShah 12y ago
  The videos are being edited and will be available shortly. We'll post them on juliacon.org and julialang.org. Does the Metaprogramming section of the manual not adequately describe macros? It would be great if you submit a pull request or an issue describing what is missing.
- ViralBShah 12y ago
  An index of macros in the standard library is a good idea. Would you mind filing an issue?
  - StefanKarpinski 12y ago
    Macros are documented along with everything else in the standard library: http://docs.julialang.org/en/latest/stdlib/base/ http://docs.julialang.org/en/latest/stdlib/base/ The stdlib docs are organized by subject matter. What would the benefit of having a separate index of macros be? If one wants a list of exported macros, that's just a matter of doing grep @ base/exports.jl in a Julia repo.
- jarvist 12y ago
  Slides are on GitHub: https://github.com/JuliaCon/presentations https://github.com/JuliaCon/presentations I believe the video is heading for their YouTube channel: https://www.youtube.com/user/JuliaLanguage/featured https://www.youtube.com/user/JuliaLanguage/featured
- scott_s 12y ago
  I had to look up Devectorize.jl: https://github.com/lindahua/Devectorize.jl https://github.com/lindahua/Devectorize.jl The point, I think, is not that vectorizing is slow. It's that you don't want to be myopic when you do it. Your point about time spent filling and copying vectors is independent of what Devectorize is trying to do. Yes, when you vectorize code, you need to have a prologue and an epilogue which gets the data in the correct format for the SIMD operations. This prologue and epilogue has a cost, and if it's larger than the gain from the SIMD operations, then it's not worth vectorizing. (This can happen if, say, you only have 4 elements to vectorize.) However, we have to do this cost-analysis all the time. Figuring out when it's beneficial to vectorize code has been around as long as we've vectorized code. Devectorize.jl is not about that problem. Rather, it's about when you have a chain of expressions, and the operators in those expressions have implied loops. The naive thing to do is to myopically execute each one of those loops, creating and passing around temporary vectors. The Devectorize framework is given an entire expression, and is able to analyze where the implied loops are, and figures out how to express that computation in a single loop. We can tell these two concepts are independent because the ideal situation is to first use Devectorize on an expression, and then vectorize the result! For the record, the approach taken by Devectorize.jl is similar to the problem that expression templates (http://en.wikipedia.org/wiki/Expression_templates http://en.wikipedia.org/wiki/Expression_templates) in C++ try to solve. Through template trickery and operator overloading, we can put off evaluating an expression, avoiding the temporaries and unnecessary memory traversal that a naive execution implies. The Boost library uBLAS (micro Base Linear Algebra Subprograms, http://www.boost.org/doc/libs/1_55_0/libs/numeric/ublas/doc/index.htm http://www.boost.org/doc/libs/1_55_0/libs/numeric/ublas/doc/...) uses this technique.
  - srean 12y ago
    Not sure that I understand the point/tone of your comment, all that you say is correct and I am not unaware of any of those points nor do I contradict them in mine, but it seems that you are offering it as a correction. From your comment it seems that you are aware of the two different meanings of "vectorization" but that you are choosing the wrong one to setup a strawman. So I am a bit puzzled. In any case let me offer some clarifications: The canonical example of ET and the one that started it all is Blitz++ (unlike its modern descendants it actually handles full n-dimensional tensors). More recent solutions are Blaze, Eigen and Armadillo. I like these over uBLAS (more verbose and not as performant). Blaze is typically the fastest (in this class) as long as you are dealing with 2D arrays and linear algebraic operations. Since Julia is homoiconic and has hygienic macros, one does not have to resort to such syntactically complex ways of doing metaprogramming such as ETs in C++. ET's are great to use, not very pleasant to write the machinery for, although things like Boost::spirit and fusion make it more tolerable. Heaven forbid that you have to understand the error messages though. Now, to another point, looping is so pathetically slow in R, Numpy and Matlab (in that order) that the traditional solution offered in these languages is to vectorize (in the Matlab/numpy/R sense not in the SIMD sense) the loops into vector-expressions. Often this needs the help of extra prefilled vectors/matrices, (some languages do it smartly with 'broadcasting', some where late to pick it up, looking at you Matlab). Even with broadcasting tricks, the extra stride indirection in the iteration slows down the code from how fast it could have been. That is but one aspect of it, the other is chaining several binary and unary operations together. This also creates temporaries and unnecessary loops (unless you use solutions of somewhat limited scope like numexpr, which BTW is lovely when it is applicable, it also does thread level parallelization as well as SIMD if you can afford to link with intel MKL). In Julia there is no need to avoid loops because they are inherently fast. In fact they are faster than the vectorized expressions, because of inefficiencies mentioned. However, loops are a lot more verbose than expressions. So this is where devectorize comes in. It transforms those expressions into loops (much like ETs) which can then be JITed to obtain superior performance. Now with SIMD vectorization thrown in on top, one can gain even more, because now one can benefit further from the instruction level parallelism. So yes you devectorize and then vectorize (and you already know this) and the term "vectorization" refer to two different concepts named vectorization , the clash in terminology is rather unfortunate. EDIT: @scott_s no offense taken and upvoted. As I said, this post was just to offer some clarifications, the clash in terminology indeed gets very confusing. So, thanks for making me improve my comment, if it confused you I am sure it would have confused others as well.
ap22213 12y ago
Anyone know the performance of julia for optimization in contrast to say, ilog cplex, or gurobi?
- cschmidt 12y ago
  It interfaces to a number of solvers, both commercial and open source. According to this page [1] it currently supports: COIN Cbc/Clp, GNU GLPK, Gurobi, Ipopt, Mosek, NLopt So you can use it to generate models that are solved in Gurobi if you want. Or you can use the available open source solvers. [1] http://www.juliaopt.org/ http://www.juliaopt.org/
- StefanKarpinski 12y ago
  There are excellent Julia bindings to CPLEX, Gurobi, and many other optimization libraries: http://www.juliaopt.org/ http://www.juliaopt.org/. At some point there may be pure Julia linear programming libraries, but initially it's much more fruitful to provide a consistent shared interfaces to existing libraries than to reimplement them.
- Tarrosion 12y ago
  One can easily use such solvers in Julia via an excellent modeling language, JuMP [1]. You can easily specify optimization problems in human readable form and have these passed to the solvers; speed is quite good. https://github.com/JuliaOpt/JuMP.jl https://github.com/JuliaOpt/JuMP.jl
- ViralBShah 12y ago
  In addition to the pointers in the comments here, also see the fast developing CVX.jl package. https://github.com/cvxgrp/CVX.jl https://github.com/cvxgrp/CVX.jl
coldtea 12y ago
It's unfortunate the Julia community doesn't have enough resources to maintain their website. The available version was 0.2 for ages (I see now they offer a pre-release 0.3), and the blog hasn't been updated for 9 months. Some simple posts every month or so would signal that this is a live, worked-on, language (which it is, and interesting stuff happens all the time). It was difficult to even find out about JuliaCon -- no banner, no blog post, nothing, just a small link in the main site navigation, that one can easily not notice.
- offsetPSK 12y ago
  julia is developed githup/maillist centric. The source+community link should show you this.
- deleted 12y ago
  [deleted]
- ihnorton 12y ago
  To get a full sense of the level of activity, see here: https://github.com/JuliaLang/julia/pulse https://github.com/JuliaLang/julia/pulse (and that is only the core - there is also considerable activity in the packages) More frequent posts on the main blog would indeed be a good idea. There will certainly be a few GSoC student posts as the summer progresses. As of a few days ago there is a also Julia blog aggregator highlighting community posts: http://www.juliabloggers.com/ http://www.juliabloggers.com/ I believe we've had 0.3-pre nightlies posted since at least January. As far as JuliaCon, it was announced on the mailing lists and did show up on HN once I believe. It sold out very quickly, so wasn't really actively promoted after that.
baldfat 12y ago
Julia looks amazing BUT as an R user: 1) RStudio is just such an amazing tool with rmarkdown and pandoc. I run my statistics in a document and can output to html, pdf or Word. I can't tell you how important the ability to print out to Word is in my corporate setting. 2) The libraries and tools are vast (Seems like Linux tons of options but really one or two are perfect for the individual use) 3) 99% of what I do takes less then 2 seconds with R Personally if I have a need for large datasets I will look to Julia if I can't get a solution with R and data.tables.
- johnmyleswhite 12y ago
  Personally, I don't see Julia replacing R for a while. As you note, the R community has much of what it needs already. When I initially switched from R to Julia, it was because R wasn't really usable for the kind of work I do. It's since become clear to me that I work on problems that don't come up for most R users: problems involving large sparse matrices, including large-scale optimization and MCMC. If you don't deal with those kinds of things, R is a good choice.
  - bachmeier 12y ago
    I think the best route is to create shared libraries using Julia and call them from R.
  - peatmoss 12y ago
    It seems like MCMC is a potentially killer app for Julia. Any thoughts as to how Julia's MCMC facilities compare to, say, something like Stan?
    - johnmyleswhite 12y ago
      I think MCMC.jl is pretty immature compared with Stan. But there is work on a Stan wrapper. Personally, I mostly write MCMC code by hand these days.
      - peatmoss 12y ago

114 points by ViralBShah 12y ago

srean 12y ago

Would the video, slides (and corresponding source repos) be uploaded sometime ? Would love to see those. Reading between the lines, devectorize.jl and the vectorization framework will make for a heady mix. I am glad that Julia is challenging the traditional mantra of "vectorize (as in the R/Numpy/Matlab sense) the loops". It is heart warming to see that the language designers get it that this style has inherent speed limitations (time is spent filling and copying temporary vectors). If any Julia core developer is seeing this, just a shout out, (i) to say you are doing awesome work, and (ii) @ macros available in the standard library are not documented well. It would be good to have a page that describes what is available now. EDIT: Just clarifying my point. Macros as a feature are quite well described in the docs. What I miss is an index of macros that come built in with the standard library. They would help both with pedagogy and with putting them to actual use, for example removing bounds checks. If they are a little more discoverable than they are now that would be very helpful to a newcomer. For a new'ish language the documentation is otherwise quite admirable. @ViralBShah I filed an issue.

ViralBShah 12y ago
The videos are being edited and will be available shortly. We'll post them on juliacon.org and julialang.org. Does the Metaprogramming section of the manual not adequately describe macros? It would be great if you submit a pull request or an issue describing what is missing.
ViralBShah 12y ago
An index of macros in the standard library is a good idea. Would you mind filing an issue?
- StefanKarpinski 12y ago
  Macros are documented along with everything else in the standard library: http://docs.julialang.org/en/latest/stdlib/base/ http://docs.julialang.org/en/latest/stdlib/base/ The stdlib docs are organized by subject matter. What would the benefit of having a separate index of macros be? If one wants a list of exported macros, that's just a matter of doing grep @ base/exports.jl in a Julia repo.
jarvist 12y ago
Slides are on GitHub: https://github.com/JuliaCon/presentations https://github.com/JuliaCon/presentations I believe the video is heading for their YouTube channel: https://www.youtube.com/user/JuliaLanguage/featured https://www.youtube.com/user/JuliaLanguage/featured
scott_s 12y ago
I had to look up Devectorize.jl: https://github.com/lindahua/Devectorize.jl https://github.com/lindahua/Devectorize.jl The point, I think, is not that vectorizing is slow. It's that you don't want to be myopic when you do it. Your point about time spent filling and copying vectors is independent of what Devectorize is trying to do. Yes, when you vectorize code, you need to have a prologue and an epilogue which gets the data in the correct format for the SIMD operations. This prologue and epilogue has a cost, and if it's larger than the gain from the SIMD operations, then it's not worth vectorizing. (This can happen if, say, you only have 4 elements to vectorize.) However, we have to do this cost-analysis all the time. Figuring out when it's beneficial to vectorize code has been around as long as we've vectorized code. Devectorize.jl is not about that problem. Rather, it's about when you have a chain of expressions, and the operators in those expressions have implied loops. The naive thing to do is to myopically execute each one of those loops, creating and passing around temporary vectors. The Devectorize framework is given an entire expression, and is able to analyze where the implied loops are, and figures out how to express that computation in a single loop. We can tell these two concepts are independent because the ideal situation is to first use Devectorize on an expression, and then vectorize the result! For the record, the approach taken by Devectorize.jl is similar to the problem that expression templates (http://en.wikipedia.org/wiki/Expression_templates http://en.wikipedia.org/wiki/Expression_templates) in C++ try to solve. Through template trickery and operator overloading, we can put off evaluating an expression, avoiding the temporaries and unnecessary memory traversal that a naive execution implies. The Boost library uBLAS (micro Base Linear Algebra Subprograms, http://www.boost.org/doc/libs/1_55_0/libs/numeric/ublas/doc/index.htm http://www.boost.org/doc/libs/1_55_0/libs/numeric/ublas/doc/...) uses this technique.
- srean 12y ago
  Not sure that I understand the point/tone of your comment, all that you say is correct and I am not unaware of any of those points nor do I contradict them in mine, but it seems that you are offering it as a correction. From your comment it seems that you are aware of the two different meanings of "vectorization" but that you are choosing the wrong one to setup a strawman. So I am a bit puzzled. In any case let me offer some clarifications: The canonical example of ET and the one that started it all is Blitz++ (unlike its modern descendants it actually handles full n-dimensional tensors). More recent solutions are Blaze, Eigen and Armadillo. I like these over uBLAS (more verbose and not as performant). Blaze is typically the fastest (in this class) as long as you are dealing with 2D arrays and linear algebraic operations. Since Julia is homoiconic and has hygienic macros, one does not have to resort to such syntactically complex ways of doing metaprogramming such as ETs in C++. ET's are great to use, not very pleasant to write the machinery for, although things like Boost::spirit and fusion make it more tolerable. Heaven forbid that you have to understand the error messages though. Now, to another point, looping is so pathetically slow in R, Numpy and Matlab (in that order) that the traditional solution offered in these languages is to vectorize (in the Matlab/numpy/R sense not in the SIMD sense) the loops into vector-expressions. Often this needs the help of extra prefilled vectors/matrices, (some languages do it smartly with 'broadcasting', some where late to pick it up, looking at you Matlab). Even with broadcasting tricks, the extra stride indirection in the iteration slows down the code from how fast it could have been. That is but one aspect of it, the other is chaining several binary and unary operations together. This also creates temporaries and unnecessary loops (unless you use solutions of somewhat limited scope like numexpr, which BTW is lovely when it is applicable, it also does thread level parallelization as well as SIMD if you can afford to link with intel MKL). In Julia there is no need to avoid loops because they are inherently fast. In fact they are faster than the vectorized expressions, because of inefficiencies mentioned. However, loops are a lot more verbose than expressions. So this is where devectorize comes in. It transforms those expressions into loops (much like ETs) which can then be JITed to obtain superior performance. Now with SIMD vectorization thrown in on top, one can gain even more, because now one can benefit further from the instruction level parallelism. So yes you devectorize and then vectorize (and you already know this) and the term "vectorization" refer to two different concepts named vectorization , the clash in terminology is rather unfortunate. EDIT: @scott_s no offense taken and upvoted. As I said, this post was just to offer some clarifications, the clash in terminology indeed gets very confusing. So, thanks for making me improve my comment, if it confused you I am sure it would have confused others as well.

ap22213 12y ago

Anyone know the performance of julia for optimization in contrast to say, ilog cplex, or gurobi?

cschmidt 12y ago
It interfaces to a number of solvers, both commercial and open source. According to this page [1] it currently supports: COIN Cbc/Clp, GNU GLPK, Gurobi, Ipopt, Mosek, NLopt So you can use it to generate models that are solved in Gurobi if you want. Or you can use the available open source solvers. [1] http://www.juliaopt.org/ http://www.juliaopt.org/
StefanKarpinski 12y ago
There are excellent Julia bindings to CPLEX, Gurobi, and many other optimization libraries: http://www.juliaopt.org/ http://www.juliaopt.org/. At some point there may be pure Julia linear programming libraries, but initially it's much more fruitful to provide a consistent shared interfaces to existing libraries than to reimplement them.
Tarrosion 12y ago
One can easily use such solvers in Julia via an excellent modeling language, JuMP [1]. You can easily specify optimization problems in human readable form and have these passed to the solvers; speed is quite good. https://github.com/JuliaOpt/JuMP.jl https://github.com/JuliaOpt/JuMP.jl
ViralBShah 12y ago
In addition to the pointers in the comments here, also see the fast developing CVX.jl package. https://github.com/cvxgrp/CVX.jl https://github.com/cvxgrp/CVX.jl

coldtea 12y ago

It's unfortunate the Julia community doesn't have enough resources to maintain their website. The available version was 0.2 for ages (I see now they offer a pre-release 0.3), and the blog hasn't been updated for 9 months. Some simple posts every month or so would signal that this is a live, worked-on, language (which it is, and interesting stuff happens all the time). It was difficult to even find out about JuliaCon -- no banner, no blog post, nothing, just a small link in the main site navigation, that one can easily not notice.

offsetPSK 12y ago
julia is developed githup/maillist centric. The source+community link should show you this.
deleted 12y ago
[deleted]
ihnorton 12y ago
To get a full sense of the level of activity, see here: https://github.com/JuliaLang/julia/pulse https://github.com/JuliaLang/julia/pulse (and that is only the core - there is also considerable activity in the packages) More frequent posts on the main blog would indeed be a good idea. There will certainly be a few GSoC student posts as the summer progresses. As of a few days ago there is a also Julia blog aggregator highlighting community posts: http://www.juliabloggers.com/ http://www.juliabloggers.com/ I believe we've had 0.3-pre nightlies posted since at least January. As far as JuliaCon, it was announced on the mailing lists and did show up on HN once I believe. It sold out very quickly, so wasn't really actively promoted after that.

baldfat 12y ago

Julia looks amazing BUT as an R user: 1) RStudio is just such an amazing tool with rmarkdown and pandoc. I run my statistics in a document and can output to html, pdf or Word. I can't tell you how important the ability to print out to Word is in my corporate setting. 2) The libraries and tools are vast (Seems like Linux tons of options but really one or two are perfect for the individual use) 3) 99% of what I do takes less then 2 seconds with R Personally if I have a need for large datasets I will look to Julia if I can't get a solution with R and data.tables.

johnmyleswhite 12y ago
Personally, I don't see Julia replacing R for a while. As you note, the R community has much of what it needs already. When I initially switched from R to Julia, it was because R wasn't really usable for the kind of work I do. It's since become clear to me that I work on problems that don't come up for most R users: problems involving large sparse matrices, including large-scale optimization and MCMC. If you don't deal with those kinds of things, R is a good choice.
- bachmeier 12y ago
  I think the best route is to create shared libraries using Julia and call them from R.
- peatmoss 12y ago
  It seems like MCMC is a potentially killer app for Julia. Any thoughts as to how Julia's MCMC facilities compare to, say, something like Stan?
  - johnmyleswhite 12y ago
    I think MCMC.jl is pretty immature compared with Stan. But there is work on a Stan wrapper. Personally, I mostly write MCMC code by hand these days.
    - peatmoss 12y ago