3 ms·

>If you're doing something which is actually new, like PhD methods research, you need to be writing a lot of stuff from scratch. Sometimes you also reuse a lot

>If you're doing something which is actually new, like PhD methods research, you need to be writing a lot of stuff from scratch. Sometimes you also reuse a lot of stuff, it depends. For machine learning in particular, almost everything uses some sort of gradient-based optimisation algorithm. In these cases, it is very useful to have an automatic differentiation library. My coworker said Julia has about 3, IIRC, and until an amalgamation of them is merged into the standard library Julia isn't completely ready.

newen 9y ago
> until an amalgamation of them is merged into the standard library Julia isn't completely ready That doesn't quite make sense. I'm sure there are a few more autodiff libraries in Julia than 3. You would just use the one that fits your use case. PS. Ohh re-reading your comment, you want an autodiff library in Julia's standard library. That is very unlikely to happen since it's not (very) hard to cook up an autodiff library and autodiff is not widely used. Julia is not like Matlab, where you have to have everything in the standard library.
- rhaps0dy 9y ago
  Yeah, you read it correctly the second time :) >That is very unlikely to happen Well, my co-worker says people are working on it. I asked him whether I should try to adopt Julia and "not until autodiff is in the standard library" was his response. >autodiff is not widely used If you look only at the machine learning community, that is false. Autodiff is used all the time, to optimise loss functions using a variant of gradient descent. For neural nets, gaussian processes, SVMs... Not decision trees/forests, but these are not more popular than all the others combined.
  - one-more-minute 9y ago
    Maybe there was some small miscommunication here – we are indeed working on "one AD to rule them all" (Capstan, [1]), but it won't go in the standard library as such (there's no need as it will be just as good as a package). That said, Capstan relies on very new compiler technology and still requires some deep changes there, so will only work on future Julia versions (which may be what he was referring to). Until then, Flux [2] has an AD that's well-suited to ML and works on current Julia versions. [1] https://github.com/JuliaDiff/Capstan.jl/ https://github.com/JuliaDiff/Capstan.jl/ [2] https://fluxml.github.io/ https://fluxml.github.io/

newen 9y ago

> until an amalgamation of them is merged into the standard library Julia isn't completely ready That doesn't quite make sense. I'm sure there are a few more autodiff libraries in Julia than 3. You would just use the one that fits your use case. PS. Ohh re-reading your comment, you want an autodiff library in Julia's standard library. That is very unlikely to happen since it's not (very) hard to cook up an autodiff library and autodiff is not widely used. Julia is not like Matlab, where you have to have everything in the standard library.

rhaps0dy 9y ago
Yeah, you read it correctly the second time :) >That is very unlikely to happen Well, my co-worker says people are working on it. I asked him whether I should try to adopt Julia and "not until autodiff is in the standard library" was his response. >autodiff is not widely used If you look only at the machine learning community, that is false. Autodiff is used all the time, to optimise loss functions using a variant of gradient descent. For neural nets, gaussian processes, SVMs... Not decision trees/forests, but these are not more popular than all the others combined.
- one-more-minute 9y ago
  Maybe there was some small miscommunication here – we are indeed working on "one AD to rule them all" (Capstan, [1]), but it won't go in the standard library as such (there's no need as it will be just as good as a package). That said, Capstan relies on very new compiler technology and still requires some deep changes there, so will only work on future Julia versions (which may be what he was referring to). Until then, Flux [2] has an AD that's well-suited to ML and works on current Julia versions. [1] https://github.com/JuliaDiff/Capstan.jl/ https://github.com/JuliaDiff/Capstan.jl/ [2] https://fluxml.github.io/ https://fluxml.github.io/