3 ms·

If you're doing "deep learning" using frameworks like Tensorflow, PyTorch or JAX you're using autodiff all the time.

• scottlocklin 7y ago
  Sure: great -it can be used with other sorts of models as well, but it's really not that big a deal, and can be implemented in a lazy afternoon. I'm pretty sure lack of this as a first class language feature is holding back exactly no DL frameworks or programming languages, and glueing it onto Swift isn't going to make DL people use it. I've never used Swift's repl; I suppose that might actually be something that gets people on board.
  • stormtroper1721 7y ago
    > can be implemented in a lazy afternoon Hi, I'm working on making a tensor lib in Rust (think numpy + autodiff) to learn about these topics. There isn't much information online about how projects like numpy and autograd work under the hood. Do you have any ideas/tips/resources about how it could be done?
    • scottlocklin 7y ago
      Numpy is basically lapack. You'd be hard pressed to replace that with something nearing its performance. For autodiff, I dunno, how about John Mount's explanation? http://www.win-vector.com/blog/2010/06/automatic-differentiation-with-scala/ http://www.win-vector.com/blog/2010/06/automatic-differentia...