3 ms·

The current state of the field is reminiscent of discrete mathematical optimization in the late nineties. The computational power was increasing rapidly back th

The current state of the field is reminiscent of discrete mathematical optimization in the late nineties. The computational power was increasing rapidly back then but our our ability to solve problems with more than a hundred of binaries hadn’t improve almost at all. It was only after new theoretical results in integer programming found their way into the solution algorithms when we saw a stepwise increase of performance.

uoaei 6y ago
I think autograd and its suffusion into the modern tools for machine learning development is initiating a paradigm shift. If you haven't already, check out the SciML ecosystem in Julia. One can leverage it to build machine learning models that can slot right into any existing simulator, dynamical system, or any other method to represent systems to fit to data, and the gradients will be handled appropriately through the entire language natively. This video is from a talk at JuliaCon 2020 and is about the SciML ecosystem: https://www.youtube.com/watch?v=QwVO0Xh2Hbg https://www.youtube.com/watch?v=QwVO0Xh2Hbg For more info, check out https://sciml.ai/ https://sciml.ai/ I am not affiliated with them, but really admire their work and think it will become an invaluable tool in the coming years as the limits of neural networks become apparent and we stop running out of little tricks to squeeze one more SOTA paper out of them.
xenonite 6y ago
Interesting. Could you point me to some history article about that?
- whatever1 6y ago
  https://www.math.uni-bielefeld.de/documenta/vol-ismp/25_bixby-robert.pdf https://www.math.uni-bielefeld.de/documenta/vol-ismp/25_bixb... The story narrated by Bixby, the cofounder of CPLEX and Gurobi the two best available commercial mathematical optimization solvers. The magic happened in CPLEX V6.5 back in 1999 with almost 10x y—o-y performance improvement.

uoaei 6y ago

I think autograd and its suffusion into the modern tools for machine learning development is initiating a paradigm shift. If you haven't already, check out the SciML ecosystem in Julia. One can leverage it to build machine learning models that can slot right into any existing simulator, dynamical system, or any other method to represent systems to fit to data, and the gradients will be handled appropriately through the entire language natively. This video is from a talk at JuliaCon 2020 and is about the SciML ecosystem: https://www.youtube.com/watch?v=QwVO0Xh2Hbg https://www.youtube.com/watch?v=QwVO0Xh2Hbg For more info, check out https://sciml.ai/ https://sciml.ai/ I am not affiliated with them, but really admire their work and think it will become an invaluable tool in the coming years as the limits of neural networks become apparent and we stop running out of little tricks to squeeze one more SOTA paper out of them.

xenonite 6y ago

Interesting. Could you point me to some history article about that?

whatever1 6y ago
https://www.math.uni-bielefeld.de/documenta/vol-ismp/25_bixby-robert.pdf https://www.math.uni-bielefeld.de/documenta/vol-ismp/25_bixb... The story narrated by Bixby, the cofounder of CPLEX and Gurobi the two best available commercial mathematical optimization solvers. The magic happened in CPLEX V6.5 back in 1999 with almost 10x y—o-y performance improvement.