4 ms·

I know a little about constraint optimization programming, but I don't know how it works (or would work, if scaled) for neural nets. Do you know of any work on

• whatever1 5y ago
  You can start the string from here http://www.optimization-online.org/DB_FILE/2020/07/7883.pdf http://www.optimization-online.org/DB_FILE/2020/07/7883.pdf The idea is that you can express the network as an integer program. Now having that you have two big capabilities: 1) Write explicit constraints that the nodes need to satisfy. Aka if you activate this node, this and that must be activated too but that needs to be inactive. These constraints will be guaranteed to be satisfied when you get your solution. 2) You can solve the network configuration to proven global optimality or at least have a bound of how far you are from the optimal solution. That is in contrast to the current approaches with stochastic gradient descends that find locally optimal solutions with no information about how far you are from the globally optimal solution. That being said, these problems are combinatorial ones, so scaling them up to millions of nodes would be challenging.