3 ms·

> Your response makes me wonder if the idea is not that RL can give more optimal solutions, but perhaps more robust ones? LP (linear programming), given an opt

> Your response makes me wonder if the idea is not that RL can give more optimal solutions, but perhaps more robust ones? LP (linear programming), given an optimal solution, has some sensitivity analysis that nicely addresses robustness. In the LP find x to solve maximize z = cx subject to Ax = b, x >= 0, small changes in c, A, and b can result in large changes in x (below I explain some of how). But, still, due to some continuity in the linear algebra, the changes in z should (I am avoiding doing the derivations) be small, and continuing to use the optimal x from before the changes should (again, some derivations are needed) be not far off (omitting details). The feasible region is much like a diamond with flat sides and sharp points (the extreme points). Uh, let set F be the feasible region, i.e., F = {x | Ax = b, x >= 0} Then the feasible region is convex, i.e., for x_1 and x_2 in the feasible region and t in [0,1], tx_1 + (1 - t)x_2 (intuitively the points on the line between the two points x_1 and x_2) is also in the feasible region F. So, easily there can be two extreme points x_1 and x_2 in F such that z = cx_1 = cx_2 and both are optimal. Then all points on the line between x_1 and x_2 will be feasible and optimal. So, now have infinitely many optimal points. And it can be that x_1 is the only optimal point but after some small changes both x_1 and x_2 are optimal. And x_2 can be a long way from x_1. Sooooo, the small changes resulted in a big change in the optimal points. And maybe after the change, x_2 is optimal but x_1 no longer is. In this case, maybe LP does not look "robust", but that's usually not a big problem since after the small changes, likely in practice can still get by using the old optimal x_1. I.e., just by continuity, the "small changes" will keep the changes in the feasible region F and the optimal value z small. Ah, here is the good news: If raising 40,000 chickens (my father in law used to do that) and there is a small change in the prices of some of the feeds, after the small change might change suppliers (big change in x) for a savings in prices. Intuitively, the changes just showed that you are not very dependent on your old suppliers but have some alternatives that involve some big changes in x (maybe drop one supplier for another one) for about the same z. That big change in x looks not "robust" but is good news, right?

taeric 3y ago
Ish? Sensitivity analysis measures, by definition, sensitivity to inputs. I suppose you can model it so that some extrinsic factors are themselves written as inputs, but I haven't seen this explored with the idea of robustness, much. (Granted I may just be ignorant in that area.) What you are pointing to, seems to be degrees of freedom on an input? The more options you have in an input, the better. Robust systems also have to deal with the unknown adversaries of the system that will work against it, though? Not just, "we were sensitive to the costs of feed changing, but there is also now an avian flu going around and we have to kill our flock." A simulation is easy to imagine that does this and forces you to see what is next. Is that common in LP formulations?
- graycat 3y ago
  > common in LP formulations? If you want to consider the effects of war, inflation, something like the 2008 housing crash, economic depression, COVID, hurricanes, train derailments, etc., sure, if those hypothetical changes still yield an LP (linear program), can just solve the resulting LPs and display the resulting data in a way can visualize and get some intuitive insight. But there is another way: Let insurance companies consider history and set insurance premiums for disasters, all disasters, ever seen before or not, and they will have quite good data, likely, sorry, better than get from RL (reinforcement learning) or, really, anything that can be obtained from anything like current analysis of data from Web crawling. A common assumption, understanding in business is that it is possible to get insurance on nearly anything against nearly anything. E.g., the whole thing about the book and movie The Big Short was that it was possible to get insurance on the BBB tranches of housing mortgage bonds, make a bet that the tranches would fail. For this would have to pay monthly premiums but when the tranches failed make huge returns on the premiums paid. With their good data and history, an insurance company will be able to set insurance premiums that give you some good information about the seriousness of the disasters, on your list of disasters or not, you are concerned about. If seeing the premiums you are still highly concerned, then pay the premiums and relax. As in The Big Short, call up Goldman-Sachs and see if they will write you some insurance on what you want protected. Sure, if I were to invest, what for me are big bucks, in raising a flock of chickens, maybe I'd get some insurance against any disease, e.g., bird flu, attacks by rabid foxes, wolves, raccoons, fire, flood, tornado, etc. destroying the flock. Sure, once I did some Monte Carlo work essentially to evaluate the odds of some survivors in a really complex, goofy situation -- with those odds, could have found premiums for writing insurance policies. The issue was, if there were global nuclear war limited to sea, how long would the US SSBN fleet survive? For input, I used some encounter rate probability derivations Koopman did as part of ocean search in WWII. Then I used some detection radii of Red-Blue pairs and battle result probabilities (one dies, the other dies, both die, neither dies). My work passed technical review by a world-class probability professor. My summary results were a graph over time of the expected number of SSBNs surviving. Also the Monte Carlo result could have also reported the probabilities of other aspects of the scenario, e.g., the probability that at least 10 survived for 30 days, enough to write an insurance policy! The professor tried to object: He said how could I hope to "fathom" the huge state space? I responded, ..., using the strong law of large numbers ..., and he agreed with me. So, sometimes can analyze situations where some of the possibilities, actually nearly all, are never encountered or sampled. So, that would beat running LP many times one at a time on many disasters could describe. Lesson: Sometimes it's possible to do Monte Carlo under some seemingly wildly complicated situations. Net, I would advise pricing some insurance instead of some DIY (do it yourself) with either LP or RL. More broadly and by analogy, a hammer and a screwdriver are both good tools, but no sense in asking which is better. Maybe the same for LP and RL. And for one more tool, insurance policies.

taeric 3y ago

Ish? Sensitivity analysis measures, by definition, sensitivity to inputs. I suppose you can model it so that some extrinsic factors are themselves written as inputs, but I haven't seen this explored with the idea of robustness, much. (Granted I may just be ignorant in that area.) What you are pointing to, seems to be degrees of freedom on an input? The more options you have in an input, the better. Robust systems also have to deal with the unknown adversaries of the system that will work against it, though? Not just, "we were sensitive to the costs of feed changing, but there is also now an avian flu going around and we have to kill our flock." A simulation is easy to imagine that does this and forces you to see what is next. Is that common in LP formulations?

graycat 3y ago
> common in LP formulations? If you want to consider the effects of war, inflation, something like the 2008 housing crash, economic depression, COVID, hurricanes, train derailments, etc., sure, if those hypothetical changes still yield an LP (linear program), can just solve the resulting LPs and display the resulting data in a way can visualize and get some intuitive insight. But there is another way: Let insurance companies consider history and set insurance premiums for disasters, all disasters, ever seen before or not, and they will have quite good data, likely, sorry, better than get from RL (reinforcement learning) or, really, anything that can be obtained from anything like current analysis of data from Web crawling. A common assumption, understanding in business is that it is possible to get insurance on nearly anything against nearly anything. E.g., the whole thing about the book and movie The Big Short was that it was possible to get insurance on the BBB tranches of housing mortgage bonds, make a bet that the tranches would fail. For this would have to pay monthly premiums but when the tranches failed make huge returns on the premiums paid. With their good data and history, an insurance company will be able to set insurance premiums that give you some good information about the seriousness of the disasters, on your list of disasters or not, you are concerned about. If seeing the premiums you are still highly concerned, then pay the premiums and relax. As in The Big Short, call up Goldman-Sachs and see if they will write you some insurance on what you want protected. Sure, if I were to invest, what for me are big bucks, in raising a flock of chickens, maybe I'd get some insurance against any disease, e.g., bird flu, attacks by rabid foxes, wolves, raccoons, fire, flood, tornado, etc. destroying the flock. Sure, once I did some Monte Carlo work essentially to evaluate the odds of some survivors in a really complex, goofy situation -- with those odds, could have found premiums for writing insurance policies. The issue was, if there were global nuclear war limited to sea, how long would the US SSBN fleet survive? For input, I used some encounter rate probability derivations Koopman did as part of ocean search in WWII. Then I used some detection radii of Red-Blue pairs and battle result probabilities (one dies, the other dies, both die, neither dies). My work passed technical review by a world-class probability professor. My summary results were a graph over time of the expected number of SSBNs surviving. Also the Monte Carlo result could have also reported the probabilities of other aspects of the scenario, e.g., the probability that at least 10 survived for 30 days, enough to write an insurance policy! The professor tried to object: He said how could I hope to "fathom" the huge state space? I responded, ..., using the strong law of large numbers ..., and he agreed with me. So, sometimes can analyze situations where some of the possibilities, actually nearly all, are never encountered or sampled. So, that would beat running LP many times one at a time on many disasters could describe. Lesson: Sometimes it's possible to do Monte Carlo under some seemingly wildly complicated situations. Net, I would advise pricing some insurance instead of some DIY (do it yourself) with either LP or RL. More broadly and by analogy, a hammer and a screwdriver are both good tools, but no sense in asking which is better. Maybe the same for LP and RL. And for one more tool, insurance policies.