6 ms·

What kind of companies would be good to get practical experience in this area? In my experience the bridge between theoretically and actually apply mathematics

by hgibbs 12y ago

What kind of companies would be good to get practical experience in this area? In my experience the bridge between theoretically and actually apply mathematics is very hard to cross. Having started working mathematically recently it seems to me that even though I can propose a solution which isn't the best, they are usually better than what is currently available. Is that a common experience as a working mathematician. That story about FedEx is hilarious!

graycat 12y ago
Part I > Having started working mathematically recently it seems to me that even though I can propose a solution which isn't the best, they are usually better than what is currently available. Is that a common experience as a working mathematician. Sure. For the politics, just report the savings. The stuff about the optimal solution, that is, its, say, moral or ethical value, is lost on business people. With some good common sense, the business people just want to know how much has been saved (or extra earned). If you can also have, say, a lower bound on the possible savings and get close to that, then you have in practice a stopping criterion. Or, what in practice is really difficult about the NP-hard problems is usually saving the very last possible tiny fraction of one penny and proving that did that. Instead, just concentrate on saving, say, the first $5 million a month on, say, the $100 million a month of jet fuel for an airline. For a while in optimization, having the optimal solution was made a moral objective with anything that wasted even 0.0001 pennies regarded as a sin. Religious nonsense. Really sloppy work is not good, but close approximation usually is plenty good. For not the best, that's the other time I kept FedEx from going out of business: The founder, Fred W. Smith, spent the afternoon in his office trying to schedule the fleet, staggered out exhausted, said "We need a computer", and a college friend of mine heard that plea and called me. Sure, digging in, I heard a lot about optimal solutions. But I knew that first just get something that helps, say, so can type in a schedule, have the software check it out for, say, planes being over loaded or having to fly too far, get the arrival times, etc., and print out all the details. So, I typed in 6000 lines of code to do that. Right, great circle distances are from just the law of cosines for spherical triangles! Soon the Board wanted a schedule; biggie Board members doubted that the scheduling could even be done; biggie funding was being held up; the company was at risk. So, I got a call from the same guy who called me to the Board meeting for the revenue projections, Roger Frock (he wrote a book about FedEx), and he and one short, easy evening he and I used my software to develop a schedule for the whole planned system of 90 cities and 33 airplanes. It printed out and looked nice. At the next senior staff meeting, Fred's reaction to the printout was "Amazing document -- solved the most important problem facing Federal Express". Our same two guys from General Dynamics went over the schedule carefully and announced "It's a little tight in a few places but it's flyable". The Board was pleased; the funding was enabled. That was the first time I saved FedEx. So, I got into how to do better scheduling. So, for one period from, say, 5 PM to midnight (again from 1 AM to, say, 8 AM) set up a table with one row for each city to be served, 90 cities, and one column for each candidate airplane tour from Memphis and back. So, may have some thousands of columns. Considering all fine details want to account for, and take expectations for random costs, find for each of the candidate tours the cost (first cut, assume that if a plane goes to a city, then it completely serves that city). Then for the schedule, want to pick no more than 33 columns from the 1000s so that all 90 cities are served and total cost is minimized. So, each column covers some cities, and want to pick columns that cover all 90 cities -- so have a set covering problem. Right, it's in NP-complete. Well, that table is essentially the matrix in a 0-1 integer linear programming least cost optimization problem. There have one unknown variable for each of the columns, and the variable is 1 if use that column and 0 otherwise. If relax the 0-1 constraint, then have a fairly routine linear programming problem but will likely end up with some fractional airplanes allocated. Could use branch and bound to get to a good or optimal 0-1 solution. Should have saved FedEx a huge bundle. Now, with software such as the IBM Optimization Subroutine Library (OSL), C-PLEX (now owned by IBM but from Robert Bixby at Rice in Dallas), etc., that would be a fun and likely fairly easy problem. Alas, for proposing that work as my project, I got too much guff and flack back from too much of the top of FedEx, wasn't getting much thanks for the two times I'd saved the company, was not seeing the promised stock, was away from my home and wife, was in a company close to going belly up, saw a lot of dumb work, so went for my Ph.D.
graycat 12y ago
Part II For what companies to go to to get going on a career in such applied math, I can't think of any. Mostly people and their organizations would rather waste money than get involved with mathematicians doing work only mathematicians understand. Again, if want to make math pay, just sell the results, usually as a vendor outside the customer. If are inside the company, then likely no good work will go unpunished. Basically to be successful, you will need the strong personal support of the CEO and hopefully the Board -- literally. And the CEO will routinely have to go around and break 2 x 4s over the heads of recalcitrant middle managers to see if they are just asleep or really dead -- otherwise those middle managers can sabotage you with high determination and creativity. E.g., twice I saved FedEx from going out of business and had much more that would have saved FedEx maybe tens of millions of dollars a year, but I got flack and push back and contempt and none of the promised stock. Such are some of the challenges in getting stuff done. But successful people have been overcoming challenges, some much worse than I faced at FedEx, for many centuries. Now that you know about the possibilities of some of the absurd challenges, don't be surprised. Really, one of the best rules is just to keep your good technical work essentially a secret until you have a solution that is just too darned good to refuse, and then pull back the curtain, let people see the results and also see that it's too late to fight with you. Better still, just provide such results as a business and own the business. E.g., in some cases, just do the work for free, show the results, and then announce "You can have this for your real operations for a fee of only 15% of the savings." Once there was an in-house research group that was successful. It was for WestVaco paper -- think specialty papers, e.g., coated papers for milk cartons. So, they had Ph.D. chem eng guys running their plants in jungles, etc., that is, where the trees were, but it was a family owned business with HQ in NYC. The research shop was in Maryland, and I got an explanation from them: Each year Research went to NYC for their budget. Always the CEO, family guy, offered much more money for the Research shop than the shop was willing to accept (first-cut, good budget situation!). Why? The rules: Research picks their own projects. Projects tend to run from a few months to a few years. One project in 10 gets to operations. When Research has a project ready for operations, they approach the relevant operational managers. In the first three years, savings are allocated half to Research and half to the operational unit. After three years, all the savings are allocated to the operational unit. With this accounting, Research was returning $3 to the company for each $1 in their budget. So, sure, HQ wanted to increase the budget, but Research didn't want to have to find ways to return $3 for each extra budget $1 so didn't want a bigger budget! It is just crucial that the CEO is supporting such in-house innovation. The rules are also important. Ah, while I was in grad school, to make enough money to support my wife and I while we both finished our Ph.D.s, I took a job at a DC area think tank working for the US Navy. At one point our problem sponsor group wanted an analysis of a special, worrisome scenario of global nuclear war but limited to sea -- they wanted to know how long the US SSBN fleet could survive if it was just held in reserve and didn't shoot its missiles with nuke warheads. And they wanted the results in two weeks. Gee, two weeks to model global nuclear war limited to sea! How generous! Good that they were not in a real hurry! Besides it was good timing since the day after the due date my wife had already gotten us reservations for a vacation at Shenandoah! Well, there was a guy B. Koopman who in WWII had written a report OEG-56 on finding things at sea. So, given A and B wandering around at sea, when might they have an encounter? Well, Koopman took the area of the sea, the two velocities, and the detection radius and came up with an arrival rate for a Poisson process. Hmm .... So, list the Red forces and Blue forces (the US SSBNs were part of the Blue forces), that is, list airplanes, helicopters, destroyers, battleships, attack submarines, etc., the speed of each, and the detection radius. Then have a table with a row for each Blue weapon type and a column for each Red weapon type and the detection radius and, if there was an encounter, the probability that one, the other, or both died. Now tap lightly with an independence assumption (maybe not wildly unjustified), realize that a sum of independent Poisson processes is again a Poisson process with arrival rate the sum of the contributing arrival rates, and get for the state of the war a continuous time, discrete state space (very large from a combinatorial explosion) Markov process subordinated to a Poisson process. So, yes, there is a closed form solution as a matrix exponential for the transition probabilities from state to state, but that matrix exponential is absurdly large. Instead using Monte Carlo to run off, say, 500 sample paths was easy. I typed in the code. Prof J. Keilson did a technical review. He said, "There's no way you can fathom that enormous state space.". I said, "At each point in time, say, 10 days into the war, the number of SSBNs remaining is a random variable. Moreover it is bounded so has finite expectation and finite variance. So, the law of large numbers applies. So, run off 500 independent, identically distributed samples, add them up, divide by 500, and get the expectation within a gnat's ass nearly all the time. Really the Monte Carlo puts the effort where the action is.". He agreed with me and passed my work. The Navy got their results on time, and my wife got her vacation on time. Later I was told that my work had been sold to a leading US intelligence agency. I could tell you which one, but then I'd have to ...! I thought that the whole effort, which ignored so much maybe crucially important detail, was close to a joke, but, surprisingly, the output did look reasonable. Also, the two weeks was also so short it made the whole exercise next to absurd, another joke. The group I was working for didn't trust me to get results on time so had another person working independently. Well, at the end of the two weeks, the other person had nothing at all. So, I'd won. Office politics: One guy said "You write nice computer programs" and otherwise I got no thanks but just some jealousy, contempt, and silence. So, soon afterward when I'd gotten my Ph.D., I wanted to leave and did. Again, no good deed need go unpunished. Again, own the business, use math as an internal secret sauce and technological advantage, and sell just the results.

by hgibbs 12y ago

graycat 12y ago

Part I > Having started working mathematically recently it seems to me that even though I can propose a solution which isn't the best, they are usually better than what is currently available. Is that a common experience as a working mathematician. Sure. For the politics, just report the savings. The stuff about the optimal solution, that is, its, say, moral or ethical value, is lost on business people. With some good common sense, the business people just want to know how much has been saved (or extra earned). If you can also have, say, a lower bound on the possible savings and get close to that, then you have in practice a stopping criterion. Or, what in practice is really difficult about the NP-hard problems is usually saving the very last possible tiny fraction of one penny and proving that did that. Instead, just concentrate on saving, say, the first $5 million a month on, say, the $100 million a month of jet fuel for an airline. For a while in optimization, having the optimal solution was made a moral objective with anything that wasted even 0.0001 pennies regarded as a sin. Religious nonsense. Really sloppy work is not good, but close approximation usually is plenty good. For not the best, that's the other time I kept FedEx from going out of business: The founder, Fred W. Smith, spent the afternoon in his office trying to schedule the fleet, staggered out exhausted, said "We need a computer", and a college friend of mine heard that plea and called me. Sure, digging in, I heard a lot about optimal solutions. But I knew that first just get something that helps, say, so can type in a schedule, have the software check it out for, say, planes being over loaded or having to fly too far, get the arrival times, etc., and print out all the details. So, I typed in 6000 lines of code to do that. Right, great circle distances are from just the law of cosines for spherical triangles! Soon the Board wanted a schedule; biggie Board members doubted that the scheduling could even be done; biggie funding was being held up; the company was at risk. So, I got a call from the same guy who called me to the Board meeting for the revenue projections, Roger Frock (he wrote a book about FedEx), and he and one short, easy evening he and I used my software to develop a schedule for the whole planned system of 90 cities and 33 airplanes. It printed out and looked nice. At the next senior staff meeting, Fred's reaction to the printout was "Amazing document -- solved the most important problem facing Federal Express". Our same two guys from General Dynamics went over the schedule carefully and announced "It's a little tight in a few places but it's flyable". The Board was pleased; the funding was enabled. That was the first time I saved FedEx. So, I got into how to do better scheduling. So, for one period from, say, 5 PM to midnight (again from 1 AM to, say, 8 AM) set up a table with one row for each city to be served, 90 cities, and one column for each candidate airplane tour from Memphis and back. So, may have some thousands of columns. Considering all fine details want to account for, and take expectations for random costs, find for each of the candidate tours the cost (first cut, assume that if a plane goes to a city, then it completely serves that city). Then for the schedule, want to pick no more than 33 columns from the 1000s so that all 90 cities are served and total cost is minimized. So, each column covers some cities, and want to pick columns that cover all 90 cities -- so have a set covering problem. Right, it's in NP-complete. Well, that table is essentially the matrix in a 0-1 integer linear programming least cost optimization problem. There have one unknown variable for each of the columns, and the variable is 1 if use that column and 0 otherwise. If relax the 0-1 constraint, then have a fairly routine linear programming problem but will likely end up with some fractional airplanes allocated. Could use branch and bound to get to a good or optimal 0-1 solution. Should have saved FedEx a huge bundle. Now, with software such as the IBM Optimization Subroutine Library (OSL), C-PLEX (now owned by IBM but from Robert Bixby at Rice in Dallas), etc., that would be a fun and likely fairly easy problem. Alas, for proposing that work as my project, I got too much guff and flack back from too much of the top of FedEx, wasn't getting much thanks for the two times I'd saved the company, was not seeing the promised stock, was away from my home and wife, was in a company close to going belly up, saw a lot of dumb work, so went for my Ph.D.

graycat 12y ago

Part II For what companies to go to to get going on a career in such applied math, I can't think of any. Mostly people and their organizations would rather waste money than get involved with mathematicians doing work only mathematicians understand. Again, if want to make math pay, just sell the results, usually as a vendor outside the customer. If are inside the company, then likely no good work will go unpunished. Basically to be successful, you will need the strong personal support of the CEO and hopefully the Board -- literally. And the CEO will routinely have to go around and break 2 x 4s over the heads of recalcitrant middle managers to see if they are just asleep or really dead -- otherwise those middle managers can sabotage you with high determination and creativity. E.g., twice I saved FedEx from going out of business and had much more that would have saved FedEx maybe tens of millions of dollars a year, but I got flack and push back and contempt and none of the promised stock. Such are some of the challenges in getting stuff done. But successful people have been overcoming challenges, some much worse than I faced at FedEx, for many centuries. Now that you know about the possibilities of some of the absurd challenges, don't be surprised. Really, one of the best rules is just to keep your good technical work essentially a secret until you have a solution that is just too darned good to refuse, and then pull back the curtain, let people see the results and also see that it's too late to fight with you. Better still, just provide such results as a business and own the business. E.g., in some cases, just do the work for free, show the results, and then announce "You can have this for your real operations for a fee of only 15% of the savings." Once there was an in-house research group that was successful. It was for WestVaco paper -- think specialty papers, e.g., coated papers for milk cartons. So, they had Ph.D. chem eng guys running their plants in jungles, etc., that is, where the trees were, but it was a family owned business with HQ in NYC. The research shop was in Maryland, and I got an explanation from them: Each year Research went to NYC for their budget. Always the CEO, family guy, offered much more money for the Research shop than the shop was willing to accept (first-cut, good budget situation!). Why? The rules: Research picks their own projects. Projects tend to run from a few months to a few years. One project in 10 gets to operations. When Research has a project ready for operations, they approach the relevant operational managers. In the first three years, savings are allocated half to Research and half to the operational unit. After three years, all the savings are allocated to the operational unit. With this accounting, Research was returning $3 to the company for each $1 in their budget. So, sure, HQ wanted to increase the budget, but Research didn't want to have to find ways to return $3 for each extra budget $1 so didn't want a bigger budget! It is just crucial that the CEO is supporting such in-house innovation. The rules are also important. Ah, while I was in grad school, to make enough money to support my wife and I while we both finished our Ph.D.s, I took a job at a DC area think tank working for the US Navy. At one point our problem sponsor group wanted an analysis of a special, worrisome scenario of global nuclear war but limited to sea -- they wanted to know how long the US SSBN fleet could survive if it was just held in reserve and didn't shoot its missiles with nuke warheads. And they wanted the results in two weeks. Gee, two weeks to model global nuclear war limited to sea! How generous! Good that they were not in a real hurry! Besides it was good timing since the day after the due date my wife had already gotten us reservations for a vacation at Shenandoah! Well, there was a guy B. Koopman who in WWII had written a report OEG-56 on finding things at sea. So, given A and B wandering around at sea, when might they have an encounter? Well, Koopman took the area of the sea, the two velocities, and the detection radius and came up with an arrival rate for a Poisson process. Hmm .... So, list the Red forces and Blue forces (the US SSBNs were part of the Blue forces), that is, list airplanes, helicopters, destroyers, battleships, attack submarines, etc., the speed of each, and the detection radius. Then have a table with a row for each Blue weapon type and a column for each Red weapon type and the detection radius and, if there was an encounter, the probability that one, the other, or both died. Now tap lightly with an independence assumption (maybe not wildly unjustified), realize that a sum of independent Poisson processes is again a Poisson process with arrival rate the sum of the contributing arrival rates, and get for the state of the war a continuous time, discrete state space (very large from a combinatorial explosion) Markov process subordinated to a Poisson process. So, yes, there is a closed form solution as a matrix exponential for the transition probabilities from state to state, but that matrix exponential is absurdly large. Instead using Monte Carlo to run off, say, 500 sample paths was easy. I typed in the code. Prof J. Keilson did a technical review. He said, "There's no way you can fathom that enormous state space.". I said, "At each point in time, say, 10 days into the war, the number of SSBNs remaining is a random variable. Moreover it is bounded so has finite expectation and finite variance. So, the law of large numbers applies. So, run off 500 independent, identically distributed samples, add them up, divide by 500, and get the expectation within a gnat's ass nearly all the time. Really the Monte Carlo puts the effort where the action is.". He agreed with me and passed my work. The Navy got their results on time, and my wife got her vacation on time. Later I was told that my work had been sold to a leading US intelligence agency. I could tell you which one, but then I'd have to ...! I thought that the whole effort, which ignored so much maybe crucially important detail, was close to a joke, but, surprisingly, the output did look reasonable. Also, the two weeks was also so short it made the whole exercise next to absurd, another joke. The group I was working for didn't trust me to get results on time so had another person working independently. Well, at the end of the two weeks, the other person had nothing at all. So, I'd won. Office politics: One guy said "You write nice computer programs" and otherwise I got no thanks but just some jealousy, contempt, and silence. So, soon afterward when I'd gotten my Ph.D., I wanted to leave and did. Again, no good deed need go unpunished. Again, own the business, use math as an internal secret sauce and technological advantage, and sell just the results.