4 ms·
Not too impressed with the course. My qualifications: I got started in optimization scheduling the fleet at FedEx. In the words of FedEx founder, COB, CEO F.
by graycat 13y ago
Not too impressed with the course.
My qualifications: I got started in optimization
scheduling the fleet at FedEx. In the words of
FedEx founder, COB, CEO F. Smith, my work "Solved
the most important problem facing the start of
Federal Express" and the output from my software was
"An amazing document".
Later my Ph.D. research was in optimization and, in
part, what I'd wished I'd known while at FedEx.
Since my Ph.D. once there a problem in 'resource
allocation' that became a 0-1 integer linear
programming problem with 40,000 constraints and
600,000 variables. I derived some math, wrote some
software, and in 905 seconds on a 90 MHz PC got a
feasible solution within 0.025% of optimality.
And there have been other problems.
For the video, a good, first suggestion for an
attack on a knapsack problem is via dynamic
programming.
By a wide margin, the most important topic in
optimization is the simplex algorithm for linear
programming. While the algorithm directly solves
many problems in practice, it is the most important
tool in solving optimization problems that are not
linear programming. Usually there the role of
linear programming is as a local linear
approximation in an iterative scheme of better
approximations.
Much of the reason for the power of linear
programming as a tool in larger iterative schemes
is the several ways can take a linear programming
problem and an optimal solution for it, make some
changes in the problem, start with the optimal
solution for the old problem, and get an optimal
solution for the new problem very quickly.
Good starts in linear programming include:
Mokhtar S. Bazaraa and John J. Jarvis, 'Linear
Programming and Network Flows', ISBN 0-471-06015-1,
John Wiley and Sons.
Vavsek Chvatal, 'Linear Programming', ISBN
0-7167-1587-2, W. H. Freeman, New York.
Yes, the min-cost capacitated (each arc has a
maximum flow it can carry) network flow problem, a
special case of which is the transshipment problem
(find the least cost way to ship some one good from
factories to warehouses; also the source of the
Kantorovich Nobel prize in economics) is a linear
programming problem, and there the simplex algorithm
simplifies and becomes the network simplex
algorithm. Curiously, and of importance for fast
computation, in the network simplex algorithm a
basic feasible solution corresponds to a spanning
tree in the network. For that problem, don't miss
the W. Cunningham idea of a 'strongly feasible
basis' or the more recent polynomial algorithm of D.
Bertsekas.
A significantly large fraction of real problems in
combinatorial and discrete optimization can be
solved as min-cost capacitated network flow
problems via the network simplex algorithm. A big
reason is, if the flow capacities are all integers
and if have an integer basic feasible solution, then
the simplex algorithm maintains integer basic
feasible solutions for no extra work and, with the
usual assumptions, produces an optimal integer basic
feasible solution. So this is a case of linear
programming where get integer linear programming for
no extra effort; really, since nearly all the
arithmetic is just integer, don't have to be
concerned about roundoff error and get the solution
for less effort.
Yes, in combinatorial and discrete optimization,
along with the rest of optimization, linear algebra
helps.
Curiously, one of the more powerful approaches to
combinatorial optimization is a technique from
nonlinear programming called Lagrangian relaxation,
and there commonly the main computational step is
linear programming. With that technique, some
nonlinear duality theory can yield some bounds on
how far are away from an optimal solution; such
duality theory is what yielded the bound of 0.025%
above.
For combinatorial optimization there is, say,
George L. Nemhauser and Laurence A. Wolsey, 'Integer
and Combinatorial Optimization', ISBN 0-471-35943-2,
John Wiley & Sons, Inc., New York.
William J. Cook, William H. Cunningham, William R.
Pulleyblank, and Alexander Schrijver, 'Combinatorial
Optimization', ISBN 0-471-55894-X, John Wiley &
Sons, Inc., New York.
In the OP, the emphasis on NP-complete and
algorithms that are worst case exponential is a bit
exaggerated. E.g., as in the work of Klee and
Minty, the simplex algorithm is worst case
exponential, but in expectation in practice it is
low order polynomial -- for why, there was some good
work by K. Borgward.
For integer, combinatorial, and discrete
optimization, I have yet to see a claim that getting
approximately optimal solutions is in NP-complete.
In particular, if in a real problem have saved $10
million and have a bound saying that can't save more
than another $2000, then why spend millions trying
to save the last tiny fraction of the last penny and
guarantee to have done so? If are willing to
sacrifice the last penny, then it is not so clear
(so far at least to me) that really are being forced
into exponential algorithms, either average or worst
case.
The claim in the OP that combinatorial and discrete
optimization are part of computer science looks like
a case of 'academic theft'! Such optimization has
long been regarded as part of applied math and there
part of operations research. The material has been
of interest in, say, departments of mathematical
sciences. Some parts of optimization, e.g., control
theory, have been regarded as parts of electronic
engineering. At times it does appear that academic
computer science wants to regard every topic that
makes heavy use of computing as part of computer
science! It looks like computer science has gotten
tired of just quicksort, AVL trees, BNF, YACC, and
RDBMS! :-)!
The claims in the OP about the importance of
optimization in the economy are a bit over
enthusiastic. Maybe the importance would, could,
and should be there, and maybe in Australia they are
there, but my experience in the US is that there are
essentially no significant non-academic career
opportunities in optimization. E.g., at one time
near year 2000, I had a nice, long talk with one of
the leading US headhunters in technical fields, and
he finally confessed that in the previous year in
all of the US there had been exactly one recruiting
request in operations research. He "packed it in"
(from the movie 'The Sting', i.e., gave up) and went
to law school!
Here is a good future -- although a wildly long long
shot -- for integer, combinatorial, and discrete
optimization: At present, all but quite simple
problems in such optimization are solved by some
experts attacking each problem one at a time,
exploiting special structure of each problem, trying
various techniques, and, hopefully, finally getting
some good results. It's custom, one-off, bench
scale, expert, creative, hand work and often,
really, some applied research.
In addition, there have long been programs that can
be used to pose or enter problems in integer,
combinatorial, and discrete optimization suitable
for solution by a 'solver'. The problem is, since
the programs are general purpose, the 'solvers' to
be used are also general purpose and, without the
hand work, too often in practice are not powerful
enough.
Presumably a polynomial algorithm of low degree that
showed that P = NP could be the core of a general
purpose solver powerful enough to make solving such
optimization problems easy and routine.
Lacking such a solver, in practice patience with
such optimization has long been a bit thin.
So, what the heck happened? In much of physics,
say, planetary motion, we can write down some
equations the real system must solve. If the
equations have a unique solution, then that solution
must be the right one for the real system. At times
since early in WWII, such physics was of enormous
value for at least the US DoD.
So, given such an equation, solve it. This
technique often worked for, say, initial value
problems for relatively simple ordinary differential
equations. Alas, for the partial differential
equations of fluid flow, the Navier-Stokes
equations, the situation was usually a bit
challenging.
Also in problems for the US DoD, e.g., 'logistics'
or how to move a large army across a large ocean in
least time within capabilities of existing
resources, could write down some equations and other
mathematical specifications that a 'best' solution
had to satisfy. So, then, to say how to move the
army, just 'solve' the equations, etc. This work
was called 'operations research', and at times the
US DoD was very enthusiastic about it. With enough
US Federal Government research grants to academics,
parts of academics got enthusiastic about it. And
so did Bell Labs.
But there was no law of the universe that said that
finding solutions to such mathematical
specifications would be easy. Yes, Navier-Stokes is
another example of difficulty. Now, in
cryptography, so is the fundamental theorem of
arithmetic, that is, factoring a positive integer
into a product of prime numbers.
Clay Mathematics Institute in Boston has more than
one prize of $1 million available for progress on
such things.
Net, it does very much appear that for now making
money by building a popular Web site and selling ads
on it is a much easier way to make money! And, at
times, in such work some applied math, possibly new,
can be an advantage. But, clearly for now, the
grand goals of integer, combinatorial, and discrete
optimization are too difficult for 'prime time' in
practice in mainstream business.
Or, problem A can make a lot of money if we have a
lot of applied math and computing. So, let's attack
problem A. Alas, that much applied math and
computing, or even just part of the computing, may
be much more valuable for solving other problems!
- wavesounds 13y agoYou really wrote the algorithm for FedEx? Thats amazing! I read your whole post but I'm not sure exactly what you don't like about the class other than you seem to think learning Discrete Optimization in general is a waste of time because theres more lucrative problems much easier to solve. Care to briefly elaborate on what you dislike about this course specifically (it does appear that Simplex is covered)?
- graycat 13y agoPart I I wrote quickly and was not very clear for people without good backgrounds in optimization. > You really wrote the algorithm for FedEx? Thats amazing! In the sense of business, it actually was "amazing": Likely it saved the company from going out of business. Too many Members of the Board of Directors were concerned that there could be no suitable schedule. So, one evening Roger Frock and I used my software to develop a schedule for all planned 33 airplanes and all planned 90 US cities. Printed out, FedEx founder, COB, CEO F. Smith said at the next senior staff meeting "Amazing document. Solved the most important problem facing Federal Express.". Two guys representing Board Member General Dynamics went over the schedule and announced "It's a little tight in a few places, but it's flyable.". Then the Board was happy. As I recall, $55 million in funding was enabled, not all equity. So, the software was "amazing" just as business. But as applied math, the software was junk! Work for the real, pressing, short term business problem? Yes. Really optimization? No! But I did continue and saw that what I should do was (1) generate some thousands of 'feasible' trips from Memphis and back and for each carefully add up the direct costs, (2) set up a 0-1 integer linear programming problem with one row for each of the 90 cities and one column for each of the thousands of feasible trips. Then each row says to serve some one city. Each variable, 1 or 0, one for each column, says to use that trip or not. Right: It's 0-1 integer linear programming set covering, yes, in NP-complete. So, that's a linear program with only 90 rows. Just as a linear program, that's promising, likely easy. Make money? My only competition was a guy with a map of the US on a sheet of 8 1/2 x 11" paper. He didn't have a reasonable way even to add the costs of a candidate schedule or print it out. So, yes, my work should have saved a bundle. If nothing else, just solve the LP without the 0-1 constraints (with only 90 rows, that should be easy), look at the results, and see if have an easy, not very expensive way to patch up the fractions to 0-1 by hand. Else do a little branch and bound. If attacking the whole problem for the whole country is too difficult, then attack it for parts of the country separately; e.g., what do in the East has next to nothing to do with what do in the West; the goal is not to announce that have the 'optimal' solution down to the last fraction of the last penny; instead, the goal is to save money. Don't be embarrassed about not guaranteeing to save the last $1000 or the last $1,000,000 a month; instead be just horrified about not saving the first $10 million a month. Also do a little duality theory to get a bound on how much more money might be saved. When have saved the first $10 million and have no more than another $1,000 to save, take the best feasible solution so far and quit. About then I'd gotten on an airplane and run off to chat about optimization with G. Nemhauser, then still at Cornell, J. Elzinga, then still at Hopkins, and J. Pierce, then in Cincinnati, got a stack of books and papers, etc. There was also another problem: Our fuel prices and availabilities varied widely by airport. So, a question was, how much fuel to buy at each stop? So, buy extra fuel where it is cheap and available. This is called 'fuel tankering'. But, yes, will burn off some of the extra fuel carrying it. How much extra fuel is burned off is fairly sensitive to the vertical flight plan used; so also need to select vertical flight plan in a coordinated way. The problem is also complicated by the fact that package pickup loads are not known until the plane arrives for the pickup, and those loads will affect the fuel burned for each candidate vertical flight plan. Then those loads will also affect how much fuel can be carried. Also relevant is that the fuel burned and flight time for a vertical flight plan and a load are affected by essentially random winds, air traffic control, and flying around summer thunder storms. Also really get charged not just for fuel but also time on the engines. So, it was complicated. Now, try to find a way in practice to advise the pilot on what to do? So, right, it's another optimization problem -- partly discrete, nonlinear, sequential, stochastic. So, right, stochastic dynamic programming. Doable? Actually, yes, quite doable. On a computer today, could solve the problem for, say, five stops, with weight discretized at 100 pounds in about the time it takes to get a finger off the Enter key or the mouse button. Also for the vertical flight plans, I went to MIT and chatted with M. Athans about deterministic optimal control theory.