4 ms·
(1) Saving FedEx. The founder of FedEx, F. Smith, was in his corner office for an afternoon to work out a schedule for his fleet of airplanes. At the end of t
by graycat 13y ago
(1) Saving FedEx. The founder of FedEx, F. Smith,
was in his corner office for an afternoon to work
out a schedule for his fleet of airplanes. At the
end of the day he walked out tired and said, "We
need a computer.".
Someone I knew in college heard Smith, called me,
and I flew from Maryland to FedEx and was hired to
solve the problem.
Back in Maryland, we had some meetings, including
one in a conference room at the Georgetown library,
with various approaches to the problem, none good.
There was some politics involved.
Really, the project was just mine, so I thought of a
first-cut approach and attacked the problem; I was
still teaching computer science at Georgetown. In
six weeks I had a program, turned in the grades for
the courses I was teaching at Georgetown, drove to
Memphis, rented a room, tweaked the program a
little, and declared it done.
Soon too many on the Board were saying that there
could be no good solution to the scheduling problem.
So, one evening a senior VP and I used my program to
schedule the whole planned fleet into all the
planned cities, printed out the schedule, and
submitted it.
At a senior staff meeting, Smith's reaction was, to
paraphrase, "An amazing document; solved the most
important problem facing FedEx".
Our two Board representatives from Board member
General Dynamics went over the schedule carefully
and announced, "It's a little tight in a few places
but it's flyable.". The Board was happy and a big
chunk of equity funding was enabled.
So, the software solved the practical problem at the
time and in time. But the software was not a grand
solution to everything in fleet scheduling.
The hard parts were, (a) the politics, (b) designing
a program that would be powerful enough to solve the
practical problem at the time but easy enough to
write to solve the problem in time.
Later I attacked the problem via 0-1 integer linear
programming set covering; but the politics got much
worse; the promised stock was very late and still
just a handshake deal with Smith with only my offer
letter on paper; my wife was still in Maryland; and
I wanted either the stock or a Ph.D. Smith's last
promise of stock was $500,000 worth that might be
worth 1000 times that now, but it was just a
handshake deal. So, left for a Ph.D.
(2) Nuclear War at Sea. To support my wife and I to
the end of our Ph.D. degrees, I took a part time job
in military systems analysis. At one point the US
Navy wanted to know how long the US SSBN fleet would
last under a special scenario of global nuclear war
but limited to sea. They wanted their answer in two
weeks.
There was an old paper of B. Koopman that argued that
'encounters' at sea between Red and Blue weapons
systems would form a Poisson point process. I added
on a little and got a continuous time, finite state
space Markov process. There is a closed form
solution as a matrix exponential, but due to a
combinatorial explosion the state space was far too
large for doing anything numerical with that
solution.
But it was easy enough to generate sample paths, so
I wrote software to generate and average, say, 500
sample paths. The work passed a technical review by
a famous mathematician, and the Navy got their
results on time. The next day we took a vacation in
Shenandoah, and my wife got a vacation she wanted on
time, too!
(3) Winning a Contract Competition. There in
Maryland I was in a software house working for a US
Navy lab. We were in a competition for a software
development process. Part of the work was to
measure the power spectrum of ocean wave noise. I
got smart on power spectral estimation and wrote
some illustrative software of passing white noise
through a filter with a specific transfer function
and accumulating the empirical power spectrum of the
output of the filter. The software showed what the
math claimed: At the low frequencies the project
wanted, an accurate power spectrum needed a
surprisingly long interval of data. The software
showed that with short intervals of data, the
estimated power spectrum could have big peaks that,
really, were just sampling noise that would go away
as the length of data increased.
So I called one of the customer's engineers and
showed them the news. As a result, our software
house won the competition.
(4) Anomaly Detection. I was working on applying
artificial intelligence to the monitoring and
management of server farms and their networks. One
of the main techniques was 'thresholds', and I
wanted something better.
So, I put my feet up, popped open a cold can of Diet
Pepsi, reviewed some of my best graduate school
material including some of ergodic theory, had some
ideas, wrote out some theorems and proofs, and wrote
some corresponding software.
So, suppose are given a system to monitor. Suppose
100 times a second we get data on each of 15
variables. Collect such data as 'history' data
('learning data') for, say, three months (assume a
stable server farm or network). Study this data,
let it 'age', and be fairly sure that the system
being monitored was 'healthy' during that time.
Then in real time 100 times a second, report if the
system is 'sick' or 'healthy'. Have false alarm
rate known in advance and adjustable over a wide
range. So, have a statistical hypothesis test that
is both multi-dimensional and distribution-free, one
of the first such. Although there is not enough
data on actual anomalies to apply the Neyman-Pearson
result, do process the data in a way to promise
relatively high detection rate for any selected
false alarm rate. Find a way to do the computations
quickly and efficiently.
I did those things.
Some office politics got involved: Suddenly I was
told to write a paper on my work and that the
company would review the paper. If the paper was
not publishable, then I would be fired. I wrote the
paper; the company claimed that the paper was not
publishable; and I was fired.
The guy who walked my out the door had been in
management for about 20 years but was demoted out of
management the next day. The main guy after my ass
was two levels higher up, pissed off at me for no
good reason, two weeks later was demoted one level
in the organization chart, ws given a "performance
plan", which he failed, and was demoted out of
management.
The company wrote me a letter giving me intellectual
property rights to my work. Out of the company, I
submitted the paper for publication. The paper was
published in a good Elsevier journal, the first
journal to which the paper was submitted, without
significant revision (one reviewer wanted to change
how the first line of each paragraph was indented).
The Editor in Chief of the journal invited me to
give the paper at a conference he was running -- I
declined.
(5) Internet Search, Discovery, Recommendation,
Curation, Notification, and Subscription. My view
is that current Internet search techniques are
effective for only about 1/3rd of Internet content,
searches users want to do, and results they want to
find. I want the other 2/3rds.
With my feet up again and another cold can of Diet
Pepsi, I had some ideas and wrote the corresponding
code. Now all the code is written for a
corresponding 'search engine' Web site except I have
a nasty little bug having to do with class instance
de/serialization. Should fix that today. Should be
live in a few more months.