4 ms·

Only tangentially related, but I solved a problem at my work this week with a couple of cleverly crafted MILP programs. Its really exciting in the nerdiest way

Only tangentially related, but I solved a problem at my work this week with a couple of cleverly crafted MILP programs. Its really exciting in the nerdiest way possible and I need to share it with someone, anyone! I can't write a blog post about it because I don't have a personal website, plus it'd be a trade secret anyways. Most of my coworkers aren't technical so they're happy to get an answer to their problem, but they don't really care about the how part.

protomikron 4y ago
Was it a classic scheduling problem (e.g. distributing resources)?
- monkeybutton 4y ago
  I suppose so. A first pass on which resources to acquire with different costs and different (and diminishing) rates of return. Then a second pass to assign those resources to workers who have different capacities to do work, and also can't do every kind of work (so bin packing?).
xarope 4y ago
Please do write it up! It's always interesting to read how others solve problems using similar tools. If you have a github account you can always gist it. If not, perhaps one of the prolific writers in HN would be willing to publish it for you?
OccamsRazr 4y ago
> I don't have a personal website It's so easy to make a free blog on github (like sub 5 minutes) with some version of jekyl. You should probably just do it.
daenz 4y ago
Doesn't hurt to ask for written approval from your manager to write about it! I'd love to read it as well. Getting to use optimization algorithms at your job is super rare (unless you're in specific industries).
- cm2187 4y ago
  I actually found a few use cases recently. Solving for the funding plan of a bank to balance the balance sheets and meet all the regulatory constraints (it actually uses a linearisation of a piecewise linear convex problem that I had learned in college decades ago, but completely forgot about, I asked for guidance from one of my professors who asked for permission to use the case as an exercise for his students). Assigning cash flows to changes in balances when the two feeds are inconsistent and the timing is off (you need to do some complex combinations to match them). But same as OP, you get a sense of self-satisfaction but that won’t get you any credit internally.
- monkeybutton 4y ago
  Super rare indeed! For me, it seems to be around one every couple of years. Just long enough that it becomes a novelty again.
ironSkillet 4y ago
The coolest instance of MILP I have encountered "in the wild" at work was a disaggregation problem where we could only receive vendor reports that contained aggregated statistics, each report aggregating the original data along a different dimension (say one by customer zip code, another by age range, etc). By lining up these aggregated reports, you could turn that into an MILP system that could be solved to unwind all of the aggregation and get back to the original data. This gave us way more flexibility in deciding how to use the vendor data.
- _Algernon_ 4y ago
  Was the purpose of aggregating the data to "anonymize" it? If so, this just illustrates how useless anonymizing data is.
  - lrem 4y ago
    Yup, k-anonymity is hard. State of the art is rapidly evolving, requiring higher and higher k (i.e. more coarse/less useful reports). Most companies out there prefer to, um, not notice that.
  - ironSkillet 4y ago
    Yes, and I mostly agree. Most companies just randomize an ID or do some group-bys and consider it "anonymized", which are susceptible to a variety of correlation techniques (such as setting up this MILP problem). And the more data sets you have with the same underlying masked primary key (e.g. person), the easier it can get. I have seen these weak attempts at hiding information in multiple industries. Just for the record, in my use case there was never an intent to map data to individuals, just to get the most granular information possible about key metrics.
    - webcomthrowaway 4y ago
      Hello, would this have been on vendor reports from a certain large online advertising and search company?
User23 4y ago
Write it up. Even if you can’t publish it now, you may not recall later. It’s quite likely circumstances will change such that your current employer has no interest in blocking publication so if you already have a document you’ll be able to advance human knowledge.

protomikron 4y ago

Was it a classic scheduling problem (e.g. distributing resources)?

monkeybutton 4y ago
I suppose so. A first pass on which resources to acquire with different costs and different (and diminishing) rates of return. Then a second pass to assign those resources to workers who have different capacities to do work, and also can't do every kind of work (so bin packing?).

xarope 4y ago

Please do write it up! It's always interesting to read how others solve problems using similar tools. If you have a github account you can always gist it. If not, perhaps one of the prolific writers in HN would be willing to publish it for you?

OccamsRazr 4y ago

> I don't have a personal website It's so easy to make a free blog on github (like sub 5 minutes) with some version of jekyl. You should probably just do it.

daenz 4y ago

Doesn't hurt to ask for written approval from your manager to write about it! I'd love to read it as well. Getting to use optimization algorithms at your job is super rare (unless you're in specific industries).

cm2187 4y ago
I actually found a few use cases recently. Solving for the funding plan of a bank to balance the balance sheets and meet all the regulatory constraints (it actually uses a linearisation of a piecewise linear convex problem that I had learned in college decades ago, but completely forgot about, I asked for guidance from one of my professors who asked for permission to use the case as an exercise for his students). Assigning cash flows to changes in balances when the two feeds are inconsistent and the timing is off (you need to do some complex combinations to match them). But same as OP, you get a sense of self-satisfaction but that won’t get you any credit internally.
monkeybutton 4y ago
Super rare indeed! For me, it seems to be around one every couple of years. Just long enough that it becomes a novelty again.

ironSkillet 4y ago

The coolest instance of MILP I have encountered "in the wild" at work was a disaggregation problem where we could only receive vendor reports that contained aggregated statistics, each report aggregating the original data along a different dimension (say one by customer zip code, another by age range, etc). By lining up these aggregated reports, you could turn that into an MILP system that could be solved to unwind all of the aggregation and get back to the original data. This gave us way more flexibility in deciding how to use the vendor data.

_Algernon_ 4y ago
Was the purpose of aggregating the data to "anonymize" it? If so, this just illustrates how useless anonymizing data is.
- lrem 4y ago
  Yup, k-anonymity is hard. State of the art is rapidly evolving, requiring higher and higher k (i.e. more coarse/less useful reports). Most companies out there prefer to, um, not notice that.
- ironSkillet 4y ago
  Yes, and I mostly agree. Most companies just randomize an ID or do some group-bys and consider it "anonymized", which are susceptible to a variety of correlation techniques (such as setting up this MILP problem). And the more data sets you have with the same underlying masked primary key (e.g. person), the easier it can get. I have seen these weak attempts at hiding information in multiple industries. Just for the record, in my use case there was never an intent to map data to individuals, just to get the most granular information possible about key metrics.
  - webcomthrowaway 4y ago
    Hello, would this have been on vendor reports from a certain large online advertising and search company?

User23 4y ago

Write it up. Even if you can’t publish it now, you may not recall later. It’s quite likely circumstances will change such that your current employer has no interest in blocking publication so if you already have a document you’ll be able to advance human knowledge.