7 ms·
Secretary Problem
- brandonhsiao 13y agoIs it possible to explain in layman's terms why 1/e is the magic number?
- gargarplex 13y agoThe function that approximates the best ideal solution can be approximated with the log function. It's kind of like why the taylor polynomials means that e^pi*i = -1
- mchusma 13y agoI don't think that was exactly a lay explanation. Was interested myself.
- gargarplex 13y agoYeah, it totally wasn't. I failed. Okay, so one of the really interesting properties of mathematics is that complicated functions can sometimes be simplified by simple functions. For example, the gamma function is an extension of the factorial function (4! = 432*1). http://upload.wikimedia.org/math/a/c/5/ac57cb1b5db9b61155d862c7a02fe425.png http://upload.wikimedia.org/math/a/c/5/ac57cb1b5db9b61155d86... Amazingly, you are able to represent one idea in another, different way. So, with the secretary problem, you are able to break down the optimal solution as this: http://upload.wikimedia.org/math/6/f/4/6f4d3a757d9efe51b8b1718dc4c58f67.png http://upload.wikimedia.org/math/6/f/4/6f4d3a757d9efe51b8b17... The | notation means "given". So x|b means "x, given b". For these abstract formulations, they are reduced to already-existing calculations.. Finally, you get the big sigma symbol. This is called a Summation. Summations tend to be able to be approximated using integrals, since an integral is the area under the curve, which is also a summation of sorts. Many integrals can be defined using the "log" function, which has base "e". In this case, when you add in "infinity" for the summation, it converges towards 1/e. Hope this helps some. If it doesn't, well, I was fired from my job TA'ing calculus =B so..
- zindlerb 13y agoVery interesting! I assume this is posted because it parallels how YC does their interview admissions. I think the YC method has some differences to this problem. YC has already seen the applications for groups, and probably already developed some kind of best to worse ranking of the applicants. The interview most likely functions as a confirmation that the teams live up to their great application. Additionally, YC expands its class to fit the number of qualified applicants. In this problem there is single spot to fill. Edit: It is also totally possible many people upvoted this article only because it is interesting. If that is true ignore my post.
- forgotprevpass 13y agoOn a related note, can someone suggest a good textbook optimal stopping?
- Cogito 13y agoThe section on experimental studies touches on it briefly, but it's important to note that costs involved in the selection process are not considered in the standard construction of the problem. From wikipedia: > In large part, this work has shown that people tend to stop searching too soon. This may be explained, at least in part, by the cost of evaluating candidates. and then > For example, when trying to decide at which gas station to stop for gas, people might not search enough before stopping. If true, then they would tend to pay more for gas than they might had they searched longer. So you might pay more for gas if you don't use the optimal strategy, but waste more money in fuel costs searching for the cheaper price than you save buy purchasing at that price. Perhaps most interestingly, there is a formulation of the problem that requires a decision to be made within a certain time period, from an unknown number of candidates who arrive over that time period. If you know (or can estimate) the arrival times of the candidates, you can use a very similar strategy to achieve optimal results. Essentially, wait until you have seen 1/e of expected candidates in the time frame you have allowed for (based on the arrival density function you know or estimated) then pick the next best option. This puts a limit on the amount of searching you do, and so provides an optimal strategy with a bounded limit on how long you search for; it provides a bound on the cost involved in the search assuming the cost is related to time taken. In the searching for gas example, you could use this strategy if you knew roughly how often you pass a gas station, and how long you are willing to search for.
- sesqu 13y agoThe formulation you're talking about is mentioned in the article, under "unknown number of applicants". However, it's worth pointing out that the strategy is unacceptable for the gas station scenario, because of the high probability of total failure.
- vehementi 13y agoWhat about needing to hire M candidates out of N applicants?
- D_Alex 13y agoAnd what about preferring the second best candidate to the average candidate... say if you interviewed 95% of applicants after rejecting the first 37%, and then you find a candidate who is almost, but not quite, as good as the very best one?
- Cogito 13y agoWithout thinking about it too much... One solution might be to try and find the the best candidate after (N/M)/e candidates, rather than N/e candidates. Then find the best candidate after (N_remaining/(M-1))/e candidates. That is recurse, by finding one candidate at a time, and discarding less then N/e each time to ensure you get enough candidates.
- hoonose 13y agoThere's a large body of work on variations of the secretary problem. The one I know of which is most relevant to your question is section 4 of the following: http://www.cs.cornell.edu/~rdk/papers/secArt4.pdf http://www.cs.cornell.edu/~rdk/papers/secArt4.pdf Section 6 is particularly interesting, where you're further restricted - you want to choose multiple secretaries, but there's certain constraints on the sets you can choose. For example, the secretaries might be edges in a graph, and you can't pick a subset of edges which would result in a cycle (this is a "graphic matroid," which is an example of a mathematical object known as a matroid). The reason why this formulation of the problem is interesting is because the best known algorithm is O(sqrt(log k))-competitive (where k is the rank of the matroid), whereas it is conjectured that an O(1)-competitive algorithm exists.
- yogeshgirdhar 13y agoI worked on a version on this problem where the goal is to maximize the probability of hiring all top M candidates while using a similar single threshold strategy. Here is a link to my paper: http://www.cim.mcgill.ca/~yogesh/publications/crv2009.pdf http://www.cim.mcgill.ca/~yogesh/publications/crv2009.pdf
- FLUX-YOU 13y agoI'm going to be the dumb guy ranting here and say that I dislike this word problem since external knowledge of the world can change your strategy. I might be stopping too soon because the time cost of evaluating candidates is far too high compared to the work that needs to be done immediately. The sooner I get someone in, the sooner that work gets done, the less behind we all get, the less workload for the new secretary, the better he/she will perform, and the better the first impression. You might also be able to recognize a rock star secretary immediately or recognize obviously incompetent people. There's a competence threshold that's present here and has to be addressed if you dissect the analogy. Going for 'best' here has diminishing returns beyond a certain level of competence. The immediate need to decline/accept also really doesn't make sense if we're trying to explain this with a hiring analogy. However, the Game of Googol nails it and I really like this approach much better when explaining this problem. It's a game with arbitrary rules so I can't easily use my worldly experience biases to solve the problem.
- chaz 13y agoI think the difference is that you're reading it as practical word problem, when the intent is to present a math problem. I first heard about this problem in a politically incorrect variation: a tribe chief is given the opportunity to choose a wife from a pool of n women, presented in a random order, one at a time. He can choose any woman, but can't choose a woman who he has already passed up. What's the strategy for the chief to choose the most beautiful wife?
- aaronsnoswell 13y agoHow is that variation politically incorrect? Genuine question here. If anything, to me it seems culturally correct.
- roel_v 13y agoDo you really need to ask? Are you seriously incapable of seeing that some people might be offended (stupidly, if you ask me, but that doesn't make them any less so) by a story where a woman is apparently picked like cattle by a feudal lord?
- sk5t 13y agoQuite interesting that discarding the first n/e candidates produces a 1/e probability of choosing the optimal one... As for real-world applications, the only one that comes readily to mind is rolling up a D&D character, supposing one has only the patience for some predetermined n rolls. Somewhat ironically it is not a realistic fit for hiring, as there is seldom a need to accept or eliminate candidates immediately, interviewing more than 5-6 people for a role is torturous, and an interviewer uses his experience interviewing and interacting with people generally to have no mathematical prejudice against hiring the very first pretty-good option.
- thaumasiotes 13y ago> Quite interesting that discarding the first n/e candidates produces a 1/e probability of choosing the optimal one... Freeform thoughts from here: 1/e is the probability of failing (every time) if you try a 1/n chance n times, where n is large. To get the optimal choice, there are two (generously defined) requirements: - #1 does not occur in the discarded group - #1 occurs before everyone else who was not discarded, but exceeds the maximum of the discarded group The second one seems hairy to me, so I'll wrap things up here. ;p > As for real-world applications, the only one that comes readily to mind is rolling up a D&D character This was introduced to me as a marriage problem, and wikipedia also notes it by that name. The analogy never seemed inaccurate to me; it's frowned upon to evaluate candidates simultaneously and rare to accept someone other than the latest candidate.
- kokey 13y agoInteresting. It describes roughly the strategy that I've settled on when looking for accommodation to rent. I think it also has the added advantage of making you feel you've made a good decision.
- jmtame 13y agoI worked on several iterations in the dating space for about a year, and this type of stuff was pretty interesting to me. Wrote a bit more about it here: https://medium.com/unfinished-thoughts/2cac1e6cc7b4 https://medium.com/unfinished-thoughts/2cac1e6cc7b4
- Johnie 13y agoThe same strategy can be used in dating too. When you live in a major city and the dating options are boundless, there is always someone around the corner that is "perfect" or more perfect than the current person that you are dating. Next thing you know, you're in your late 30s and still single.
- gargarplex 13y agoI always feel bad when I see these posts. I'm 26 and in a major city, but finding dates is a serious problem for me. My looks are only slightly below average (4/10) but my personality is probably the deal breaker.
- aegiso 13y agoHaving been where you've been, may I suggest that the dealbreaker is that you're treating dating like a World of Warcraft inventory checklist? I wish someone had told me that when I was saying the same things.
- gargarplex 13y agoOkay, so I definitely treat dating like a WoW inventory checklist... (not sure what the next step is)
- tunesmith 13y agoThere's no secret to it. You just have to doggedly keep at it.
- riffraff 13y agoFWIW, I do think there are similarities between dating and RPGs that I wish I had understood before * you can improve your character: confidence is important, if you feel ugly sometimes you can do stuff about it (i.e. go to gym). Ditto if you are a boring talker, or if you are a poor listener. * grind. If you are, like I was, a guy that has a couple hours on a weekend to meet a new possible date it's hard to find the "right" one. Try to increase the chances (go out more, frequent new places) and do try more.
- blinry 13y agoNote that if you want to optimize the quality of the candidate, not the probability of picking the best one, it's better to stop after sqrt(n) candidates: https://en.wikipedia.org/wiki/Secretary_problem#Cardinal_payoff_variant https://en.wikipedia.org/wiki/Secretary_problem#Cardinal_pay...
- philjohn 13y agoThere's a TV show in the UK that uses a spin on this - it's called 4 rooms. The premise is that people come on the show with, what they consider to be, a valuable artifact. They then have the chance to take it to 4 collectors who will offer them a sum of money for said artifact. The aim is to come away with the best offer you can get - but you only get one shot with each collector, you can't go back to a previous one because all other offers have been lower.
- oggy 13y agoAre the collectors aware of the order in which they talk to the seller? That is, is the 4th collector aware that he's the "last chance" for the seller?
- jonlucc 13y agoAccording to the article, with such a small n, the contestant should visit ~1.47 of collectors before picking the best one. That's hard to do in practice.
- cshimmin 13y agoSee also Feynman's Restaurant Problem: http://www.feynmanlectures.info/exercises/Feynmans_restaurant_problem.html http://www.feynmanlectures.info/exercises/Feynmans_restauran...
- smartiq 13y agointeresting
- philbarr 13y agoHmm, I wonder if this could be used to find a decent pub whilst in a strange city? It's always a problem when you're on holiday and you know there are n pubs around, but you don't want to spend all your time going around and checking each and every pub 'cos that gets tedious. The question is - how many pubs should I visit before I give up? As a general rule is this saying you should visit n/e pubs and then just pick the next best one?
- prof_hobart 13y agoIt would. But you'd need to know, or at least guess, how many pubs there were in the city (to know when you'd reached n/e), and it would also depend on whether the pubs were randomly distributed or not. If there's nice end and a trashy end of town, you could easily have exhausted all of the good pubs before you hit the n/e.
- philbarr 13y agoYes, I guess you'd have to use real world factors to try and trim down your n before you start with the n/e thing. Although if you're in a strange place you probably won't know that much about which areas are good or not.
- sesqu 13y agoHowever, to find merely a good pub, √n is enough, given the assumptions mentioned later in the article.
- vijucat 13y agoI, like many others, used to have the attitude that "puzzles are for interviews". But recently, I have started seeing that there can be practical applications for seemingly academic ideas. For example, I know a proprietary trader who uses a concept similar to this from the field of Optimal Stopping for his trading system exclusively, i.e., his entire trading strategy, managing tens of millions of dollars for a big bank, is basically an application of Optimal Stopping to the markets!
- imdsm 13y agoWell, that was fun. (use console output) Single candidate: http://jsbin.com/cupiloye/1/edit http://jsbin.com/cupiloye/1/edit M candidates: http://jsbin.com/woyehiyu/1/edit http://jsbin.com/woyehiyu/1/edit
- cschmidt 13y agoThis gets posted on HN occasionally, and when it does I post this fun blog post by Michael Trick, a CMU Operations Research professor, who used the Secretary problem to pick his wife: http://mat.tepper.cmu.edu/blog/?p=1392 http://mat.tepper.cmu.edu/blog/?p=1392