3 ms·

It is this useful in real life? You usually don’t actually need the best from an arbitrary sample, you need better than $threshold. Aiming for best from sampl

It is this useful in real life? You usually don’t actually need the best from an arbitrary sample, you need better than $threshold. Aiming for best from sample when you have $threshold already could just result in opportunity cost. Be it in hiring, dating, researching holiday destinations or anything.

exmadscientist 4y ago
This is useful when you don't have a good prior for where to set your threshold. If you have a firm threshold, yes, you're right that it isn't all that useful. The form of the solution also suggests a "good-enough" variant, useful when speed or simplicity is more important than optimality: reject the first option you find that genuinely meets all your needs. Accept the next thing you find that beats that first option. It is kind of surprising how often having this sort of approach in your back pocket can come in handy.
- quickthrower2 4y ago
  If you are threshold hunting, how do you set n? Or is n set based on resources. Would be interesting to see if there are n setting heuristics. Maybe based on how fast the best is getting better in a small sample?
  - prennert 4y ago
    n depends on your resource budget. Your resources might be limited by availability (how many secretary applicants are there), time (how long does each test take vs how much time can you invest), etc.
    - quickthrower2 4y ago
      But your budget might be flexible depending in what you get out of it. For example you have cashed up investors.
22c 4y ago
It's useful if you don't know what $threshold is or if you want to get more "bang for your buck". I believe there's a chapter in Freakonomics that deals with this problem and, if I'm not mistaken, talks about a situation where you might want to figure out some of the best places to eat in town without needing to visit all the places in town.
Sniffnoy 4y ago
This particular problem I would guess not, but the more general family of problems that is optimal stopping I would expect so.
NohatCoder 4y ago
Not really. The problem as described has two issues that makes it unlike most real problems: 1. The problem only deals with optimising finding the best candidate, the "optimal" solution therefore has a high probability of picking a really bad candidate. In reality this failure mode is quite unacceptable. 2. While many real problems will have some opportunity loss by waiting, it is quite extraordinary that options are presented in such a linear fashion. In a real hiring situation you'd generally be able to go back to a candidate you interviewed earlier and hire them.
- bo1024 4y ago
  It's even worse than (1), the optimal solution has a high probability of picking no candidate at all.
  - wikfwikf 4y ago
    Not picking any candidate at all doesn't matter if you are just trying to optimize your probability of picking the best one. Choosing one which you know not to be the best is not considered any better than choosing none.
  - NohatCoder 4y ago
    Some versions of the problem would have you just pick the last candidate then.
- TrackerFF 4y ago
  Regarding 2: This may seem somewhat contrived, but I tried to come up with some tech analogy. Let's say that you have some analyzer which takes in a finite amount of physical matter, and then produces some dataset. The analysis is a destructive process, so that the analyzer will only be able to produce N different datasets - until it runs out of the physical matter it is analyzing. The datasets are huge, and will take up all available memory. Before the next dataset is finished, you need to decide whether to store or delete/discard the current. The permanent storage can only hold one of these datasets, and it is impossible to write over it. Once you have stored a dataset permanently, the next ones will get discarded as soon as they've been generated. Or something like that. Edit: Wouldn't surprise me if there's some embedded system / machine out there that's bound to select stuff like that.
alexb_ 4y ago
The example I first heard about this problem for was with portapotties. At what point do you stop looking through the line of open portapotties to find the cleanest one? So I guess that's a use.
- yunohn 4y ago
  But that’s the best example of thresholds. You’re not going to ever find a perfectly clean one, you’ll obviously choose the first one that meets your standards.

exmadscientist 4y ago

This is useful when you don't have a good prior for where to set your threshold. If you have a firm threshold, yes, you're right that it isn't all that useful. The form of the solution also suggests a "good-enough" variant, useful when speed or simplicity is more important than optimality: reject the first option you find that genuinely meets all your needs. Accept the next thing you find that beats that first option. It is kind of surprising how often having this sort of approach in your back pocket can come in handy.

quickthrower2 4y ago
If you are threshold hunting, how do you set n? Or is n set based on resources. Would be interesting to see if there are n setting heuristics. Maybe based on how fast the best is getting better in a small sample?
- prennert 4y ago
  n depends on your resource budget. Your resources might be limited by availability (how many secretary applicants are there), time (how long does each test take vs how much time can you invest), etc.
  - quickthrower2 4y ago
    But your budget might be flexible depending in what you get out of it. For example you have cashed up investors.

22c 4y ago

It's useful if you don't know what $threshold is or if you want to get more "bang for your buck". I believe there's a chapter in Freakonomics that deals with this problem and, if I'm not mistaken, talks about a situation where you might want to figure out some of the best places to eat in town without needing to visit all the places in town.

Sniffnoy 4y ago

This particular problem I would guess not, but the more general family of problems that is optimal stopping I would expect so.

NohatCoder 4y ago

Not really. The problem as described has two issues that makes it unlike most real problems: 1. The problem only deals with optimising finding the best candidate, the "optimal" solution therefore has a high probability of picking a really bad candidate. In reality this failure mode is quite unacceptable. 2. While many real problems will have some opportunity loss by waiting, it is quite extraordinary that options are presented in such a linear fashion. In a real hiring situation you'd generally be able to go back to a candidate you interviewed earlier and hire them.

bo1024 4y ago
It's even worse than (1), the optimal solution has a high probability of picking no candidate at all.
- wikfwikf 4y ago
  Not picking any candidate at all doesn't matter if you are just trying to optimize your probability of picking the best one. Choosing one which you know not to be the best is not considered any better than choosing none.
- NohatCoder 4y ago
  Some versions of the problem would have you just pick the last candidate then.
TrackerFF 4y ago
Regarding 2: This may seem somewhat contrived, but I tried to come up with some tech analogy. Let's say that you have some analyzer which takes in a finite amount of physical matter, and then produces some dataset. The analysis is a destructive process, so that the analyzer will only be able to produce N different datasets - until it runs out of the physical matter it is analyzing. The datasets are huge, and will take up all available memory. Before the next dataset is finished, you need to decide whether to store or delete/discard the current. The permanent storage can only hold one of these datasets, and it is impossible to write over it. Once you have stored a dataset permanently, the next ones will get discarded as soon as they've been generated. Or something like that. Edit: Wouldn't surprise me if there's some embedded system / machine out there that's bound to select stuff like that.

alexb_ 4y ago

The example I first heard about this problem for was with portapotties. At what point do you stop looking through the line of open portapotties to find the cleanest one? So I guess that's a use.

yunohn 4y ago
But that’s the best example of thresholds. You’re not going to ever find a perfectly clean one, you’ll obviously choose the first one that meets your standards.