5 ms·

Ctrl+f "explore" on OP: 0 results. Ctrl+f "explore" in the HN comments: 0 results. The generalized disinclination to learn towards the end of your life is kno

Ctrl+f "explore" on OP: 0 results. Ctrl+f "explore" in the HN comments: 0 results. The generalized disinclination to learn towards the end of your life is known as the explore-exploit tradeoff: https://en.wikipedia.org/wiki/Multi-armed_bandit https://en.wikipedia.org/wiki/Multi-armed_bandit >In probability theory, the multi-armed bandit problem (sometimes called the K-[1] or N-armed bandit problem[2]) is a problem in which a fixed limited set of resources must be allocated between competing (alternative) choices in a way that maximizes their expected gain, when each choice's properties are only partially known at the time of allocation, and may become better understood as time passes or by allocating resources to the choice.[3][4] This is a classic reinforcement learning problem that exemplifies the exploration–exploitation tradeoff dilemma. The name comes from imagining a gambler at a row of slot machines (sometimes known as "one-armed bandits"), who has to decide which machines to play, how many times to play each machine and in which order to play them, and whether to continue with the current machine or try a different machine.[5] The multi-armed bandit problem also falls into the broad category of stochastic scheduling. >In the problem, each machine provides a random reward from a probability distribution specific to that machine. The objective of the gambler is to maximize the sum of rewards earned through a sequence of lever pulls.[3][4] The crucial tradeoff the gambler faces at each trial is between "exploitation" of the machine that has the highest expected payoff and "exploration" to get more information about the expected payoffs of the other machines. The trade-off between exploration and exploitation is also faced in machine learning.

pmontra 6y ago
Looks like a variation of the Law of Diminishing Returns https://en.wikipedia.org/wiki/Diminishing_returns https://en.wikipedia.org/wiki/Diminishing_returns I can learn many new things for fun but I already know a lot of things I'm fond of, which one should I give away given that time is finite? I didn't know anything when I was little and much less than now when I was half my age. Furthermore there are a few things I like and I became reasonably good at, professionally or for fun. I won't give them away unless I have to.
- ImaCake 6y ago
  I think the multi-armed bandit problem would assume that the probability distribution, for each choice, would remain a constant for the duration of the problem. Diminishing returns would imply that some subset of those choices would have decreasing probability of a positive pay-off the more they are used. This would be more realistic, and much harder to model. With the constant probabilities with the payoff being for a single score, it makes sense to just search for the highest expected value score. But that expected value relies on the probability distribution remaining the same!
- kokey 6y ago
  Perhaps it's just a habit I've stumbled into because of ADHD, but I find that in the long run choosing to spend time on learning something very different doesn't feel like a zero sum game. It's because the different new things to learn compliment the old things and you can leverage that.
jdsalaro 6y ago
I love contributions like yours and pmontra's, I really do. It's a huge hobby of mine to find, read about, correlate, compare, distill and share similar, and at times not so similar, concepts from different industries, disciplines and ages. I really don't know why it's so, so, satisfying to me. Probably I am on the more extreme end of the spectrum, as I assume most HN readers are, of the dopamine-driven reward of information, which brings me to the link and quite I wanted to share ;) https://www.frontiersin.org/articles/10.3389/fnhum.2013.00762/full?utm_source=newsletter&utm_medium=web&utm_campaign=Neuroscience-w49-2013 https://www.frontiersin.org/articles/10.3389/fnhum.2013.0076... > The general function of dopamine is to promote exploration, by facilitating engagement with cues of specific reward (value) and cues of the reward value of information (salience). This theory constitutes an extension of the entropy model of uncertainty (EMU; Hirsh et al., 2012), enabling EMU to account for the fact that uncertainty is an innate incentive reward as well as an innate threat.
kubanczyk 6y ago
OP says that it's possible to use medications to shift towards "explore". Are you implying that the true "exploit" phase (for a major part of life) is more desirable? Someone had managed to learn Borland Delphi in the first 50 years of their life, has been mostly exploiting Delphi for 20 years, they would be 70 now. This example closely matches the statistical model that you are proposing. Originally I wanted to post a strong critique, but now I wonder about the actual real life of such Delphi graybeard. Looks somewhat tempting.
- sbierwagen 6y ago
  I imply nothing. Must everyone harbor a shadowy agenda? The math does not care. For some number of slot machines, for some variance of payout, there is a single optimal explore-exploit strategy. If there are two slot machines, then a optimal agent does very little exploring. If there are a million slot machines, almost all with a $1 payout, but one with a $1 trillion payout, then you are inclined to do more exploring.[1] Your disagreement is not with the theorem, but the shape of the distribution of slot machines. And, of course, that distribution changes over time. For most of human history there either was no technological progress or it was imperceptible across a lifetime. Our current riot of ever-changing software tools is unprecedented. And, as hard as it is to believe when we're in the thick of it, eventually computing will become a mature technology. A century from now, you can be sure there will be fewer programming languages in use, and they will be more similar to each other. If humans are still programming, the Delphi graybeard will be the norm, not the exception. The only thing that would force our current milieu to be permanent is if computing technology is inherently vulnerable to Red Queen's race-style situations, where platforms have to constantly mutate to stay ahead of attackers. -- 1: This is the actual useful takeaway from this submission, not the text from the article, which is the usual "science news" trash.

pmontra 6y ago

Looks like a variation of the Law of Diminishing Returns https://en.wikipedia.org/wiki/Diminishing_returns https://en.wikipedia.org/wiki/Diminishing_returns I can learn many new things for fun but I already know a lot of things I'm fond of, which one should I give away given that time is finite? I didn't know anything when I was little and much less than now when I was half my age. Furthermore there are a few things I like and I became reasonably good at, professionally or for fun. I won't give them away unless I have to.

ImaCake 6y ago
I think the multi-armed bandit problem would assume that the probability distribution, for each choice, would remain a constant for the duration of the problem. Diminishing returns would imply that some subset of those choices would have decreasing probability of a positive pay-off the more they are used. This would be more realistic, and much harder to model. With the constant probabilities with the payoff being for a single score, it makes sense to just search for the highest expected value score. But that expected value relies on the probability distribution remaining the same!
kokey 6y ago
Perhaps it's just a habit I've stumbled into because of ADHD, but I find that in the long run choosing to spend time on learning something very different doesn't feel like a zero sum game. It's because the different new things to learn compliment the old things and you can leverage that.

jdsalaro 6y ago

I love contributions like yours and pmontra's, I really do. It's a huge hobby of mine to find, read about, correlate, compare, distill and share similar, and at times not so similar, concepts from different industries, disciplines and ages. I really don't know why it's so, so, satisfying to me. Probably I am on the more extreme end of the spectrum, as I assume most HN readers are, of the dopamine-driven reward of information, which brings me to the link and quite I wanted to share ;) https://www.frontiersin.org/articles/10.3389/fnhum.2013.00762/full?utm_source=newsletter&utm_medium=web&utm_campaign=Neuroscience-w49-2013 https://www.frontiersin.org/articles/10.3389/fnhum.2013.0076... > The general function of dopamine is to promote exploration, by facilitating engagement with cues of specific reward (value) and cues of the reward value of information (salience). This theory constitutes an extension of the entropy model of uncertainty (EMU; Hirsh et al., 2012), enabling EMU to account for the fact that uncertainty is an innate incentive reward as well as an innate threat.

kubanczyk 6y ago

OP says that it's possible to use medications to shift towards "explore". Are you implying that the true "exploit" phase (for a major part of life) is more desirable? Someone had managed to learn Borland Delphi in the first 50 years of their life, has been mostly exploiting Delphi for 20 years, they would be 70 now. This example closely matches the statistical model that you are proposing. Originally I wanted to post a strong critique, but now I wonder about the actual real life of such Delphi graybeard. Looks somewhat tempting.

sbierwagen 6y ago
I imply nothing. Must everyone harbor a shadowy agenda? The math does not care. For some number of slot machines, for some variance of payout, there is a single optimal explore-exploit strategy. If there are two slot machines, then a optimal agent does very little exploring. If there are a million slot machines, almost all with a $1 payout, but one with a $1 trillion payout, then you are inclined to do more exploring.[1] Your disagreement is not with the theorem, but the shape of the distribution of slot machines. And, of course, that distribution changes over time. For most of human history there either was no technological progress or it was imperceptible across a lifetime. Our current riot of ever-changing software tools is unprecedented. And, as hard as it is to believe when we're in the thick of it, eventually computing will become a mature technology. A century from now, you can be sure there will be fewer programming languages in use, and they will be more similar to each other. If humans are still programming, the Delphi graybeard will be the norm, not the exception. The only thing that would force our current milieu to be permanent is if computing technology is inherently vulnerable to Red Queen's race-style situations, where platforms have to constantly mutate to stay ahead of attackers. -- 1: This is the actual useful takeaway from this submission, not the text from the article, which is the usual "science news" trash.