4 ms·

The question reminds me of the dilemma between exploration and exploitation [1]. The art (or skill) is learning when to stop and redirect. Talking to friends ab

The question reminds me of the dilemma between exploration and exploitation [1]. The art (or skill) is learning when to stop and redirect. Talking to friends about certain learning curves and making sure I don't waste my time just researching or building things that are overloaded with features has helped me a lot. I think it's generally good to know that there's a tradeoff no matter what you're trying to learn or explore. [1] https://en.wikipedia.org/wiki/Exploration-exploitation_dilemma https://en.wikipedia.org/wiki/Exploration-exploitation_dilem...

chatmasta 3y ago
This is my second time seeing a reference to this concept today, but apparently the Wikipedia article was created in 2023. It's an interesting idea though... did it come from machine learning research? I remember learning something akin to this in social psychology in the context of a single risk-taker fish breaking from the school of fish to explore and take risks which ultimately benefitted the whole group.
- hlfshell 3y ago
  The terminology is widely used, and has been for decades, in reinforcement learning; so you're close. The general idea is that your agent, however it is training, has to balance trying new things to possibly find the global maxima instead of getting hooked on a rewarding local maxima. https://en.wikipedia.org/wiki/Reinforcement_learning#Exploration https://en.wikipedia.org/wiki/Reinforcement_learning#Explora...
- gryn 3y ago
  it's about a century old but the recent popularity of AI made more people aware of it, I personally became aware of it more than a decade while learning about reinforcement learning (the old kind with GOFAI) but it's also used in social fields, biology stuff like foraging theory or really anything that involve an agent with a brain. here an n-gram of the first mentions: https://books.google.com/ngrams/graph?content=Exploration-exploitation%2C+Exploration+exploitation&year_start=1800&year_end=2019&corpus=en-2019&smoothing=3 https://books.google.com/ngrams/graph?content=Exploration-ex...
- pininja 3y ago
  I remember discussing this idea with my friends 10 years ago, when I started my first engineering job and felt my time shifting from mostly exploration to exploitation. I had spent years learning for the sake of learning, with no strings attached. Then, all of a sudden, I was applying what I had learned in exchange for money, and the people I worked for expected something valuable to come from how I spent my time. I didn’t research machine learning, but my friends did, so it could have been an idea I picked up from someone else. Anyway, I think it’s a valuable framework because we need to make time for exploration—it’s a great way to let go, have fun, and dissolve the fear of failure.
Hbruz0 3y ago
Isn't this just breadth- vs depth-first search ?
- hansvm 3y ago
  Not really. You're making the choice to explore or exploit at each branch. It's closer to an A-star search, where the weights and heuristics are uncovered over time. Also, the typical framing of the problem is the same "kind" of choice being repeatedly executed (e.g., betting on a coin-flip of unknown bias, or balancing the gain of consumer purchasing information vs exploiting known information when setting items into aisle end-caps in a grocery store). That has a lot more structure than arbitrary graphs, enough so to make it worthy of its own dedicated study (especially given the real-world applicability).

chatmasta 3y ago

This is my second time seeing a reference to this concept today, but apparently the Wikipedia article was created in 2023. It's an interesting idea though... did it come from machine learning research? I remember learning something akin to this in social psychology in the context of a single risk-taker fish breaking from the school of fish to explore and take risks which ultimately benefitted the whole group.

hlfshell 3y ago
The terminology is widely used, and has been for decades, in reinforcement learning; so you're close. The general idea is that your agent, however it is training, has to balance trying new things to possibly find the global maxima instead of getting hooked on a rewarding local maxima. https://en.wikipedia.org/wiki/Reinforcement_learning#Exploration https://en.wikipedia.org/wiki/Reinforcement_learning#Explora...
gryn 3y ago
it's about a century old but the recent popularity of AI made more people aware of it, I personally became aware of it more than a decade while learning about reinforcement learning (the old kind with GOFAI) but it's also used in social fields, biology stuff like foraging theory or really anything that involve an agent with a brain. here an n-gram of the first mentions: https://books.google.com/ngrams/graph?content=Exploration-exploitation%2C+Exploration+exploitation&year_start=1800&year_end=2019&corpus=en-2019&smoothing=3 https://books.google.com/ngrams/graph?content=Exploration-ex...
pininja 3y ago
I remember discussing this idea with my friends 10 years ago, when I started my first engineering job and felt my time shifting from mostly exploration to exploitation. I had spent years learning for the sake of learning, with no strings attached. Then, all of a sudden, I was applying what I had learned in exchange for money, and the people I worked for expected something valuable to come from how I spent my time. I didn’t research machine learning, but my friends did, so it could have been an idea I picked up from someone else. Anyway, I think it’s a valuable framework because we need to make time for exploration—it’s a great way to let go, have fun, and dissolve the fear of failure.

Hbruz0 3y ago

Isn't this just breadth- vs depth-first search ?

hansvm 3y ago
Not really. You're making the choice to explore or exploit at each branch. It's closer to an A-star search, where the weights and heuristics are uncovered over time. Also, the typical framing of the problem is the same "kind" of choice being repeatedly executed (e.g., betting on a coin-flip of unknown bias, or balancing the gain of consumer purchasing information vs exploiting known information when setting items into aisle end-caps in a grocery store). That has a lot more structure than arbitrary graphs, enough so to make it worthy of its own dedicated study (especially given the real-world applicability).