3 ms·

This is my second time seeing a reference to this concept today, but apparently the Wikipedia article was created in 2023. It's an interesting idea though... di

This is my second time seeing a reference to this concept today, but apparently the Wikipedia article was created in 2023. It's an interesting idea though... did it come from machine learning research? I remember learning something akin to this in social psychology in the context of a single risk-taker fish breaking from the school of fish to explore and take risks which ultimately benefitted the whole group.

hlfshell 3y ago
The terminology is widely used, and has been for decades, in reinforcement learning; so you're close. The general idea is that your agent, however it is training, has to balance trying new things to possibly find the global maxima instead of getting hooked on a rewarding local maxima. https://en.wikipedia.org/wiki/Reinforcement_learning#Exploration https://en.wikipedia.org/wiki/Reinforcement_learning#Explora...
gryn 3y ago
it's about a century old but the recent popularity of AI made more people aware of it, I personally became aware of it more than a decade while learning about reinforcement learning (the old kind with GOFAI) but it's also used in social fields, biology stuff like foraging theory or really anything that involve an agent with a brain. here an n-gram of the first mentions: https://books.google.com/ngrams/graph?content=Exploration-exploitation%2C+Exploration+exploitation&year_start=1800&year_end=2019&corpus=en-2019&smoothing=3 https://books.google.com/ngrams/graph?content=Exploration-ex...
pininja 3y ago
I remember discussing this idea with my friends 10 years ago, when I started my first engineering job and felt my time shifting from mostly exploration to exploitation. I had spent years learning for the sake of learning, with no strings attached. Then, all of a sudden, I was applying what I had learned in exchange for money, and the people I worked for expected something valuable to come from how I spent my time. I didn’t research machine learning, but my friends did, so it could have been an idea I picked up from someone else. Anyway, I think it’s a valuable framework because we need to make time for exploration—it’s a great way to let go, have fun, and dissolve the fear of failure.

hlfshell 3y ago

The terminology is widely used, and has been for decades, in reinforcement learning; so you're close. The general idea is that your agent, however it is training, has to balance trying new things to possibly find the global maxima instead of getting hooked on a rewarding local maxima. https://en.wikipedia.org/wiki/Reinforcement_learning#Exploration https://en.wikipedia.org/wiki/Reinforcement_learning#Explora...

gryn 3y ago

it's about a century old but the recent popularity of AI made more people aware of it, I personally became aware of it more than a decade while learning about reinforcement learning (the old kind with GOFAI) but it's also used in social fields, biology stuff like foraging theory or really anything that involve an agent with a brain. here an n-gram of the first mentions: https://books.google.com/ngrams/graph?content=Exploration-exploitation%2C+Exploration+exploitation&year_start=1800&year_end=2019&corpus=en-2019&smoothing=3 https://books.google.com/ngrams/graph?content=Exploration-ex...

pininja 3y ago

I remember discussing this idea with my friends 10 years ago, when I started my first engineering job and felt my time shifting from mostly exploration to exploitation. I had spent years learning for the sake of learning, with no strings attached. Then, all of a sudden, I was applying what I had learned in exchange for money, and the people I worked for expected something valuable to come from how I spent my time. I didn’t research machine learning, but my friends did, so it could have been an idea I picked up from someone else. Anyway, I think it’s a valuable framework because we need to make time for exploration—it’s a great way to let go, have fun, and dissolve the fear of failure.