20 ms·

Young children would rather explore than get rewards

698 points by kyle_morris_ 6y ago

RcouF1uZ4gsC 6y ago
One thing neither this article nor the linked study measure is if there is a difference in this behavior between boys and girls. I looked through the actual article to see if there was a statement that said if there was a statistical difference between the two or not, but although they do specify the numbers for the genders of the participants in the study, none of the analysis does any comparison between the genders. It would be fascinating to see if there are gender differences in this behavior in childhood and adulthood and compare childhood and adulthood.
- SftwreEngnr 6y ago
  Did you mean to say sex, not gender?
- habitue 6y ago
  With only 37 adults, there probably wasn't enough power to say anything significant about gender
  - RcouF1uZ4gsC 6y ago
    But even then a statement, “there was no difference between genders, p=...” would have been useful
    - jiggunjer 6y ago
      Not really. Even if there was was medium sized true difference they'd probably not detect it.
- renewiltord 6y ago
  They weren't testing that hypothesis, so that's why they didn't do this. If you go back and torture the data like this, you'll get acne from green jellybeans.
  - jiggunjer 6y ago
    Not if you correct for multiple comparisons.
kqr 6y ago
For anyone interested in this type of stuff, I can recommend reading Drive. Briefly, human behaviour can be motivated for extrinsic and intrinsic reasons. The rewards in this article are extrinsic motivators. Exploring is rewarding in an intrinsic way; it is a strive towards autonomy, mastery, and purpose. In adults (and in older children) applying extrinsic motivators kills intrinsic motivation. Once the extrinsic motivators stop coming in, there is no desire left to do the task. Intrinsic motivation is practically infinite, as long as the environment is set up right to enable it. Extrinsic motivation also tends to produce behaviour that does the bare minimum to get the reward (or avoid the negative consequences) whereas intrinsic motivation is what makes us want to excel. Of course, I've skipped many important points and not countered any counterargument here, but I recommend reading Drive first if you think you disagree. But the worst part of it all? The schooling system, with its grades, signed slips, and whatnot, is set up through extrinsic motivation to teach obedience, conformity, and smothering the intrinsic drive so necessary for the creative work we will expect from the children later in life.
- andrei_says_ 6y ago
  Also Alfie Kohn’s Punished by Rewards, a thick tome dedicated to the research and reasons of how and why punishment and rewards don’t teach what’s intended and don’t motivate as intended... and how extrinsic motivation kills intrinsic motivation. Withdrawing rewards is punishment so both are really the same. Human beings are not designed to be trained like rats and birds, regardless of what many believe. If your kid is good at something and you want them to lose interest, encourage or reward them for doing the thing :) Or force them to do it. Just like school system does with learning.
  - tpxl 6y ago
    >If your kid is good at something and you want them to lose interest, encourage or reward them for doing the thing :) Genuine question: What do you do if you want them to get better then?
    - kqr 6y ago
drewcoo 6y ago
The "exploring" is because they're told to make a choice. Is there something similar to developing object permanence but with abstract concepts or probabilities that happens after that age? And the abstract is here: https://onlinelibrary.wiley.com/doi/10.1111/desc.13026 https://onlinelibrary.wiley.com/doi/10.1111/desc.13026
op03 6y ago
Academia and Corporations. Academics are dealt with as we do kids i.e. they are protected and allowed to explore. Corporations exploit what has already been explored for reward - without the reward there is nothing protecting them.
- adamnemecek 6y ago
  Academia really doesn't let you explore.
  - asdff 6y ago
    Depends on your advisor.
    - randomsearch 6y ago
      You don’t have an “advisor” for most of an academic career
    - bart_spoon 6y ago
      Academia isn't being a grad student with an advisor. Academia is being the advisor to a grad student (along with research, publishing, probably teaching, etc).
- tolbish 6y ago
  Academia was like that before the turn on the century. I don't know too many people in academia who would agree with you now. Academia is the same cut-throat pursuit for funding as industry, but with fewer options.
jimkleiber 6y ago
TIL I mostly think like a 4-year-old.
- david422 6y ago
  Very mature for a 3 year old.
  - jimkleiber 6y ago
    What can I say? #blessed
brnt 6y ago
I guess I must have bever lost the child in me. I've been a fan of open world games, just so I can roam the world. I sometimes dont even play the storyline at all. I still don't like that Steam added 'achievements' that I can't turn off, I'll be in charge of what constitutes a goal thank you very much :) Another poster had an interesting observation about academia vs corporate work. I'm 33 and still in academia, and I now realize a large part of that is the freedom it provides. I've see salary as a score on someone else's scoreboard, and very uninteresting for that reason. OK, I'm old enough to have experienced a person needs income for practical reasons, but I'm not going to work for it ;) I'm allergic to competition, scores, rewards, prestige, I guess that makes me child-like?
- mamurphy 6y ago
  >I'm 33 and still in academia, I'm allergic to competition, scores, rewards, prestige I had been under the impression that academia was mostly very competitive to be able to get career (tenure track) positions. Are you not aiming for that, is it less competitive in your field, or am I missing something else?
  - brnt 6y ago
    Indeed, I moved into a supportive role (software dev, what else) for that reason. Tenure is worse than corporate.
  - dguest 6y ago
    I've described academia (to my friends who work in software) as a company where the most coveted promotion will land you a job in marketing. It does indeed come with a lot of freedom, but the relatively flat structure means that there's usually no one above the group's PI to promote their research. As a result PIs spend a lot of time writing grant proposals.
  - cycomanic 6y ago
    Yes academia is very competitive, even after you get tenure you still need to write grants all the time to be able to do your research.
- 6y ago
sporkologist 6y ago
Exploring is its own reward apparently.
ta1234567890 6y ago
This video shows a slime mold doing exploration/discovery and then exploitation: https://m.youtube.com/watch?v=GwKuFREOgmo https://m.youtube.com/watch?v=GwKuFREOgmo Relating it to the article, it seems we function in a similar way to the mold, but our balance between exploration and exploitation changes as we grow up. Also reading other comments, it seems like different people have different balances or proportions between the exploration/exploitation modes. I personally am very imbalanced towards exploration; exploitation makes me demotivate and stop doing very quickly.
twowatches 6y ago
Science proving things that people already know anyway, as usual.
bergstromm466 6y ago
Has anyone heard of Sudbury Valley schools? Peter Hartkamp started one in the Netherlands, and wrote a book about it. “Our current education systems do not trust children to learn what they need to know. Governments believe that children must be coerced into learning what they prescribe as necessary for future life even though they do not and cannot know what that future will be like. Yet we do know that coercion produces anxiety and fear of failure and that this in turn inhibits learning and destroys confidence. We also know that creativity, innovation and empathy are not encouraged in such a climate even though above all else these are the survival qualities for coping with an uncertain future. In this short book Peter Hartkamp develops his arguments against coercive ‘education’ with a needle-sharp engineer’s logic. He invites us to imagine what schools would be like if they took the articles of the UN Convention on the Rights of the Child seriously, a not unreasonable position as all states are signatories. I wholeheartedly and without reservation recommend this book to the many parents, students, teachers, employers and policy makers who know in their hearts that something is very wrong with the examination and testing factories that we are allowing our schools to become. It represents a beacon of hope that another way is possible.” [1] [1] http://hetgedwongenonderwijsvoorbij.nl/en/ http://hetgedwongenonderwijsvoorbij.nl/en/
- jkhdigital 6y ago
  Yes, and I’ve been fantasizing about sending my child to one since before he was born. Unfortunately I don’t live near any :-(
fxtentacle 6y ago
While the study classifies it as exploration and claims an intrinsic reward that will die out later in life, I would describe the behavior as novelty-seeking. I'd wager that for adults, a game on a screen is just not new enough. But if you'd give adult males a choice of which female to undress, I'm pretty sure they would value "exploration" very highly again. In a similar vain, when smartphones were new, the adult population was very active at buying new ones, comparing, and exploring the market. Now that we've had them for some years, phone upgrades have become a necessity and a lot less exciting. So my theory would be that there is no loss in the intrinsic motivation, there is a loss in the amount of novelty. If that is correct, then our school system is actually very appropriate even for highly intrinsically motivated kids, because they are introduced to new topics and areas of knowledge that are (hopefully) new to them. That also aligns with my personal experience. While I hated getting up early and the concept of sitting in a chair just listening, I was excited about all the different things they showed me. And I liked best the teachers that would mention slightly irrelevant side details just for the fun of it.
- stinos 6y ago
  But if you'd give adult males a choice of which female to undress, I'm pretty sure they would value "exploration" very highly again Likely, but that's not the same experiment anymore since the sex factor comes into play and living beings are basically programmed to reproduce as much as possible and also with as much variety as possible (maybe oversimplified, but you get the point). So for a smartphone you can talk abut novelty-seeking/exploration (which btw I'm not sure are really different things), but when reproduction comes into play it's a whole different game anymore and it'll alter/override parts of the behavior and it shouldn't be underestimated how string the effectss can be.
  - nickelpro 6y ago
    Sure but you're missing OP's point. They're saying that novelty is the fundamental drive behind the observed behavior, not some priority towards "exploration" that children have in greater abundance than adults. The children are seeking pleasure through novelty, and the researchers are interpreting that as exploration. The parent comment illustrates how silly that is, if you substitute something that still holds pleasure for an adult (ie sex), they exhibit the same behavior. The difference between the adult and the child is the adult has the world experience that the trivial game doesn't hold novel (therefore pleasurable) implications for them anymore, not that they don't prioritize "exploration".
danschumann 6y ago
Read: exploration is rewarding.
- amatic 6y ago
  That's what I get, too. A "reward" is a subjective experience. If someone doesn't care about grades, then good grades will not be rewarding, and bad grades will not be punishment.
OneGuy123 6y ago
Offtopic but regarding the supression of the child from the parent: few people realize how severely the large majority of parents damage their children and supress their innate creativity and drive. This is also very hard to realize as an adult because it gets unconsiously supressed. The following books opened my eyes at the fact how dangerous it is for the grown up adult to then keep thinking "yeah, but my parents did the best you could you know?" even though they completely destroyed his innate creativity and freedom. Alice Miller: The Drama of the Gifted Child Alice Miller: The Body Never Lies TLDR version of these books is that a large majority of parents tried to mold them as children and fit them into boxes or supressed their needs and wants as children and never truly listened to them. This then causes uncousious guilt/anxiety/etc... feeling in the person as adult and prevents him from "taking oportunities" etc.. And at the core of the issue is the forced-upon-us belief that we always have to respect our parents, even though we secretely hate them. Saying "yeah but they did something good for me you know" is actually a huge problem that will only keep us from seeing the fact that many in fact do not like/love their parents very much because the parents did not truly listen to them etc... As long as we try to weight the good vs the bad moments we will relapse into the false indoctrinated compassion/guilt towards our parents, into denial of the cruelties we experienced as a child, all because people say that we should try to "balance the good vs the bad". But this does not work in practice. This balancing process seems to be a reflection of how we tried to cope as children to survive. The adult must reject this balancing process because it only causes confusion without resolving anything. Many people when they break contact from parents feel guilty and think even more about their own parents: the "experts" and her parents have reinforced the indoctrinated belief in this person that "this individual has no right to his own life, feelings and needs." It was this desire of the child to believe that the parents were good that led to these supressions because the child blamed himself. This is why when a therapists says "try to see the good" it will only cause great harm. regression and confusion: since it was this need of the child to "try and see the good" in his parents when there was none that caused these issues in the first place. So anyone who even slightly suspects that their parents supressed him as a child and denied him freedom of expression etc... will find many answers in those books above.
toomanybeersies 6y ago
Reading between the lines, as we grow up, we get conditioned to follow instructions, rather than act independently. For the adults, there's no incentive to maximise how many virtual candies they collect, beyond doing it because they were told it was the aim.
anigbrowl 6y ago
This is by no means limited to young people; I much prefer video games where I can keep discovering things to the tedious business of performing tasks or winning beating bosses.
- PikachuEXE 6y ago
  Yes I prefer games with good stories I am having fun discovering the plots bit by bit
dzink 6y ago
Exploration as the primary algorithm continues as long as we continue to get increasing variable rewards from it. When the rewards tapper off or we are pressured for money, we choose the maximum we've found so far. This instinct to collect options has probably served us well in foraging for food, picking a mate, and any other exploration endeavor that involves finding the best of a range of newly available options. However it may also be the root of gambling addiction and any other variable rewards game systems. Gambling, lotteries can also become more attractive the more you are pressured for money - one could argue as a means of concentrating on your most approachable local maximum.
cycomanic 6y ago
For people interested in this topic I highly recommend reading about summerhill school [1] and similar projects. The topic of alternative schooling systems is quite fascinating. [1] https://en.m.wikipedia.org/wiki/Summerhill_School https://en.m.wikipedia.org/wiki/Summerhill_School
- messo 6y ago
  For a US-based variation of democratic schooling, check out the Sudbury Valley School: https://en.m.wikipedia.org/wiki/Sudbury_Valley_School https://en.m.wikipedia.org/wiki/Sudbury_Valley_School
- qntty 6y ago
  The best book on education that I've ever read references Summerhill several times (Compulsory Miseducation by Paul Goodman).
flooo 6y ago
There is an interesting relation to this with regard to reinforcement learning (RL) where the trade-off between exploration and exploitation is one of the fundamental issues. Systematic or structured exploration has shown some success here as well. What 'structured' means here deeply depends on the representation the agent has of the world, i.e. which states and actions are similar to each other. Current RL setups typically learn such representations on the go. Abstractions can be very powerful here. Arguably, people also perform structured exploration on learned representations. It may be the case that the adults have learned a representation of the 'game' and their lies little exploratory value in suboptimal actions.
Razengan 6y ago
Still holds true as an adult for me at least. It’s just that “survival” takes away too much time and energy from exploration.
rv-de 6y ago
> But while adults then used that knowledge to maximize their prizes, children continued exploring the other options, just to see if their value may have changed. I think it is not just "semantics" to consider that for children the reward is simply the rush of playing instead of what the researchers design as the intended award. And frankly speaking this is not at all a surprising result. Children learn through playing and they very much enjoy that process.
irjustin 6y ago
Overall, this style is what Montessori-schools of teaching believe and how they operate. Essentially a self-exploratory based learning style. There's lots of benefit in letting kids simply explore. As a parent, I find I have to stop myself from pushing my child to the 'best' result and let them be. That it's okay to pick a less optimal solution. Where I find this alarmingly true is when outcome/results are negative (usually ending in pain). "Don't stand like that on the chair." "Climb in this way" "Don't eat that dirt, soap, car, whatever!!" I know my job as a parent is to guide while keeping them safe at the same time letting exploration happen, but man sometimes it's just so hard. For example, letting my child explore while walking home can make a 5 minute walk take 10, 15 minutes longer. Such things tend to be at odds with what I want but it is more important than myself.
- macspoofing 6y ago
  >Where I find this alarmingly true is when outcome/results are negative (usually ending in pain) Pain is a very good teacher. Letting children hurt themselves and learn from that (within reason, obviously) is not a bad thing.
  - chmod775 6y ago
    Especially since while you're small, falling hurts a lot less. It's surprising to me how people consistently try to discourage traits in children which they at the same time glorify in adults. I suppose considering how much effort society puts into raising meager people, it's only right we glorify those who manage to manage to grow as a person regardless.
    - germinalphrase 6y ago
      I’ve also noticed that my toddler falls “better” than most adults do. If you ever take up judo, early on they need to teach you to fall correctly so you avoid injury. My toddler does most of it be instinct which is cool.
      - 6y ago
pul 6y ago
This has surprising similarities with data science theory: multi-armed bandit with a decreasing-epsilon strategy (children) or an epsilon-first strategy (adults).
082349872349872 6y ago
Anyone have access to the paper? As far as I can tell, the young children were perfect convex utilitarians, in deciding that more than "enough" stickers were not worth the more fun of clicking around. How were the adults rewarded? Did they also get something intrinsically valuable at the end of the experiment, or did they just collect virtual points because they were told they were supposed to? Working dogs have been specialised by breeding for overexpression of the various initial phases of the wildtype hunting sequence: herding, retrieval, etc. Does "sex sells" indicate we may have been bred to overexpress the "identify attractive potential partner" and "do things they (appear to) like doing" phases of the wildtype reproductive sequence?
wcoenen 6y ago
I'm reminded of a paper about an AI system with curiosity. Basically, the AI would learn to predict the effect of its actions, and would prefer to take actions where its prediction error was large. That way it would gather more data about stuff it didn't know yet. This works well to learn to play video games without defining an extrinsic reward. However, there is a problem when the system encounters the equivalent of TV. It then gets stuck watching the unpredictable stream of events. paper and code: https://pathak22.github.io/large-scale-curiosity/ https://pathak22.github.io/large-scale-curiosity/ Coverage by "2 minute papers" youtube channel: https://youtu.be/fzuYEStsQxc https://youtu.be/fzuYEStsQxc
trombonechamp 6y ago
Monkeys do the same thing: Many non-human animals show exploratory behaviors. It remains unclear whether any possess human-like curiosity. We previously proposed three criteria for applying the term curiosity to animal behavior: (1) the subject is willing to sacrifice reward to obtain information, (2) the information provides no immediate instrumental or strategic benefit, and (3) the amount the subject is willing to pay depends systematically on the amount of information available. In previous work on information-seeking in animals, information generally predicts upcoming rewards, and animals’ decisions may therefore be a byproduct of reinforcement processes. Here we get around this potential confound by taking advantage of macaques’ ability to reason counterfactually (that is, about outcomes that could have occurred had the subject chosen differently). Specifically, macaques sacrificed fluid reward to obtain information about counterfactual outcomes. Moreover, their willingness to pay scaled with the information (Shannon entropy) offered by the counterfactual option. These results demonstrate the existence of human-like curiosity in non-human primates according to our criteria, which circumvent several confounds associated with less stringent criteria. Paper: https://www.sciencedirect.com/science/article/abs/pii/S0010027719300642 https://www.sciencedirect.com/science/article/abs/pii/S00100... Non-paywalled version: https://www.biorxiv.org/content/biorxiv/early/2018/03/29/291708.full.pdf https://www.biorxiv.org/content/biorxiv/early/2018/03/29/291...
- officehero 6y ago
  Doesn't this just mean that the "reward" definition is poor in this case i.e. the agent does not care about maximizing it because there are additional components left out of it (e.g. survival)?
sillypuddy 6y ago
Young people of any age
distant_hat 6y ago
This is completely expected. If you look at multiarmed bandits, getting rewards would be exploiting and the other option is exploring. If you have a longer time horizon as children will, it is better to explore and find a better return than exploit something already known which is likely to be suboptimal.
hunterx 6y ago
Some adults do as well, they are the ones who usually discover be things...
deleted 6y ago
[deleted]
gomes33 6y ago
For those interested in this topic i recommend reading about the Montessori method. The basic idea of the Montessori philosophy of education is that every child carries within him/her the man/woman he/she will become. In order to develop the physical, intellectual, and spiritual powers required for this task to the fullest, there must be freedom—freedom to interact with a prepared environment.
- andrekandre 6y ago
  along those lines, seymour papert talked about imersion being necessary for students to learn in thier own way, and for teachers to be not just instructors but guides who also share and learn with the students i guess said more simply, its learning by doing (go to france if you wanna learn french, or at least make a mini-france in your classroom)
known 6y ago
Interesting; Wish adults do the same;
jkhdigital 6y ago
Seems like further evidence that nature has developed an optimal algorithm for solving the multi-armed bandit problem as it manifests in the typical environment humans evolved in. Unlike the artificial games used by the researchers, there is no such thing as a fixed reward in nature—everything is subject to change unexpectedly. Children are wired with an algorithm that heavily favors exploration in light of the dynamic environment.
eloff 6y ago
We use a variation of this "four squares" game at Sparkademy. The four squares of randomized point values (there's a pattern to it, but I don't want to give it away here.) The goal is to maximize your score which involves switching between exploring for a better payoff and exploiting what you think is the highest paying square over a large number of trials. We use it to as part of a criteria to identify if a person might be well suited for our innovation online course, and later be able to apply that knowledge in their company.
sjg007 6y ago
Man I thought this was obvious!
ddingus 6y ago
As a kid, exploring was, and remains a reward. Someone to help, share in it, make more possible? Golden.
- PikachuEXE 6y ago
  And this makes me think of Minecraft (on multiplayer servers)
ncr100 6y ago
What does this say about Capitalism and the accumulation of wealth?
Aviatore 6y ago
If OP's premise is true, I wonder how much that is changing now that parents are throwing electronics at their children. Many of the things that they get to interact with on an electronic aims to optimize rewards. Will this shape generations that are always in search of quick hits and less interested in the exploratory nature of things? How will something that seems to small as a parent reshape the children and society?
- GuB-42 6y ago
  I don't think so. I suspect the "exploration" bias observed in that study is somewhat hardwired into the brain. Kids need to learn, adults need to be efficient. The test was clearly designed as a way to optimize reward. And indeed, adults quickly went to the optimal solution. But kids kept exploring, even if further tests show that they knew the optimal solution.
kingkawn 6y ago
That’s why we train them to want rewards first, then use that to get them to do what we want
blobbers 6y ago
Isn't this the children solving the multi armed bandit problem? In a child's mind, they haven't yet fully decided how things behave and react, so they are willing to try things to see if they've changed (because in fact often they do in their own mental models, as well as the resolution of those models. If you think of it as a regression tree, their models start out fairly shallow and then slowly get deeper as they age. That's a square thing. That's a book. That's a book about animals. That's a book about marine animals. That's a book about marine animals that I like. That's a book about whales. In trying to solve the multi armed bandit, they're willing to explore to try to find better rewards because in their own world there are often better rewards. If the game shifted to having a different alien be the best one half way through the game, I'd be curious how quickly the more maturely cognitive kids did. It's possible that becoming fixated on something (the way adults are) can ultimately limit your abilities in a game. In adulthood, "creative" solutions can sometimes be better than more "standard" solutions, but incur more risk.
- mam2 6y ago
  Yes its called "simulated annealing for your own life". In 3 days hacker news will discover GANs and make a shocking headline like "challenges and adversity makes you better / stronger". #mindblown
  - blobbers 6y ago
    Haha. Don't blame hackernews! This is social science encounters real research ;)
  - weatherman2 6y ago
    I'm going to read my children a picture book titled "Scraggles the Puppy Explores the Forest of Simulated Annealing."
zackmorris 6y ago
Well ya. I've found at middle age that I'm most interested in connection and transcendence, finding meaning and such. When one realizes that material things truly are valueless compared to things like consciousness, friendship, love, dignity, etc etc etc, then everything else falls away. I often find myself realizing that I am looking at the world with the same childlike wonder I had when I was 5. To me, this search for meaning was what the 1960s was all about, as well as every other cultural renaissance and era of spiritual awakening. Which makes the conceptual basis of things like economics suspect. The idea that humans are motivated by monetary rewards and material things turned out to be false. So did concepts like scarcity, reciprocity, commoditization, and so on. Those things worked at one time, up until the middle of the last century and dawn of the information age (where technology surpassed what was required to meet humanity's basic needs), but now actively inhibit human evolution IMHO. So how to fix the mess we're in? I think it starts with questioning basic assumptions. Sadly, American culture seems to be racing away from that as fast as it can in these times. Dunno about the rest of the world.
- thefucnjosh 6y ago
  It's been known for years that oftentimes paying people more for intellectually demanding tasks, leads to decreases in productivity Capitalism and the ideologies surrounding it, fall apart when you start looking at real assets and how the exchanges value of those assets takes a hit during economic down turns. The idea that "humans [were] motivated by monetary rewards and material things" didn't turn out to be false, it's just always been the case that it wasn't true.
  - pg_1234 6y ago
    There's converse to this however, when you charge a company more they're less inclined to waste your time.
- Florin_Andrei 6y ago
  It's just a matter of optimal strategies. When you're young, take risks, explore, maybe fail on occasion. There's still time to fix all that. When you're old, go for the more likely reward. If you screw up, that's how it stays, forever.
ZainRiz 6y ago
I can attest to this. I tried offering my toddler a thousand dollars. She preferred to blow bubbles.

698 points by kyle_morris_ 6y ago

One thing neither this article nor the linked study measure is if there is a difference in this behavior between boys and girls. I looked through the actual article to see if there was a statement that said if there was a statistical difference between the two or not, but although they do specify the numbers for the genders of the participants in the study, none of the analysis does any comparison between the genders. It would be fascinating to see if there are gender differences in this behavior in childhood and adulthood and compare childhood and adulthood.

SftwreEngnr 6y ago
Did you mean to say sex, not gender?
habitue 6y ago
With only 37 adults, there probably wasn't enough power to say anything significant about gender
- RcouF1uZ4gsC 6y ago
  But even then a statement, “there was no difference between genders, p=...” would have been useful
  - jiggunjer 6y ago
    Not really. Even if there was was medium sized true difference they'd probably not detect it.
renewiltord 6y ago
They weren't testing that hypothesis, so that's why they didn't do this. If you go back and torture the data like this, you'll get acne from green jellybeans.
- jiggunjer 6y ago
  Not if you correct for multiple comparisons.

kqr 6y ago

For anyone interested in this type of stuff, I can recommend reading Drive. Briefly, human behaviour can be motivated for extrinsic and intrinsic reasons. The rewards in this article are extrinsic motivators. Exploring is rewarding in an intrinsic way; it is a strive towards autonomy, mastery, and purpose. In adults (and in older children) applying extrinsic motivators kills intrinsic motivation. Once the extrinsic motivators stop coming in, there is no desire left to do the task. Intrinsic motivation is practically infinite, as long as the environment is set up right to enable it. Extrinsic motivation also tends to produce behaviour that does the bare minimum to get the reward (or avoid the negative consequences) whereas intrinsic motivation is what makes us want to excel. Of course, I've skipped many important points and not countered any counterargument here, but I recommend reading Drive first if you think you disagree. But the worst part of it all? The schooling system, with its grades, signed slips, and whatnot, is set up through extrinsic motivation to teach obedience, conformity, and smothering the intrinsic drive so necessary for the creative work we will expect from the children later in life.

andrei_says_ 6y ago
Also Alfie Kohn’s Punished by Rewards, a thick tome dedicated to the research and reasons of how and why punishment and rewards don’t teach what’s intended and don’t motivate as intended... and how extrinsic motivation kills intrinsic motivation. Withdrawing rewards is punishment so both are really the same. Human beings are not designed to be trained like rats and birds, regardless of what many believe. If your kid is good at something and you want them to lose interest, encourage or reward them for doing the thing :) Or force them to do it. Just like school system does with learning.
- tpxl 6y ago
  >If your kid is good at something and you want them to lose interest, encourage or reward them for doing the thing :) Genuine question: What do you do if you want them to get better then?
  - kqr 6y ago

drewcoo 6y ago

The "exploring" is because they're told to make a choice. Is there something similar to developing object permanence but with abstract concepts or probabilities that happens after that age? And the abstract is here: https://onlinelibrary.wiley.com/doi/10.1111/desc.13026 https://onlinelibrary.wiley.com/doi/10.1111/desc.13026

op03 6y ago

Academia and Corporations. Academics are dealt with as we do kids i.e. they are protected and allowed to explore. Corporations exploit what has already been explored for reward - without the reward there is nothing protecting them.

adamnemecek 6y ago
Academia really doesn't let you explore.
- asdff 6y ago
  Depends on your advisor.
  - randomsearch 6y ago
    You don’t have an “advisor” for most of an academic career
  - bart_spoon 6y ago
    Academia isn't being a grad student with an advisor. Academia is being the advisor to a grad student (along with research, publishing, probably teaching, etc).
tolbish 6y ago
Academia was like that before the turn on the century. I don't know too many people in academia who would agree with you now. Academia is the same cut-throat pursuit for funding as industry, but with fewer options.

jimkleiber 6y ago

TIL I mostly think like a 4-year-old.

david422 6y ago
Very mature for a 3 year old.
- jimkleiber 6y ago
  What can I say? #blessed

brnt 6y ago

I guess I must have bever lost the child in me. I've been a fan of open world games, just so I can roam the world. I sometimes dont even play the storyline at all. I still don't like that Steam added 'achievements' that I can't turn off, I'll be in charge of what constitutes a goal thank you very much :) Another poster had an interesting observation about academia vs corporate work. I'm 33 and still in academia, and I now realize a large part of that is the freedom it provides. I've see salary as a score on someone else's scoreboard, and very uninteresting for that reason. OK, I'm old enough to have experienced a person needs income for practical reasons, but I'm not going to work for it ;) I'm allergic to competition, scores, rewards, prestige, I guess that makes me child-like?

mamurphy 6y ago
>I'm 33 and still in academia, I'm allergic to competition, scores, rewards, prestige I had been under the impression that academia was mostly very competitive to be able to get career (tenure track) positions. Are you not aiming for that, is it less competitive in your field, or am I missing something else?
- brnt 6y ago
  Indeed, I moved into a supportive role (software dev, what else) for that reason. Tenure is worse than corporate.
- dguest 6y ago
  I've described academia (to my friends who work in software) as a company where the most coveted promotion will land you a job in marketing. It does indeed come with a lot of freedom, but the relatively flat structure means that there's usually no one above the group's PI to promote their research. As a result PIs spend a lot of time writing grant proposals.
- cycomanic 6y ago
  Yes academia is very competitive, even after you get tenure you still need to write grants all the time to be able to do your research.
6y ago

sporkologist 6y ago

Exploring is its own reward apparently.

ta1234567890 6y ago

This video shows a slime mold doing exploration/discovery and then exploitation: https://m.youtube.com/watch?v=GwKuFREOgmo https://m.youtube.com/watch?v=GwKuFREOgmo Relating it to the article, it seems we function in a similar way to the mold, but our balance between exploration and exploitation changes as we grow up. Also reading other comments, it seems like different people have different balances or proportions between the exploration/exploitation modes. I personally am very imbalanced towards exploration; exploitation makes me demotivate and stop doing very quickly.

twowatches 6y ago

Science proving things that people already know anyway, as usual.

bergstromm466 6y ago

Has anyone heard of Sudbury Valley schools? Peter Hartkamp started one in the Netherlands, and wrote a book about it. “Our current education systems do not trust children to learn what they need to know. Governments believe that children must be coerced into learning what they prescribe as necessary for future life even though they do not and cannot know what that future will be like. Yet we do know that coercion produces anxiety and fear of failure and that this in turn inhibits learning and destroys confidence. We also know that creativity, innovation and empathy are not encouraged in such a climate even though above all else these are the survival qualities for coping with an uncertain future. In this short book Peter Hartkamp develops his arguments against coercive ‘education’ with a needle-sharp engineer’s logic. He invites us to imagine what schools would be like if they took the articles of the UN Convention on the Rights of the Child seriously, a not unreasonable position as all states are signatories. I wholeheartedly and without reservation recommend this book to the many parents, students, teachers, employers and policy makers who know in their hearts that something is very wrong with the examination and testing factories that we are allowing our schools to become. It represents a beacon of hope that another way is possible.” [1] [1] http://hetgedwongenonderwijsvoorbij.nl/en/ http://hetgedwongenonderwijsvoorbij.nl/en/

jkhdigital 6y ago
Yes, and I’ve been fantasizing about sending my child to one since before he was born. Unfortunately I don’t live near any :-(

fxtentacle 6y ago

While the study classifies it as exploration and claims an intrinsic reward that will die out later in life, I would describe the behavior as novelty-seeking. I'd wager that for adults, a game on a screen is just not new enough. But if you'd give adult males a choice of which female to undress, I'm pretty sure they would value "exploration" very highly again. In a similar vain, when smartphones were new, the adult population was very active at buying new ones, comparing, and exploring the market. Now that we've had them for some years, phone upgrades have become a necessity and a lot less exciting. So my theory would be that there is no loss in the intrinsic motivation, there is a loss in the amount of novelty. If that is correct, then our school system is actually very appropriate even for highly intrinsically motivated kids, because they are introduced to new topics and areas of knowledge that are (hopefully) new to them. That also aligns with my personal experience. While I hated getting up early and the concept of sitting in a chair just listening, I was excited about all the different things they showed me. And I liked best the teachers that would mention slightly irrelevant side details just for the fun of it.

stinos 6y ago
But if you'd give adult males a choice of which female to undress, I'm pretty sure they would value "exploration" very highly again Likely, but that's not the same experiment anymore since the sex factor comes into play and living beings are basically programmed to reproduce as much as possible and also with as much variety as possible (maybe oversimplified, but you get the point). So for a smartphone you can talk abut novelty-seeking/exploration (which btw I'm not sure are really different things), but when reproduction comes into play it's a whole different game anymore and it'll alter/override parts of the behavior and it shouldn't be underestimated how string the effectss can be.
- nickelpro 6y ago
  Sure but you're missing OP's point. They're saying that novelty is the fundamental drive behind the observed behavior, not some priority towards "exploration" that children have in greater abundance than adults. The children are seeking pleasure through novelty, and the researchers are interpreting that as exploration. The parent comment illustrates how silly that is, if you substitute something that still holds pleasure for an adult (ie sex), they exhibit the same behavior. The difference between the adult and the child is the adult has the world experience that the trivial game doesn't hold novel (therefore pleasurable) implications for them anymore, not that they don't prioritize "exploration".

danschumann 6y ago

Read: exploration is rewarding.

amatic 6y ago
That's what I get, too. A "reward" is a subjective experience. If someone doesn't care about grades, then good grades will not be rewarding, and bad grades will not be punishment.

OneGuy123 6y ago

Offtopic but regarding the supression of the child from the parent: few people realize how severely the large majority of parents damage their children and supress their innate creativity and drive. This is also very hard to realize as an adult because it gets unconsiously supressed. The following books opened my eyes at the fact how dangerous it is for the grown up adult to then keep thinking "yeah, but my parents did the best you could you know?" even though they completely destroyed his innate creativity and freedom. Alice Miller: The Drama of the Gifted Child Alice Miller: The Body Never Lies TLDR version of these books is that a large majority of parents tried to mold them as children and fit them into boxes or supressed their needs and wants as children and never truly listened to them. This then causes uncousious guilt/anxiety/etc... feeling in the person as adult and prevents him from "taking oportunities" etc.. And at the core of the issue is the forced-upon-us belief that we always have to respect our parents, even though we secretely hate them. Saying "yeah but they did something good for me you know" is actually a huge problem that will only keep us from seeing the fact that many in fact do not like/love their parents very much because the parents did not truly listen to them etc... As long as we try to weight the good vs the bad moments we will relapse into the false indoctrinated compassion/guilt towards our parents, into denial of the cruelties we experienced as a child, all because people say that we should try to "balance the good vs the bad". But this does not work in practice. This balancing process seems to be a reflection of how we tried to cope as children to survive. The adult must reject this balancing process because it only causes confusion without resolving anything. Many people when they break contact from parents feel guilty and think even more about their own parents: the "experts" and her parents have reinforced the indoctrinated belief in this person that "this individual has no right to his own life, feelings and needs." It was this desire of the child to believe that the parents were good that led to these supressions because the child blamed himself. This is why when a therapists says "try to see the good" it will only cause great harm. regression and confusion: since it was this need of the child to "try and see the good" in his parents when there was none that caused these issues in the first place. So anyone who even slightly suspects that their parents supressed him as a child and denied him freedom of expression etc... will find many answers in those books above.

toomanybeersies 6y ago

Reading between the lines, as we grow up, we get conditioned to follow instructions, rather than act independently. For the adults, there's no incentive to maximise how many virtual candies they collect, beyond doing it because they were told it was the aim.

anigbrowl 6y ago

This is by no means limited to young people; I much prefer video games where I can keep discovering things to the tedious business of performing tasks or winning beating bosses.

PikachuEXE 6y ago
Yes I prefer games with good stories I am having fun discovering the plots bit by bit

dzink 6y ago

Exploration as the primary algorithm continues as long as we continue to get increasing variable rewards from it. When the rewards tapper off or we are pressured for money, we choose the maximum we've found so far. This instinct to collect options has probably served us well in foraging for food, picking a mate, and any other exploration endeavor that involves finding the best of a range of newly available options. However it may also be the root of gambling addiction and any other variable rewards game systems. Gambling, lotteries can also become more attractive the more you are pressured for money - one could argue as a means of concentrating on your most approachable local maximum.

cycomanic 6y ago

For people interested in this topic I highly recommend reading about summerhill school [1] and similar projects. The topic of alternative schooling systems is quite fascinating. [1] https://en.m.wikipedia.org/wiki/Summerhill_School https://en.m.wikipedia.org/wiki/Summerhill_School

messo 6y ago
For a US-based variation of democratic schooling, check out the Sudbury Valley School: https://en.m.wikipedia.org/wiki/Sudbury_Valley_School https://en.m.wikipedia.org/wiki/Sudbury_Valley_School
qntty 6y ago
The best book on education that I've ever read references Summerhill several times (Compulsory Miseducation by Paul Goodman).

flooo 6y ago

There is an interesting relation to this with regard to reinforcement learning (RL) where the trade-off between exploration and exploitation is one of the fundamental issues. Systematic or structured exploration has shown some success here as well. What 'structured' means here deeply depends on the representation the agent has of the world, i.e. which states and actions are similar to each other. Current RL setups typically learn such representations on the go. Abstractions can be very powerful here. Arguably, people also perform structured exploration on learned representations. It may be the case that the adults have learned a representation of the 'game' and their lies little exploratory value in suboptimal actions.

Razengan 6y ago

Still holds true as an adult for me at least. It’s just that “survival” takes away too much time and energy from exploration.

rv-de 6y ago

> But while adults then used that knowledge to maximize their prizes, children continued exploring the other options, just to see if their value may have changed. I think it is not just "semantics" to consider that for children the reward is simply the rush of playing instead of what the researchers design as the intended award. And frankly speaking this is not at all a surprising result. Children learn through playing and they very much enjoy that process.

irjustin 6y ago

Overall, this style is what Montessori-schools of teaching believe and how they operate. Essentially a self-exploratory based learning style. There's lots of benefit in letting kids simply explore. As a parent, I find I have to stop myself from pushing my child to the 'best' result and let them be. That it's okay to pick a less optimal solution. Where I find this alarmingly true is when outcome/results are negative (usually ending in pain). "Don't stand like that on the chair." "Climb in this way" "Don't eat that dirt, soap, car, whatever!!" I know my job as a parent is to guide while keeping them safe at the same time letting exploration happen, but man sometimes it's just so hard. For example, letting my child explore while walking home can make a 5 minute walk take 10, 15 minutes longer. Such things tend to be at odds with what I want but it is more important than myself.

macspoofing 6y ago
>Where I find this alarmingly true is when outcome/results are negative (usually ending in pain) Pain is a very good teacher. Letting children hurt themselves and learn from that (within reason, obviously) is not a bad thing.
- chmod775 6y ago
  Especially since while you're small, falling hurts a lot less. It's surprising to me how people consistently try to discourage traits in children which they at the same time glorify in adults. I suppose considering how much effort society puts into raising meager people, it's only right we glorify those who manage to manage to grow as a person regardless.
  - germinalphrase 6y ago
    I’ve also noticed that my toddler falls “better” than most adults do. If you ever take up judo, early on they need to teach you to fall correctly so you avoid injury. My toddler does most of it be instinct which is cool.
    - 6y ago

pul 6y ago

This has surprising similarities with data science theory: multi-armed bandit with a decreasing-epsilon strategy (children) or an epsilon-first strategy (adults).

082349872349872 6y ago

Anyone have access to the paper? As far as I can tell, the young children were perfect convex utilitarians, in deciding that more than "enough" stickers were not worth the more fun of clicking around. How were the adults rewarded? Did they also get something intrinsically valuable at the end of the experiment, or did they just collect virtual points because they were told they were supposed to? Working dogs have been specialised by breeding for overexpression of the various initial phases of the wildtype hunting sequence: herding, retrieval, etc. Does "sex sells" indicate we may have been bred to overexpress the "identify attractive potential partner" and "do things they (appear to) like doing" phases of the wildtype reproductive sequence?

wcoenen 6y ago

I'm reminded of a paper about an AI system with curiosity. Basically, the AI would learn to predict the effect of its actions, and would prefer to take actions where its prediction error was large. That way it would gather more data about stuff it didn't know yet. This works well to learn to play video games without defining an extrinsic reward. However, there is a problem when the system encounters the equivalent of TV. It then gets stuck watching the unpredictable stream of events. paper and code: https://pathak22.github.io/large-scale-curiosity/ https://pathak22.github.io/large-scale-curiosity/ Coverage by "2 minute papers" youtube channel: https://youtu.be/fzuYEStsQxc https://youtu.be/fzuYEStsQxc

trombonechamp 6y ago

Monkeys do the same thing: Many non-human animals show exploratory behaviors. It remains unclear whether any possess human-like curiosity. We previously proposed three criteria for applying the term curiosity to animal behavior: (1) the subject is willing to sacrifice reward to obtain information, (2) the information provides no immediate instrumental or strategic benefit, and (3) the amount the subject is willing to pay depends systematically on the amount of information available. In previous work on information-seeking in animals, information generally predicts upcoming rewards, and animals’ decisions may therefore be a byproduct of reinforcement processes. Here we get around this potential confound by taking advantage of macaques’ ability to reason counterfactually (that is, about outcomes that could have occurred had the subject chosen differently). Specifically, macaques sacrificed fluid reward to obtain information about counterfactual outcomes. Moreover, their willingness to pay scaled with the information (Shannon entropy) offered by the counterfactual option. These results demonstrate the existence of human-like curiosity in non-human primates according to our criteria, which circumvent several confounds associated with less stringent criteria. Paper: https://www.sciencedirect.com/science/article/abs/pii/S0010027719300642 https://www.sciencedirect.com/science/article/abs/pii/S00100... Non-paywalled version: https://www.biorxiv.org/content/biorxiv/early/2018/03/29/291708.full.pdf https://www.biorxiv.org/content/biorxiv/early/2018/03/29/291...

officehero 6y ago
Doesn't this just mean that the "reward" definition is poor in this case i.e. the agent does not care about maximizing it because there are additional components left out of it (e.g. survival)?

sillypuddy 6y ago

Young people of any age

distant_hat 6y ago

This is completely expected. If you look at multiarmed bandits, getting rewards would be exploiting and the other option is exploring. If you have a longer time horizon as children will, it is better to explore and find a better return than exploit something already known which is likely to be suboptimal.

hunterx 6y ago

Some adults do as well, they are the ones who usually discover be things...

deleted 6y ago

[deleted]

gomes33 6y ago

For those interested in this topic i recommend reading about the Montessori method. The basic idea of the Montessori philosophy of education is that every child carries within him/her the man/woman he/she will become. In order to develop the physical, intellectual, and spiritual powers required for this task to the fullest, there must be freedom—freedom to interact with a prepared environment.

andrekandre 6y ago
along those lines, seymour papert talked about imersion being necessary for students to learn in thier own way, and for teachers to be not just instructors but guides who also share and learn with the students i guess said more simply, its learning by doing (go to france if you wanna learn french, or at least make a mini-france in your classroom)

known 6y ago

Interesting; Wish adults do the same;

jkhdigital 6y ago

Seems like further evidence that nature has developed an optimal algorithm for solving the multi-armed bandit problem as it manifests in the typical environment humans evolved in. Unlike the artificial games used by the researchers, there is no such thing as a fixed reward in nature—everything is subject to change unexpectedly. Children are wired with an algorithm that heavily favors exploration in light of the dynamic environment.

eloff 6y ago

We use a variation of this "four squares" game at Sparkademy. The four squares of randomized point values (there's a pattern to it, but I don't want to give it away here.) The goal is to maximize your score which involves switching between exploring for a better payoff and exploiting what you think is the highest paying square over a large number of trials. We use it to as part of a criteria to identify if a person might be well suited for our innovation online course, and later be able to apply that knowledge in their company.

sjg007 6y ago

Man I thought this was obvious!

ddingus 6y ago

As a kid, exploring was, and remains a reward. Someone to help, share in it, make more possible? Golden.

PikachuEXE 6y ago
And this makes me think of Minecraft (on multiplayer servers)

ncr100 6y ago

What does this say about Capitalism and the accumulation of wealth?

Aviatore 6y ago

If OP's premise is true, I wonder how much that is changing now that parents are throwing electronics at their children. Many of the things that they get to interact with on an electronic aims to optimize rewards. Will this shape generations that are always in search of quick hits and less interested in the exploratory nature of things? How will something that seems to small as a parent reshape the children and society?

GuB-42 6y ago
I don't think so. I suspect the "exploration" bias observed in that study is somewhat hardwired into the brain. Kids need to learn, adults need to be efficient. The test was clearly designed as a way to optimize reward. And indeed, adults quickly went to the optimal solution. But kids kept exploring, even if further tests show that they knew the optimal solution.

kingkawn 6y ago

That’s why we train them to want rewards first, then use that to get them to do what we want

blobbers 6y ago

Isn't this the children solving the multi armed bandit problem? In a child's mind, they haven't yet fully decided how things behave and react, so they are willing to try things to see if they've changed (because in fact often they do in their own mental models, as well as the resolution of those models. If you think of it as a regression tree, their models start out fairly shallow and then slowly get deeper as they age. That's a square thing. That's a book. That's a book about animals. That's a book about marine animals. That's a book about marine animals that I like. That's a book about whales. In trying to solve the multi armed bandit, they're willing to explore to try to find better rewards because in their own world there are often better rewards. If the game shifted to having a different alien be the best one half way through the game, I'd be curious how quickly the more maturely cognitive kids did. It's possible that becoming fixated on something (the way adults are) can ultimately limit your abilities in a game. In adulthood, "creative" solutions can sometimes be better than more "standard" solutions, but incur more risk.

mam2 6y ago
Yes its called "simulated annealing for your own life". In 3 days hacker news will discover GANs and make a shocking headline like "challenges and adversity makes you better / stronger". #mindblown
- blobbers 6y ago
  Haha. Don't blame hackernews! This is social science encounters real research ;)
- weatherman2 6y ago
  I'm going to read my children a picture book titled "Scraggles the Puppy Explores the Forest of Simulated Annealing."

zackmorris 6y ago

Well ya. I've found at middle age that I'm most interested in connection and transcendence, finding meaning and such. When one realizes that material things truly are valueless compared to things like consciousness, friendship, love, dignity, etc etc etc, then everything else falls away. I often find myself realizing that I am looking at the world with the same childlike wonder I had when I was 5. To me, this search for meaning was what the 1960s was all about, as well as every other cultural renaissance and era of spiritual awakening. Which makes the conceptual basis of things like economics suspect. The idea that humans are motivated by monetary rewards and material things turned out to be false. So did concepts like scarcity, reciprocity, commoditization, and so on. Those things worked at one time, up until the middle of the last century and dawn of the information age (where technology surpassed what was required to meet humanity's basic needs), but now actively inhibit human evolution IMHO. So how to fix the mess we're in? I think it starts with questioning basic assumptions. Sadly, American culture seems to be racing away from that as fast as it can in these times. Dunno about the rest of the world.

thefucnjosh 6y ago
It's been known for years that oftentimes paying people more for intellectually demanding tasks, leads to decreases in productivity Capitalism and the ideologies surrounding it, fall apart when you start looking at real assets and how the exchanges value of those assets takes a hit during economic down turns. The idea that "humans [were] motivated by monetary rewards and material things" didn't turn out to be false, it's just always been the case that it wasn't true.
- pg_1234 6y ago
  There's converse to this however, when you charge a company more they're less inclined to waste your time.
Florin_Andrei 6y ago
It's just a matter of optimal strategies. When you're young, take risks, explore, maybe fail on occasion. There's still time to fix all that. When you're old, go for the more likely reward. If you screw up, that's how it stays, forever.

ZainRiz 6y ago

I can attest to this. I tried offering my toddler a thousand dollars. She preferred to blow bubbles.