12 ms·
Is human behavior just elaborate running and tumbling?
- tomcam 5y agoSadly not at my gross tonnage and displacement
- PaulHoule 5y agoMakes me think of this book https://www.amazon.com/Vehicles-Experiments-Psychology-Valentino-Braitenberg/dp/0262521121 https://www.amazon.com/Vehicles-Experiments-Psychology-Valen...
- StopTheWorld 5y ago> Why do some “gym rats” build their muscles well past the point of being a way to attract potential mates? > Why are some people willing to free solo climb icy mountains despite the danger? > Why are some people willing to have fewer children in order to pursue career goals? > Why are most billionaire tech CEOs interested in space travel and extraterrestrials? > There are no doubt potential evolutionary explanations for each of these phenomena. No doubt? > But, from a purely evolutionary standpoint, these phenomena are at least (a) not obviously explained or (b) seemingly irrational. Correct. Many phenomena have an evolutionary explanation. However some phenomena do not - random mutations happen and things pop into existence without much connection to mating, survival etc. Trying to shoehorn evolutionary narratives onto them is not letting the evidence fit the theory, but instead trying to get facts to fit some preexisting theory. Some traits just pop up due to random mutations, and trying to find an evolutionary mating/survival/etc. narrative for each of them does not make sense.
- jjk166 5y ago> Why do some “gym rats” build their muscles well past the point of being a way to attract potential mates? Being a gym rat is not a heritable trait, but the ape who keeps doing whatever they were doing that made them strong and sexually desirable could be much more evolutionarily successful than the ape who stopped and became weak and undesirable. More generally, a positive feedback loop for beneficial behaviors is a desirable trait. > Why are some people willing to free solo climb icy mountains despite the danger? Being a free solo climber is not a heritable trait, but exploring your environment and incrementally pushing one's limits are both desirable traits. The ape that doesn't know which rocks he can climb falls to his death, the one who knows where to place their hands survives to reproduce. > Why are some people willing to have fewer children in order to pursue career goals? Corporate ladder climbing is not a heritable trait, but investing more resources in fewer offspring is advantageous when the odds of those offspring surviving are high. Even with no kids of your own, you pass on your genes through nieces, nephews, etc. The desire for resources has obvious evolutionary benefit. > Why are most billionaire tech CEOs interested in space travel and extraterrestrials? Neither being a tech billionaire nor being interested in space is a heritable trait, but the desire to explore is undoubtedly a desireable trait. The apes that expand into virgin territory can acquire more resources and become more resistant to local catastrophe than the apes who remain in one location. Elon Musk just has much better odds of becoming king of mars than I do. 500 years ago this may have manifested as building big ships instead of big rockets. Of course random mutations happen, but those random mutations get eliminated quickly if they are harmful and spread quickly if they are helpful. Saying something is just due to random mutation is not only an unsatisfying answer, it is the answer to the wrong question: it doesn't matter why we started doing it, why do we still do it? Evolution doesn't create any trait, it merely perpetuates or fails to perpetuate them. Not all behaviors are hardcoded into our DNA, certainly not at the level of specific actions we typically take day to day - my ancestors did not pass on their genes more successfully because of their affinity for arguing on the internet - but those traits which are heritable are unavoidably influenced by evolution. Evolution is the null hypothesis - if a system left to its own devices would do something without any external intervention, why invoke any additional explanation? It is those saying that something is genetic but which has somehow defied evolutionary pressure which are asserting a narrative, and who must show evidence in support of that narrative. Some narratives are legitimate, for example the traits of domesticated plants and animals aren't explained by natural selection; but oftentimes it is just "magnets, you can't explain that!"
- coliveira 5y agoTrying to find an evolutionary explanation in existing behaviors is putting the cart before the horse. Organisms can do many things that have nothing to do with evolution. However, evolution will take care that only some of them will survive. So, what was good for evolution can only be inferred from the past.
- spicybright 5y agoAlso, probably obvious, but us humans have been countering evolution for a while now. Medical science (both for sickness and fertility) + modern lifestyle keeps people alive that wouldn't otherwise be if we were still living in the woods. So (at least from my armchair) it's no wonder we see more "pointless" behaviors.
- lanstin 5y agoThis isn`t at all countering evolution. It is changing the fitness pressure. being able to parent effectively with smart phones and sedentary jobs and needing a lot of education is more fit than not having metabolic disorders or a weak immune system. If anything our tendency to not have reproductively isolated subgroups and large population is making speciation less likely. But rest assured our genome is slowly changing as we change our fitness landscape.
- cblconfederate 5y agoRandomness is also evolutionary. It's wrong to equate evolution with optimization
- stabbles 5y agoThis is what Thomas Kuhn describes as "normal science": fitting observations into the current scientific paradigm.
- snek_case 5y agoI would say the evolutionary explanation for each of these is actually pretty straightforward. We live in an environment that's very different than the one we were in for most of our evolution. Our evolution hasn't had nearly enough time to account for the modern environment. > Why are some people willing to have fewer children in order to pursue career goals? The obvious explanation here is that there wasn't really birth control when humans lived as hunter-gatherers. You couldn't make that kind of sacrifice for the most part. There also weren't really careers in the modern day sense. Evolution hasn't had a chance to account for birth control yet. It's only been around for 60 years or so. > Why are most billionaire tech CEOs interested in space travel and extraterrestrials? Because all life has some instinctive drive to explore and spread. By exploring, you have an opportunity to find new resources and territory, which is essential for survival or to outcompete other life. > Why do some “gym rats” build their muscles well past the point of being a way to attract potential mates? When humans lived as hunter-gatherers, there were no gyms and no muscle building magazines, no cult of bodybuilding. For the most part, you got as fit as you needed to be to hunt animals, to fight, and build whatever you needed to build. However, I think the part about being "well past the point of being a way to attract potential mates?" is a value judgment. In actuality, growing a lot of muscle mass is a way of showing off your determination and fitness and standing out. Male bobybuilders probably have a much easier time attracting mates than the average joe. > Why are some people willing to free solo climb icy mountains despite the danger? Evolutionary drive to explore. Evolutionary drive to show off prowess to attract mates and gain status. It's a high risk, high reward strategy.
- kortex 5y agoI think these "overdriven goals" might be the result of leaky abstraction. We are basically seeing an implementation detail come to light. As TFA points out, there is really just one reward-taxis algorithm. Everything drive-related is a variation on it. Meaning there is no "obtain visual appeal via body building until sufficiently attractive" algorithm. There is a top-level algorithm of "find mate and reproduce", which is enjoyable. How does it boost the odds? By doing things to be more attractive, for example. But how does one "know" one is on the right path? Activities which we learn increase attractiveness are enjoyable. There's a gradient we follow. More attractive = more enjoyable. So you end up with complex behaviors that basically become self-hoisting; the activity itself becomes the reward. Basically, games and hobbies. Something becomes enjoyable vis a vis various learned reward functions, so we just pursue the enjoyable thing. There's also a meta-loop wherein staying mentally stimulated and curious is itself a useful fitness, so we do things which are interesting, because it's enjoyable to do that.
- psyc 5y agoOoof. There's my yearly reminder to go check if Alex Honnold is still alive. My heart skips a beat as Bing Search autocompletes "Alex Honn" to "Alex Honnold Death". But he's still alive! And he's married now. "Individual organisms are best thought of as adaptation-executers rather than as fitness-maximizers" -- John Tooby and Leda Cosmides
- matwood 5y agoWatch The Alpinist on Netflix if you haven't yet.
- dpark 5y ago> Some traits just pop up due to random mutations This seems like an extraordinary claim. I can get behind the idea that it’s possible for a random mutation to impact personality traits. It is entirely different to turn to this as a general purpose mechanism for things that are hard to explain. It seems overwhelmingly likely that most traits are in some way selected for, because useful traits are naturally more likely to propagate than random useless (which will pass on, or not, at random)
- echelon 5y agoI think our species is post-genome for a lot of the complex behavior you see in society. Cultural and ideological memes use us as information carriers. We fixate on certain ideals and values, and some of us make them our entire lives. Preferring Pokemon over Halo doesn't entirely reduce to one's genes. Being too busy to have kids is situational and part of a complicated modern life. There is genetic contribution to these outcomes, but we're well past the point of genes being the most complicated factor. Now you have to model society and a person's random walk through it.
- dpark 5y agoI agree with most of what you said except for this notion that we’re “post genome”. We are absolutely not. What you described is just “nature vs nurture”. Genes don’t make anyone prefer Pokémon over Halo directly, but the mechanism was built by evolution. A person who makes friends playing Pokémon is certainly likely to develop a fondness for Pokémon, because having friends is beneficial to survival and reproduction.
- wizzwizz4 5y agoModern computing is “post hardware”. That doesn't mean the hardware doesn't matter, or that changes to the hardware don't make a difference; it means describing the hardware is insufficient to describe computers, whereas describing the software is mostly sufficient.
- 5y ago
- mensetmanusman 5y agoIt’s tautological at this point to say that any behavior arose from an evolutionary standpoint. It’s conveniently tautological because it’s impossible to prove or disprove without time travel :) A more interesting conversation which is also impossible to prove/disprove is whether there is any behavior that isn’t just behavior/instinct. It sure feels like it with consciousness, but if humans are weights and biases, it was inevitable that I typed this :)
- prometheus76 5y agoEverything gets squirrely and confusing when consciousness tries to look at itself. The recursion breaks everything.
- DarylZero 5y agoSome behaviors or phenotypical features represent adaptations that have been selected for, others don't.
- machinerychorus 5y agoThis is also ignoring a crucial aspect of human behavior: we have moved past simple evolutionary optimization. Our behaviors can now change very rapidly due to spoken and written language. In The Selfish Gene, Richard Dawkins called this "memetics". E.O. Wilson also wrote about this, I think referring to it as "culture" (I don't remember in which book, sorry). Evolution is still necessary for understanding human behavior, but it is not sufficient by itself.
- rini17 5y agoYou completely ignored the "4e. What Space Are We Navigating?" and are thinking in terms of navigating evolutionary fitness/random mutation. I can see how our large brains create a new kind of virtual space we move around in, and it is unlike either of the above.
- visarga 5y agoWe're navigating "survival space", the space of all possibilities of existence in the future, under limited resources. It's pretty open-ended, we don't know what we could be, the peaks of evolution are clouded and we don't know if any particular path leads to it or is a detour. Only when you're close enough can you make progress by going directly towards the target.
- rini17 5y agoI see survival space like an outer hard boundary. Brain can and in reality often makes its own vaguely related space and we navigate it oblivious to the hard survival boundaries..till they hit us.
- visarga 5y agoYes, that's what I think happens. The more sheltered we are, the more divorced from reality our thinking can be.
- visarga 5y agoNovel behavior and random mutations do have an explanation - they help explore possibilities and find better ways. Most of the time it's a loss, but it's still worth it in the long run. It's also a desirable sexual trait to have unusual abilities or achievements, to be the explorer or the conqueror (for men, for women there are other criteria).
- achenatx 5y agoany trait can be positive, negative, or neutral for evolutionary fitness. If a trait is neutral it can potentially survive and be passed down. There is evolutionary fitness at an individual level, but cultures have evolutionary fitness as well. A culture might preserve individuals that have low individual fitness because they create cultural fitness. For example the need to explore and expand the boundaries of knowledge may not be useful at an individual level, however a culture with those individuals might survive better than a culture without. All members of the culture that does not have those individuals may die, even if individuals are very fit. Sociopathy is likely conserved at a cultural level because cultures periodically need sociopaths to completely obliterate competition or ruthlessly compete to survive. Cultures also are constantly spawning mutations (civil rights, black lives matter, nazi ism, capitalism, democracy. social justice etc) those mutations can be positive, negative or neutral. Neutral traits can hang around if they carried alongside traits that do provide cultural fitness.
- neutronicus 5y agoThe thing about evolution by natural selection is that selection stops when you get to "good enough" Past that point you just get whatever random sexual selection criteria emerge, and the ensuing feedback loops Same reason the world is full of brightly colored loud-ass birds
- telxosser 5y agohah I love the brightly colored loud-ass bird point. That is spot on.
- goodlinks 5y agois there not an additional level added with mental abstractions? e.g. i blast it at the gym (i dont actually but play along ;)) and i build an abstraction of life that that is part of. the fact that the abstraction has limitations is irrelevant. we dont process the real world we process our learned abstracctions of it.
- PaulHoule 5y agoIt's funny because I have been a "gym rat" much of my life and until very recently wasn't motivated by vanity, specifically, being attractive to the opposite sex. I've seen a burst of assertions lately from 20-somethings that seemed to deny anyone would have any other motivation for physical fitness. Oddly compared to my cohort I am both a nerd and a jock. I find intense exercise be it hiking or cardio or weights helps me keep my sanity in terms of sitting all day and the other stresses of my work. It's not like it is abstractly good for my health in the long term and I'm discovering now that it probably wasn't but that I feel better immediately in the short term from physical exercise.
- deleted 5y ago[deleted]
- matwood 5y agoI perform hard physical fitness for a lot of reasons, but the primary reason is simply that I enjoy it. If I skip a workout I miss it and don't feel well. Atomic Habits touches on this. People think that I have all this discipline to get up and workout, but the reality is that I enjoy getting up early and working out. I enjoy training JiuJitsu and more often than not getting my ass kicked. I can't really explain why, somewhere along the way I associated feeling good with hard physical work.
- netizen-936824 5y agoWhy is it not good in the long term?
- PaulHoule 5y agohttps://www.merckmanuals.com/professional/cardiovascular-disorders/sports-and-the-heart/athlete%E2%80%99s-heart https://www.merckmanuals.com/professional/cardiovascular-dis... I wore a heart monitor for a month and we observed about 30 seconds worth of AFib right at the very end. (Oddly I had experienced a very interesting setback in my personal life around then.) I had the signs described above. My doc said I should go on exercising but I shouldn't train more than 1 hour a day. (In a time when my life went to crap I had got into the 2 hours a day habit to fight my depression.)
- atlantas 5y agoThose are mostly examples of peacocking (displaying oneself ostentatiously), which has an evolutionary basis.
- readams 5y agoEven beyond random mutation, it also makes sense to think about behavior in totality. Curiosity and a willingness to do insane things seem to be tied up in an overall system that does very well in maximizing reproductive success. But this doesn't mean that literally every behavior that results in individually optimal at spreading genes. Human backs are maybe a good simple example of this: walking upright has been great overall, but has a lot of drawbacks that we suffer from such as difficult childbirth, back problems, etc. You could ask "what is the evolutionary advantage of lower back pain" and the answer of course is there's none, but there is still a broader explanation that makes sense when you consider the whole system.
- eli_gottlieb 5y agoEvolutionary explanation? There's a whole missing level of analysis here. Evolution doesn't operate directly on behaviors. It operates on physiology.
- kmonad 5y agoOf note, evolution is not equal to natural selection. "random mutations" are causes of evolution, drift (random) or selection (fitness benefit) are both mechanisms by which mutation can manifest in a population.
- iroh2727 5y agoThanks for quoting this part of the paper. It's probably the most interesting part. To add to what you said, though, I don't think of these phenomena in terms of random mutations. In other words, I don't think of these people as anomalies that do anomalous things; moreso, I think of it as following our growth imperative (i.e. the expected reward gradient), which, if we follow it long and hard enough, can lead to behaviors that appear anomalous. One commenter below (kortex) really hit on what is at least my interpretation when he said "we are basically seeing an implementation detail come to light."
- w_for_wumbo 5y agoThis seems like something Buddhism attempts to address (from my understanding) > Desire is like an never-ending thirst > No amount of success ever will quench it > Their solution - Remove desire. The only way to get everything you'll ever want, is to want nothing.
- goatlover 5y agoDoesn't seem like a desirable tradeoff.
- arbuge 5y agoLet me give it a shot... > Why do some “gym rats” build their muscles well past the point of being a way to attract potential mates? Margin of safety. Many would probably disagree with the author's assessment that they're well past that point. > Why are some people willing to free solo climb icy mountains despite the danger? Conquering new land is deeply embedded into our psyches for evolutionary reasons. The species that's not afraid to do that will probably be more successful in the long run. Conquest of more land is generally correlated with better survival chances. > Why are some people willing to have fewer children in order to pursue career goals? Perhaps many, like myself, are aware of how expensive kids are but unaware of how difficult having kids gets later in life. So they optimize this wrong. I wish I had started earlier, but I put it off until later because I thought I'd be able to provide for them better. This thinking was correct, but having them earlier would still have been better, for me at least. The point is, it's certainly not due to a lack of evolutionary interest in having kids. > Why are most billionaire tech CEOs interested in space travel and extraterrestrials? See conquest of land above...
- watwut 5y ago> Conquering new land is deeply embedded into our psyches for evolutionary reasons. The species that's not afraid to do that will probably be more successful in the long run. Conquest of more land is generally correlated with better survival chances. Overwhelming majority of people stay where they are. Generally humans need huge practical motivation to conquer lands. Plus, interpreting climbing as conquering lands is super odd. It has more to do with any other sport achievemt then anything else.
- amelius 5y ago> > Why do some “gym rats” build their muscles well past the point of being a way to attract potential mates? Probably because once they start building their muscles, they compete in their minds with other gym rats and it becomes a vicious cycle. And also: ads, social media.
- throwaway889900 5y agoGotta build those muscles for those sweet sweet impressions!
- TheSpiceIsLife 5y agoSimple explanation: those things can all be considered play, which is common among young mammals and a neotenous feature in human adults. Play is important for social cohesion.
- gherkinnn 5y ago> Many phenomena have an evolutionary explanation. However some phenomena do not Some traits might just present themselves as {seemingly nonsensical activity} in an industrialised world. Where back in time, these same people might have an advantage in {seemingly sensible activity}. This does not preclude random traits being passed on, it is merely another way too look at things. > and trying to find an evolutionary mating/survival/etc. narrative for each of them does not make sense It is also a very lazy and uninteresting line of thought. Of course {trait} had en evolutionary advantage. Why else would it be here? And if it is indeed random, then it may be of slightly more interest, but tells me very little. We’re observing shadows cast against a wall. Eagerly pointing out this and that pattern. When the interesting part is knowing the individual shapes that make up the dancing display.
- uoaei 5y agoCall it "linear regression brain" -- the inescapable need to consider minute, limited traits as having some fixed, measurable contribution to the outcome, rather than a component in a rich tapestry of complex dynamics.
- pjc50 5y agoAnything can be "just" anything else if you simplify it enough. The idea that humans are goalseeking systems motivated through a single parameter of dopamine response seems like it might be reasonable enough, but that then just moves the problem around: how is a particular set of stimuli + past stimuli linked to a dopamine response? Furthermore, are they intending to force all behavior analysis through a control system with a single parameter? Can we ask questions like "what is the frequency response of the dopamine system?"
- datameta 5y agoAnother question is - what stimuli, changes in environment, or thought patterns affect dopamine levels and reuptake, and how? Also how do other neurotransmitters attenuate or floor/ceiling our behavior gamut in a way which modifies how we react to or are informed by dopamine levels and their rate of change. Essentially, if dopamine can be boiled down to a single lever, then I think that lever has as many buttons as a Formula 1 steering wheel.
- kortex 5y agoIt's definitely not a single parameter. I've been down this rabbit hole a bunch. There's actually 5 main subtypes of the dopamine receptor. They express in different concentrations in different brain regions, and have different functions. DRD2 is kind of the heavy lifter when it comes to goal seeking. If you had to boil it down to one idea, I'd call it "go/no go". It might be tempting to try to map D2 to something like a coefficient, perhaps even the "run coefficient" in run and tumble. But it's not at all like that, even on its own. It's very nonlinear. Furthermore, the action of D1 modulates D2 in a concentration-dependent way. So you have not just D2 signaling, D1 signalling, but also the differential signal as well. We don't know exactly how it maps out, but we know it plays a roll in weighing risk/reward ratios. Screwing with this pathway often results in "low risk low reward" mild manner folks switching to "high risk high reward", often going into gambling, manic spending, hypersexuality, and grandiose thinking. This can be seen occasionally with e.g. Parkinson's drugs such as L-DOPA. Actually my headcannon is there are some parallels between Parkinson's and control theory. Net loss of dopaminergic neurons results in lower bandwidth on your control loop. In PID loops, one of the symptoms of insufficient I/D gain is ringing, where the controller constantly hunts for the setpoint and overshoots. Tremor is a common symptom of Parkinson's (though it affects much more than just tremor), but if you notice, it's usually not a continuous tremor, it's only when trying to execute motor control, especially fine motor control. The body knows kinematically where it wants to be in space, but the feedback loop rings constantly as it isn't able to get sufficient bandwidth to error correct. Anyway, I think the point of TFA is not to suggest that this loop is a single parameter, it's just to model it as such. It's a spherical cow reduction. That kind of reduction lets you build practicable theories and test hypotheses. Such as: if Parkinson's is the result of reduced bandwith, can we treat the symptoms by increasing dopamine gain? And in fact, we can, that checks out.
- eutectic 5y agoIf this can be turned into mechanistic theory, then implement a version of it that can solve CartPole (as an easy first test). Otherwise, it's just hand waving and conjecture.
- zanecodes 5y agoRun-and-tumble sounds an awful lot like stochastic gradient descent
- beaconstudios 5y agoyep, the author is describing gradient descent over state-space - which is the driving force behind basically all adaptive systems*, though varying in both state-space complexity and descent algorithm. This should be expected though, as neural net ML practices are a direct descendant of cybernetics, and are essentially an attempt to digitise natural adaptive control systems. * you can reasonably analogise a single sensor/effector control loop as gradient descent against a 1d state.
- lanstin 5y agoDo we assume that searching in real physical space for a literal 3d smooth gradient max corresponds mathematically to maximizing the output of a planned exertion model that is internal to the computing world and probably responding itself to the neurotransmitters?
- beaconstudios 5y agoIf I understand the second half of your comment correctly, then yes - optimising an n-vector fitness value over a state space (for example 2 floats from 0-1 would form a 2d cartesian state space from 0,0 to 1,1 with a continuous output value at all points), is basically the same as finding the tallest sand dune in a desert by walking around, only able to know your current elevation and the angle of the slope underfoot. Of course this is an evaluation where x and z are inputs to our fitness function and y is the output, so this is a 2d space - but this is directly equivalent to say, trying to optimise heat and humidity for maximising yield for a species of plant. You could add further variables like soil acidity or atmospheric CO2 concentration to increase the dimensionality of the state space. I may have some of the exact terminology wrong here (whether 2in-1out is considered 2d or 3d for instance) - my interest is in cybernetics generally not gradient descent specifically - but hopefully you get the gist.
- deleted 5y ago[deleted]
- andrewflnr 5y agoI don't think it makes sense to tie dopamine to physical movement and analogize that to "movement" in the ludicrously high dimensional and topologically weird "space" of all possible actions, including but not limited to job interviews. If a space analogy still works after all the complexity we've added to our "run and tumble" algorithms, I think that's probably a coincidence.
- cyber_kinetist 5y agoThe major problem with this argument that it relies on a very important (yet questionable) assumption: the reward hypothesis, the idea that the maximization of the expected sum of a single signal variable describes the organism’s behavior (http://incompleteideas.net/rlai.cs.ualberta.ca/RLAI/rewardhypothesis.html http://incompleteideas.net/rlai.cs.ualberta.ca/RLAI/rewardhy...). First we need to be careful defining what an “organism” actually is (because we are actually made out of billions of systems interacting with each other at the same time), but it is also hard to believe that all those interlocked systems move towards a single purpose of maximizing one variable, whether it be by chance or by coordination. The other problem is that the researchers might all be performing the error of defining the reward function based on our own narrow views and try to shoehorn it on other species as a process of pattern-matching. The third is that, as shown in the bacteria example, the organism and the environment are not independent from each other, and what we call “intelligent behavior” is not an inherent aspect of a creature’s “algorithm” and cannot be studied independently in isolation from the environment. (These can all be extended as a critique of evolutionism as a useful scientific theory; the problem is not that it’s wrong, but that it becomes more of a tautological statement as you bury yourself deeper into it, so its usefulness as a theory needs to be questioned.) A bit more tangential, but the article really falls apart when the author starts talking about examples related to humans, using the careerism of current middle class society as an example (The notion of a “career” is actually pretty recent from an anthropological perspective! Those anecdotes about hierarchical milestones and pushing ourselves for career advancement has more to do with the culture and ideology of today’s world than any other weird evolutionary/neurological reason!)
- andi999 5y agoSo you think your CNS and liver might disagree about the amount of alcohol to drink on the weekend? Part of me agrees with that assessment.
- lanstin 5y agoPretty sure even different subsystems of my CNS will push in different directions on that one. One just waits for the networks to settle and see what your choice is this time. Possibly biasing the outcomes with thoughts like, "what a week" or "I was looking forward to meditating"
- abeppu 5y agoI kept expecting the phrase "reinforcement learning" to show up, but even after mentioning a Bellman-style time-decayed sum over future rewards, it doesn't appear, which seems odd to me. And roughly, I think "runs" == "exploitation", "tumbles" == "exploration" in terms other people use. A thing that the author glosses past is the relationship between having a model of the world and maximizing rewards. It's taken as given that a model-based approach leads to better predictions about rewards ... but when we try to get systems to actually solve problems, this often isn't the case. Learning policies which yield high rewards is often easier than learning a model which actually describes the environment. I think of value iteration as learning "I feel this is a good state", policy iteration as "I feel this is a good action", and a real model as implicitly providing "I think this is a good action because I expect that ..." If artificial systems have the greatest successes with model-free approaches following algorithms very similar to the one the author describes, but humans seem to have a rich model _and_ do better than those artificial systems (at least accounting for data efficiency), doesn't that at least suggest that we're thinking differently?
- Enginerrrd 5y agoI think it really just suggests the models are being trained improperly. We just don't train alogorithms on data as complex as the real world. So, it's not surprising that simpler (less correct) models win out over more accurate nuanced ones.
- abeppu 5y agoMy take is more that the framing of maximizing an externally provided reward describing a single task only helps an agent who can rely on the rules of that task remaining fixed. Humans don't just try to do one task well; we switch between tasks. We don't have episodes that reset really; we have one long episode, and our actions decades ago can have bearing on our well-being today. What is a good state or a good policy varies substantially over time. The concept of the value of a state or (state, action) is ambiguous, because who knows what you'll want tomorrow? If the definition of value shifts, but the rules of the world don't, then a modeling becomes more "valuable".
- jvanderbot 5y agoThis is interesting for its breadth, but they are abstracting a huge set of gradient-climbing behaviors that I'm not sure are of the same class. Also, this has been known forever (since before I was born) in game theory / repeated games as "exploration vs exploitation". But, it's not surprising to me that a mathematically simple, yet provably convergent algorithm appears over and over. For example, reward maximization in repeated bayesian games looks a lot like "Run and tumble", once you add this critical step (from OP): > When a “tumble” occurs, rather than sampling a new direction from a uniformly random distribution, we sample according to the distribution of expected rewards You may want to read the book Algorithms to Live By, also.
- Barrin92 5y agoIf you want to compare humans moving through decision space to bacteria tumbling through physical space I suppose it's one way to think about how humans act but honestly that's a lot of text for a trivial kind of system that you can apply to almost anything. You can probably look at cars in traffic like this as well. It's so vague and general of an idea that it's almost a truism. I don't think it provides anything meaningful in regards to figuring out the actual complexity of human behavior both individually or collectively. And I think this kind of over-generalization that is a little bit vapid has been a trend because there's simply not been a lot of theoretical progress when it comes to the functioning of the human mind. When people use this kind of thinking to apply it to artificial intelligence I always think, why not just ditch intelligence and build a physics simulator? Clearly intelligence is somewhere in the subspace of simulating the physical world, so why bother? It doesn't add much to understanding anything in particular.
- iroh2727 5y agoIf you make it to the psychology section, that's where the explanatory power becomes apparent (or at least is speculated on). Though, yes, it is a lot of text to get there (I'm relieved that some people made it that far). If you do make it that far, there is a payoff, though. This "trivial kind of system" isn't merely an analogy but the actual algorithm that we use, and the "implementation details" of this algorithm manifest in unintuitive ways that have a lot to say about how psychology is understood in general. Though, given your skepticism, more research will likely have to be done before this is satisfactorily proven to you. This breakthrough (which I'd say is primarily K&A's breakthrough that I merely am speculating on top of), is actually a great example of theoretical progress. I typically consider myself an anti-reductionist, but this is a lesson in how studying simple systems (in this case, chemotaxis), can lead to better understanding of complex systems (because, well, it turns out at least in this case that these complex systems are actually the same simple systems at the top level—the complexity gets added in "layers" much like the evolution of brains).
- kingkawn 5y agoToo reductionist.
- pjdorrell 5y agoIn the bacterial example, each individual tumble chooses a new direction at random, and the effectiveness of the algorithm depends on the smoothness of the chemical density function in the 3D space that the bacteria is swimming around in. In his human examples, all the "tumbles" are conscious decisions made by the person in response to their circumstances. If there was some human equivalent of the "tumble", it would have to be something that was change just for the sake of change. Like: "I'm bored and/or frustrated, so I am going to do something stupidly different".
- NaNDude 5y agothis is categorical thinking. behavioral psychology, molecular genetics, behavioral genetics, ethology, endocrinology, etc. will all provide their own "demonstrated answer" to the same question, even when they will be contradictory to each other.
- lifeisstillgood 5y agoI am pretty sure this dopamine algorithm was part of the "ten equations that rule the world" book. And it is much more clearly explained here. Worth reading the book though :-)
- ada2342gh 5y agoSo the master algorithm matches a somewhat rational behavior, from time to time. But humans wouldn't be really rational beings. The seemingly fixed by algorithm human standard behavior will ultimately, eventually, lead to irrational behavior, by not fully able of "matching" actions with objective really. Just like a trying to use knife (the human algorithm), as a hammer (the objective reality would be the nail).
- EVa5I7bHFq9mnYK 5y agoDunno, my dopamine is released the fastest when I'm sitting in my chair with a tv remote in hand, not running anywhere ...