10 ms·

The article makes a big unfounded assertion in one of the first few sentences "We learn them at a very early age, without being explicitly instructed by anyone

The article makes a big unfounded assertion in one of the first few sentences "We learn them at a very early age, without being explicitly instructed by anyone and just by observing the world." - we do not learn these inferences "just by observing the world", we learn these inferences by acting on the world and observing the results, and arguably they are simply not learnable by pure observation alone. There's some theoretical grounding for that (IIRC Judea Pearl's book Causality might be a good source for that) and some experimental grounding for that (e.g. the two kitten experiment for which I can't quickly find the actual paper but it's described at https://io9.gizmodo.com/the-seriously-creepy-two-kitten-experiment-1442107174 https://io9.gizmodo.com/the-seriously-creepy-two-kitten-expe... ), also other fields like medicine have a good understanding of limitations of what knowledge can be gained from purely observational studies versus experimental interventions. So it seems a bit misleading to write a whole article about "why machine learning struggles with causality" without even addressing the key difference of learning from observations vs learning from interaction, which IMHO is a much more fundamental obstacle than everything the article menstions.

deleted 6y ago
[deleted]
MeteorMarc 6y ago
Of course it makes a difference whether you observe someone acting or you are the actor yourself, but why could not you learn from someone else acting and observing the results?
- karpierz 6y ago
  You can think of "acting on the world" as resolving questions about the world, like "what happens when I do X"? You could observe the results of others acting, but it means that the questions you're getting answers to are outside of your control. So if you need to know the answer to a particular question, you either need to test it yourself, or hope that whoever you're watching will test it for you.
- t0mas88 6y ago
  It's possible to learn from someone else acting and observing the results only if you understand what the observed person is doing and could map it to yourself. Which babies for example cannot do. They like you making crazy faces but when very young they don't learn how to do the same thing from that. I think for two reasons, first they don't know what you did to make that face (which in the "experimenting yourself" case they would have known even if it was a random action) and second they can't map what they see to actions with their own body. Both need to be learned first before you can learn from observation. While learning from your own (random) interactions with the world is possible from day 1.
- PeterisP 6y ago
  A key part there is agentive intent. Pure passive observation lacks the information about which parts of the observed result were intended and expected (by the actor) and which are irrelevant noise or side-effects. In some sense the equivalent of looking at the data of a study and knowing which column represents the "intervention" and which column represents the outcome of intervention - as opposed to pure observation of those data points without knowing which is which. We can learn well from a demonstration, where someone else is acting and we correctly understand the intent and expected results (perhaps they've explicitly communicated that, perhaps we just know implicitly through our previous shared experience) and observe what happens. However, if we observe someone else - at an entirely different skill level - who is essentially testing a hypothesis (and we don't know that, we have no observation of the agent's internal) and getting useful data about where their world model diverges from reality, then it's not nearly as helpful for us to determine where our world model diverges from reality, that would require different actions and different data.
jasonwatkinspdx 6y ago
Yeah, Judea Pearl's book is the definitive work. His formalism also has an aspect that you could say is similar to imagining an intervention in a counterfactual way, and computing its likelihood given the data. His paper "Why I'm only half Bayesian" lays out how he sees the epistemology of this.
skybrian 6y ago
Both the article and the paper mention interventions and refer to Pearl’s work, so I think we can assume this is just a matter of poor phrasing and a child’s observations of its own interventions aren’t meant to be excluded. But for the purposes of machine learning, it seems like it should be possible to learn from observations of someone else’s interventions?
darksaints 6y ago
This is something I discussed with a friend recently. I talked about how the key to unlocked more potential for AI was to stop sandboxing it away from the world and let the AI start interacting with it. And the immediate pushback to that is that that would cause immediate chaos. Can you imagine AI-driven cars that out of nowhere decide to brake-check just to see what happens? Kids are often an unpleasant annoyance in restaurants, and many people that don't like that annoyance try to convince restaurants and lawmakers to ban them from restaurants. The problem with those ideas is that by banning kids from restaurants, you are just going to create annoying adults in restaurants over time. Kids are annoying in restaurants, but they are also learning how to interact with the world. If you don't find a way to let them explore boundaries, they never learn, and they'll become obnoxious restaurant patrons even as fully grown adults. Which kind of goes back to ethical AI. You can't unleash unbounded AI on the world, or else you'll cause chaos. And you can't sandbox AI, or it will never truly learn. What are you supposed to do then? I don't know, but the answer isn't firing the ethical AI department because you don't want them criticizing your ad empire ;)
- Radim 6y ago
  Makes you wonder how life operated near the beginning, before "how to behave sustainably" evolved. Before individual death, before sex, before speciation, before children, maybe even before genes. The world must have seen some wild, explosive action in its day. Immortal organisms still exist (two-headed planaria!), but on the whole the ecosystem seems pretty well calibrated by now.
  - bserge 6y ago
    Death upon death upon death, it seems.
- beaconstudios 6y ago
  The broader theoretical basis behind this idea is called embodied cognition.
- cpleppert 6y ago
  Except AI isn't being 'sandboxed away' in any discernible way. A child being an unpleasant annoyance isn't comparable to a an AI that can't drive in either risks of failure or capability because the child is far more capable of moderating his behavior to conform to social norms than an AI.
mlthoughts2018 6y ago
This has been discussed widely in computer vision for many decades - https://www.routledge.com/The-Ecological-Approach-to-Visual-Perception-Classic-Edition/Gibson/p/book/9781848725782 https://www.routledge.com/The-Ecological-Approach-to-Visual-... - http://fmdb.cs.ucla.edu/Treports/soatto_extended_v18.pdf http://fmdb.cs.ucla.edu/Treports/soatto_extended_v18.pdf
meroes 6y ago
Not saying you are wrong but I learned to ride a bike by watching my classmates be taught in preschool. Maybe direct acting is a quicker learning method. Or maybe seeing others' learning processes and instructions allows one to leapfrog ahead by not duplicating mistakes. Maybe always learn off a good dataset if it exists first?
- beaconstudios 6y ago
  First, you make a mental model. Then, you test it. If you don't test it then you don't know if it works, and you can't improve on it. Knowledge is derived from experience.
  - musingsole 6y ago
    Scoring a model's predictions are a means of testing the model without having it coupled to the system under observation. Observation is a type of experience.
    - Jtsummers 6y ago
      And you can gain experience/understanding without needing to either observe or directly experience something, merely by thinking about it or being told about it. Not every kid has to be or see a burn caused by touching a hot stove to learn it's a bad idea to try it. If you require actual experience or direct observation to learn, then you're not using your brain to its full potential.
      - shkkmo 6y ago
        You can certainly generalized previously gained knowledge without direct experience in that particular instance. Would a child who has never experienced the human body's pain response be able to infer the causal connection between the heat of a stove and the response after touching it? Arguably, language is a tool that allows us to generalize the direct experience of other agents. It is unclear if it is possible to remove direct interaction from a learning system and still reach the same level of understanding.
musingsole 6y ago
> we do not learn these inferences "just by observing the world" We most certainly can, just not as well or as strongly as when we're able to influence the system under observation. You're speaking way too strongly and simplifying a complex mechanism down past anyone's expertise.
- dumbfoundded 6y ago
  An interesting related idea is inverse reinforcement learning. We watch how other people interact with a system to estimate a reward function and then we later test it out ourselves. Either directly with an environment or inside our own mental model of the environment. Simply "observing the world" can give us data about how to learn in an environment we've never interacted with before; only by watching others interact with it.
  - still_grokking 6y ago
    That's true. But the key seems to remain the "interaction with the world".
    - dumbfoundded 6y ago
      Maybe it's semantics but interacting with a simulation of the world is actually more important. In a pure sense, this doesn't require any actual real world interaction. This concept is usually referred to offline and off-policy reinforcement learning. If you're saying that interaction in any sense is important, I'd very much agree that unsupervised learning and supervised learning aren't equipped to handle reinforcement learning problems. Correct framing of a problem is necessary to achieve a desired property like causality.
- galaxyLogic 6y ago
  Right. "World" includes us. When we perform actions we observe ourselves doing them and then we observe the consequences. That is the only way we can learn anything, by observing. And what else is there to observe than the "world"? We can observe our own thinking process but that is part of the world too. I would say. It is definitely not "out of this world" :-)
jonplackett 6y ago
I saw a study a while back that kids only a few months old will preferentially go to a person who helped their mum open a jar rather than one who refused. There’s a hell of a lot of observation happening in kids minds very early on. No doubt it’s easier once you can pick up a baseball bat yourself, but I have no doubt a young kid would understand basic objects connected together without ever having used them.
- mempko 6y ago
  Exactly. People aren't born a blank slate. We have brain structures for understanding objects in the world and also language. A baby can automatically detect speech from other sounds. Evolution is a learning process it turns out.
- watwut 6y ago
  Few monts old kids cant go anywhere. 6 month old kids are starting to crawl and then they are unable to get where they want for some time. And there is incredible amount of experimentation at that age. They dont learn things by observation alone for sure.
BenoitEssiambre 6y ago
I've read Pearl's book too and it wasn't clear if it was possible to systematically "observe interventions" instead of doing them yourself and what the rules would be for that. I mean there was some stuff about instrumental variables but it seems the theory is a bit incomplete in that area. What distinguishes an intervention vs just normal observable randomness? Does it have to do with the complexity of the entity performing the "intervention", with the fact that this entity can observe and act on knowledge? I guess it's kind of the debate about where determinism ends and free will begins. Are there mathematical bounds to help us sort it out though? Maybe there is something information theoretic? It's very unclear in my mind. This is all about fitting generative models that are robust to counterfactual changes, that remain predictive even if you run your models with data you've never observed, this beyond simple interpolation/extrapolation. Are there priors in model structure that tend to naturally make models much more robust to counterfactual changes and that make models work well beyond the data? Do these priors get more effective when they include some latent variables that distinguish between observations and interventions? How to you train these latent variables?
- zwaps 6y ago
  Pearl has a particular pet model for causality, but there are other approaches. I have yet to see someone actually apply Pearl's model in prominent research, but we are probably getting there. I think the question that is really on your mind is the one of (causal) identification. For ML researchers, this may be explained as follows: How, and to what degree, can we find a "deep" parameter that is a general or constant relationship between our inputs and outputs? Is this possible? When? Under which assumptions? If we do this, THEN we can run counterfactuals with our model. But as it turns out, to convince yourself that your model really generalizes, you will have to prove to yourself that there exists a set of general relationships (parameters) in the data - which may for example be general because they are causal parameters - and that your model is sufficient to "identify" these parameters given the data source. You will quickly find that you need to examine both the model, and the data (and assumptions that you have about it). Generally, no "model" itself is ever causal. It's always the combination of model and DGP. One way to have a causal model is to simply assume some physical relationship exists, and estimate parameters from data. This is called structural modeling in econometrics and scientific ML in, well, physics mostly. You bring in prior science and knowledge - like how weather fronts move, or how prices relate to costs - and use this structure to contrain your model. If you do this well, then this may be enough to identify causal parameters! Or ranges thereof. Here is a technical treatment of such issues: http://fmwww.bc.edu/EC-P/wp957.pdf http://fmwww.bc.edu/EC-P/wp957.pdf Another way is to find some measure of interest that you can derive without a complete model. For example, you may be able to "identify" so-called average marginal/treatment effects of some input variable without a parametric model. Then, as long as the changes in your inputs are "exogenous" in some sense, you can get a causal parameter. In experiments, the differences in inputs are randomized. What remains as group differences between placebo and treatment are the causal effects. In observational data, you may look for "natural experiments". Here, some exogenous variation can be found that identifies a similar treatment effect between individuals that are affected, and those that are not. Depending on your issue at hand, further techniques may be necessary to identify causal effects. Let's make an example: You built your start up to a huge and successful company, made up of programming teams. As CEO, you finally think about realizing your dream: Firing every non-technical team lead (say, everyone with an MBA). What effect would this have on productivity in the short term? Well, you have lots of data about your teams, and have certainly fired a lot of people, so you build a model. That model tells you, that firing such team leads is super good for productivity. However, after you do the firing, you find that productivity drops. What happened? Your model did not identify a causal effect. In this case, the firings you have observed probably occurred because the team lead was simply a bad manager. However, that data does not identify the "casual effect" of firing team leads. The "intervention" or the "treatment" you observed was not "exogenous". Okay, let's do more science here. Let's say you try to find out the effect of technical ability of the team lead on team productivity. You run your fancy ML models, and again conclude that more technical ability of the manager makes for a better team. But when you implement your effective training measures (or hiring measures, for that matter), the benefit is less than you expect. Again, what happened? Well, you might have a selection effect. Technically competent team leads may select into productive teams. Or, there is endogenity: productive teams mean that the team lead learns technical stuff, instead of managing chaos. The effect is actually reversed: good teams make leaders more competent. Your estimation - your model - overestimates the causal effect. It is not sufficient to create a counterfactual, simply because you did not find the real causal chain. Some key words to look into that are probably simpler than Pearl's full framework (and more practical, as they are applied every day): Natural Experiments, Instrumental Variables, Difference-in-Difference, Synthetic Controls, Matching, Heckman Selection If you are a person into technical papers, here's an approach that combines ML with causal analysis https://economics.mit.edu/files/12538 https://economics.mit.edu/files/12538
GistNoesis 6y ago
Formally known as "do-calculus" which helps solve the correlation is not causation issue which usually affect statistical methods (like ML).
JulianMorrison 6y ago
TBH the idea that humans learn anything on a blank slate pure learning architecture is messed up. Our brains are literally evolved to interpret causality into the world. It isn't randomly just "learned" any more than walking is.
- andyxor 6y ago
  exactly, besides ignoring the innate structures, heuristics and biases hardcoded via evolution, the whole notion of "learning" became highly intertwined with reinforcement kind of learning, i.e trial & error, stimulus and response behaviorist terms popularized by Pavlov and Skinner a century ago, which is just one type in a large repertoire of adaptation mechanisms. Memory in these models is used as afterfact, or some side utility for complex iterative routines based on calculus of function optimization. While in living organisms memory and its "hardcoded" shortcuts allow to cut through the search space quickly as in a large database index. Speaking in database terms we have something like "materialized views" on acquired and genetically inherited knowledge, built from compressed and hierarchically organized sensory data and prior related actions and associations, including causal links. Causality is just a way to associate items in the memory graph. Error correction doesn't play as much role in storing and retrieving information and pattern recognition, as current machine learning models may lead you to believe. Instead, something akin to self-organized clustering is going on, with new info embedded in the existing "concept" graph via associations and generalizations, through simple LINK and JOIN mechanisms on massive scale.[1] The formation of this graph in long term memory is tightly coupled with sleep cycles and memory consolidation, while short term memory serves as a kind of cache. Knowledge is organized hierarchically starting from principal components [2] of sensory data from e.g. visual receptive fields, and increasing in level of abstraction via "chunking", connecting objects A and B to form a new object C via JOIN mechanism, or associating objects A and B via LINK mechanism. Both LINK and JOIN outputs are "persisted" to memory via Hebbian plasticity. All knowledge including causal links are expressed via this simple mechanism. Generating a prediction given a new sensory signal is just LINKing the signal with existing cluster by similarity. Navigation in this abstract space is facilitated via coordinate system similar or perhaps identical to the role hippocampal place & grid cells play in spatial navigation. Similarity between objects is determined as similarity between their "embeddings" in this abstract concept space. It's possible that innate structures are genetically pre-wired in this graph which represent high level "schemas", such as innate language grammar which distinguishes e.g. verb from noun, visual object grammar which distinguishes "up" from "down", etc. It is also possible these are embodied, i.e. connected to some representation of motor and sensory embeddings. And serve to bootstrap the graph structure for subsequent knowledge acquisition. I.e. no blank slate. The information is passed, stored and retrieved via several (analogue) means both in point-to-point and broadcast communication, with electromagnetic oscillations playing primary role in synchronization in neural assemblies, facilitating e.g. speech segmentation (or boundary detection in general), and coupling an input signal "embedding" to existing knowledge embeddings in short term memory; while neural plasticity/LTP/STDP as storage mechanisms on single neuron level. [1] See Leslie Valiant "neuroidal" model and his book https://www.amazon.com/Circuits-Mind-Leslie-G-Valiant/dp/0195126688 https://www.amazon.com/Circuits-Mind-Leslie-G-Valiant/dp/019... [2] See Oja Rule http://www.scholarpedia.org/article/Oja_learning_rule http://www.scholarpedia.org/article/Oja_learning_rule and Olshausen & Field classic work on sparse coding http://www.scholarpedia.org/article/Sparse_coding http://www.scholarpedia.org/article/Sparse_coding
Krabbyos 6y ago
citation for the two kitten experiment: Held, R., & Hein, A. (1963). Movement-produced stimulation in the development of visually guided behavior. Journal of Comparative and Physiological Psychology, 56(5), 872–876. https://doi.org/10.1037/h0040546 https://doi.org/10.1037/h0040546
_0ffh 6y ago
Another thing which is often forgotten, we have evolutionarily adapted biases built into our learning machinery. That helps us to learn some things that tended to be essential for survival (much) more quickly, but it can also hinder us in learning some other things that we are not adapted for.
Der_Einzige 6y ago
I disagree here. We DO "learn them at a very early age, without being explicitly instructed by anyone and just by observing the world" the majority of information that a human ingests doesn't include direct actions upon the world (in any more than a philosophical sense anyway). We may feel as though actions are more memorable, but I claim that we observe than take actions, not the other way around. We learn things all the time that are not "consciously learned". Addition (e.g. 2 apples are more than one apple) is learned by most humans in an unsupervised manner far before labels are added later. I claim that early childhood development and thus much of our "foundation" is primarily rooted in having no explicit information AND not enough power to directly label data ourselves (through experiments, play, etc) This is important because "learning without explicit instructions" in ML speak is Unsupervised learning (clustering and dimensionality reduction). There are no labels except the ones that you decide upon yourself (cluster membership). Unsupervised Learning is still far in its infancy in effectiveness compared to supervised systems, and it's no surprise that its algorithms are generally extremely easy to implement from scratch (e.g. K-means or DBSCAN) compared to relatively difficult work like automatic-differentiation in neural networks. Learning by reading information in a book or by direct didactic teaching would be supervised learning. Learning through a dialectical format would be reinforcement learning . Self-supervised learning would be equivalent to autodidactic learning and the creative act upon the world. (maybe the distinction between self-supervised and reinforcement is arbitrary) The point is that we want to learn as much as we can given the information available to us. We should not rule out the role that the biological analogue to unsupervised learning plays in human development. It is for all of these reasons that I become far more excited when a new clustering or dimensionality reduction algorithm comes out than I am when a new neural network architecture becomes state of the art.
- shkkmo 6y ago
  Children do not learn completely "unsupervised" and recieve frequent feedback from the agents. I would argue that a significant amounts of childhood development, (especially around "labels"), is due to our hardwired attachment to human faces. I have always felt that the signifance of the "social software of human culture" in our general intelligence and learning capacity was underestimated by the AGI community. So personally, I see more potential in communities of learning agents than any developments in the underpinnings.
jfengel 6y ago
I suspect that we have some of it built in, even before we observe or act. Something in us starts us on the road of the hypothesis that time is a thing we can reason about. The illusion of causality is incredibly strong -- so much so that it's really hard to get people to give it up, even when faced with paradoxes (First Mover, quantum mechanics, multiple causation, etc.) We don't come into the world with a fully formed sense of causality, and it appears over the course of months. But at least some of it may be wired into the hardware, like the language instinct. It's just that the wiring isn't done the day we leave the womb, and is influenced by what comes after.
- t0mas88 6y ago
  Indeed, but it comes really early in development. A 2 months old baby understands that if you smile at other people they will interact with you. And they'll try to use the to get more interaction / playing (in a very simple way) which also makes sense evolutionary because more attention from adults means higher chance of survival.
danans 6y ago
> we learn these inferences by acting on the world and observing the results, and arguably they are simply not learnable by pure observation alone. Tangentially, wouldn't sampling based ML methods like particle filters / kalman filters or other randomized state space exploration algorithms be analogous to the person learning by acting on the world? In this case, the "action" would be bouncing the radar off the object being tracked. Of course these models are far more limited than a child in the way they can act on the world, and in the number of aspects of reality they can model. And furthermore, they have no concept of causality, and represent only the current state of knowledge they are modeling.
mcguire 6y ago
"...we do not learn these inferences "just by observing the world", we learn these inferences by acting on the world and observing the results..." The article: "“Machine learning often disregards information that animals use heavily: interventions in the world, domain shifts, temporal structure — by and large, we consider these factors a nuisance and try to engineer them away,” write the authors of the causal representation learning paper. “In accordance with this, the majority of current successes of machine learning boil down to large scale pattern recognition on suitably collected independent and identically distributed (i.i.d.) data.”" The key words are "interventions in the world." The article goes on to say, "“Generalizing well outside the i.i.d. setting requires learning not mere statistical associations between variables, but an underlying causal model,” the AI researchers write." The point being that, whether or not acting in the world is an essential condition for learning causality, current machine learning approaches are not even trying for causality.
6gvONxR4sf7o 6y ago
> we learn these inferences by acting on the world and observing the results, and arguably they are simply not learnable by pure observation alone. Also crucially, we learn these inferences by acting on the world and knowing something about why we acted. "I was playing with it" is a conditional independence statement that we use all the time while learning how things work, we just usually don't use the mathy language to describe it. We're running randomized controlled trials constantly, but implicitly. Coincidentally, it's a common anecdote that people who seem to learn things quickly and deeply have this curiosity and will play with a thing/twiddle the knobs while they're learning how it works. When you're playing a video game and someone says "hang on, let me figure out the controls for a sec" they're changing the conditional independence structure of their observations and running an RCT.
smusamashah 6y ago
We do learn these by recognizing what actions usually comes before this outcome. That's how we learn the causality. That's how we learn the wrong causality as well (required outcome after praying, that's how we created all Gods). We see what did we really do to get this. We often don't have any idea. And we put any label we find most appealing and plausible based on our previous assumptions. Machines can learn to do the same by going back one step by answering "What could have happened before" in terms of probability.

deleted 6y ago

[deleted]

MeteorMarc 6y ago

Of course it makes a difference whether you observe someone acting or you are the actor yourself, but why could not you learn from someone else acting and observing the results?

karpierz 6y ago
You can think of "acting on the world" as resolving questions about the world, like "what happens when I do X"? You could observe the results of others acting, but it means that the questions you're getting answers to are outside of your control. So if you need to know the answer to a particular question, you either need to test it yourself, or hope that whoever you're watching will test it for you.
t0mas88 6y ago
It's possible to learn from someone else acting and observing the results only if you understand what the observed person is doing and could map it to yourself. Which babies for example cannot do. They like you making crazy faces but when very young they don't learn how to do the same thing from that. I think for two reasons, first they don't know what you did to make that face (which in the "experimenting yourself" case they would have known even if it was a random action) and second they can't map what they see to actions with their own body. Both need to be learned first before you can learn from observation. While learning from your own (random) interactions with the world is possible from day 1.
PeterisP 6y ago
A key part there is agentive intent. Pure passive observation lacks the information about which parts of the observed result were intended and expected (by the actor) and which are irrelevant noise or side-effects. In some sense the equivalent of looking at the data of a study and knowing which column represents the "intervention" and which column represents the outcome of intervention - as opposed to pure observation of those data points without knowing which is which. We can learn well from a demonstration, where someone else is acting and we correctly understand the intent and expected results (perhaps they've explicitly communicated that, perhaps we just know implicitly through our previous shared experience) and observe what happens. However, if we observe someone else - at an entirely different skill level - who is essentially testing a hypothesis (and we don't know that, we have no observation of the agent's internal) and getting useful data about where their world model diverges from reality, then it's not nearly as helpful for us to determine where our world model diverges from reality, that would require different actions and different data.

jasonwatkinspdx 6y ago

Yeah, Judea Pearl's book is the definitive work. His formalism also has an aspect that you could say is similar to imagining an intervention in a counterfactual way, and computing its likelihood given the data. His paper "Why I'm only half Bayesian" lays out how he sees the epistemology of this.

skybrian 6y ago

Both the article and the paper mention interventions and refer to Pearl’s work, so I think we can assume this is just a matter of poor phrasing and a child’s observations of its own interventions aren’t meant to be excluded. But for the purposes of machine learning, it seems like it should be possible to learn from observations of someone else’s interventions?

darksaints 6y ago

This is something I discussed with a friend recently. I talked about how the key to unlocked more potential for AI was to stop sandboxing it away from the world and let the AI start interacting with it. And the immediate pushback to that is that that would cause immediate chaos. Can you imagine AI-driven cars that out of nowhere decide to brake-check just to see what happens? Kids are often an unpleasant annoyance in restaurants, and many people that don't like that annoyance try to convince restaurants and lawmakers to ban them from restaurants. The problem with those ideas is that by banning kids from restaurants, you are just going to create annoying adults in restaurants over time. Kids are annoying in restaurants, but they are also learning how to interact with the world. If you don't find a way to let them explore boundaries, they never learn, and they'll become obnoxious restaurant patrons even as fully grown adults. Which kind of goes back to ethical AI. You can't unleash unbounded AI on the world, or else you'll cause chaos. And you can't sandbox AI, or it will never truly learn. What are you supposed to do then? I don't know, but the answer isn't firing the ethical AI department because you don't want them criticizing your ad empire ;)

Radim 6y ago
Makes you wonder how life operated near the beginning, before "how to behave sustainably" evolved. Before individual death, before sex, before speciation, before children, maybe even before genes. The world must have seen some wild, explosive action in its day. Immortal organisms still exist (two-headed planaria!), but on the whole the ecosystem seems pretty well calibrated by now.
- bserge 6y ago
  Death upon death upon death, it seems.
beaconstudios 6y ago
The broader theoretical basis behind this idea is called embodied cognition.
cpleppert 6y ago
Except AI isn't being 'sandboxed away' in any discernible way. A child being an unpleasant annoyance isn't comparable to a an AI that can't drive in either risks of failure or capability because the child is far more capable of moderating his behavior to conform to social norms than an AI.

mlthoughts2018 6y ago

This has been discussed widely in computer vision for many decades - https://www.routledge.com/The-Ecological-Approach-to-Visual-Perception-Classic-Edition/Gibson/p/book/9781848725782 https://www.routledge.com/The-Ecological-Approach-to-Visual-... - http://fmdb.cs.ucla.edu/Treports/soatto_extended_v18.pdf http://fmdb.cs.ucla.edu/Treports/soatto_extended_v18.pdf

meroes 6y ago

Not saying you are wrong but I learned to ride a bike by watching my classmates be taught in preschool. Maybe direct acting is a quicker learning method. Or maybe seeing others' learning processes and instructions allows one to leapfrog ahead by not duplicating mistakes. Maybe always learn off a good dataset if it exists first?

beaconstudios 6y ago
First, you make a mental model. Then, you test it. If you don't test it then you don't know if it works, and you can't improve on it. Knowledge is derived from experience.
- musingsole 6y ago
  Scoring a model's predictions are a means of testing the model without having it coupled to the system under observation. Observation is a type of experience.
  - Jtsummers 6y ago
    And you can gain experience/understanding without needing to either observe or directly experience something, merely by thinking about it or being told about it. Not every kid has to be or see a burn caused by touching a hot stove to learn it's a bad idea to try it. If you require actual experience or direct observation to learn, then you're not using your brain to its full potential.
    - shkkmo 6y ago
      You can certainly generalized previously gained knowledge without direct experience in that particular instance. Would a child who has never experienced the human body's pain response be able to infer the causal connection between the heat of a stove and the response after touching it? Arguably, language is a tool that allows us to generalize the direct experience of other agents. It is unclear if it is possible to remove direct interaction from a learning system and still reach the same level of understanding.

musingsole 6y ago

> we do not learn these inferences "just by observing the world" We most certainly can, just not as well or as strongly as when we're able to influence the system under observation. You're speaking way too strongly and simplifying a complex mechanism down past anyone's expertise.

dumbfoundded 6y ago
An interesting related idea is inverse reinforcement learning. We watch how other people interact with a system to estimate a reward function and then we later test it out ourselves. Either directly with an environment or inside our own mental model of the environment. Simply "observing the world" can give us data about how to learn in an environment we've never interacted with before; only by watching others interact with it.
- still_grokking 6y ago
  That's true. But the key seems to remain the "interaction with the world".
  - dumbfoundded 6y ago
    Maybe it's semantics but interacting with a simulation of the world is actually more important. In a pure sense, this doesn't require any actual real world interaction. This concept is usually referred to offline and off-policy reinforcement learning. If you're saying that interaction in any sense is important, I'd very much agree that unsupervised learning and supervised learning aren't equipped to handle reinforcement learning problems. Correct framing of a problem is necessary to achieve a desired property like causality.
galaxyLogic 6y ago
Right. "World" includes us. When we perform actions we observe ourselves doing them and then we observe the consequences. That is the only way we can learn anything, by observing. And what else is there to observe than the "world"? We can observe our own thinking process but that is part of the world too. I would say. It is definitely not "out of this world" :-)

jonplackett 6y ago

I saw a study a while back that kids only a few months old will preferentially go to a person who helped their mum open a jar rather than one who refused. There’s a hell of a lot of observation happening in kids minds very early on. No doubt it’s easier once you can pick up a baseball bat yourself, but I have no doubt a young kid would understand basic objects connected together without ever having used them.

mempko 6y ago
Exactly. People aren't born a blank slate. We have brain structures for understanding objects in the world and also language. A baby can automatically detect speech from other sounds. Evolution is a learning process it turns out.
watwut 6y ago
Few monts old kids cant go anywhere. 6 month old kids are starting to crawl and then they are unable to get where they want for some time. And there is incredible amount of experimentation at that age. They dont learn things by observation alone for sure.

BenoitEssiambre 6y ago

I've read Pearl's book too and it wasn't clear if it was possible to systematically "observe interventions" instead of doing them yourself and what the rules would be for that. I mean there was some stuff about instrumental variables but it seems the theory is a bit incomplete in that area. What distinguishes an intervention vs just normal observable randomness? Does it have to do with the complexity of the entity performing the "intervention", with the fact that this entity can observe and act on knowledge? I guess it's kind of the debate about where determinism ends and free will begins. Are there mathematical bounds to help us sort it out though? Maybe there is something information theoretic? It's very unclear in my mind. This is all about fitting generative models that are robust to counterfactual changes, that remain predictive even if you run your models with data you've never observed, this beyond simple interpolation/extrapolation. Are there priors in model structure that tend to naturally make models much more robust to counterfactual changes and that make models work well beyond the data? Do these priors get more effective when they include some latent variables that distinguish between observations and interventions? How to you train these latent variables?

zwaps 6y ago
Pearl has a particular pet model for causality, but there are other approaches. I have yet to see someone actually apply Pearl's model in prominent research, but we are probably getting there. I think the question that is really on your mind is the one of (causal) identification. For ML researchers, this may be explained as follows: How, and to what degree, can we find a "deep" parameter that is a general or constant relationship between our inputs and outputs? Is this possible? When? Under which assumptions? If we do this, THEN we can run counterfactuals with our model. But as it turns out, to convince yourself that your model really generalizes, you will have to prove to yourself that there exists a set of general relationships (parameters) in the data - which may for example be general because they are causal parameters - and that your model is sufficient to "identify" these parameters given the data source. You will quickly find that you need to examine both the model, and the data (and assumptions that you have about it). Generally, no "model" itself is ever causal. It's always the combination of model and DGP. One way to have a causal model is to simply assume some physical relationship exists, and estimate parameters from data. This is called structural modeling in econometrics and scientific ML in, well, physics mostly. You bring in prior science and knowledge - like how weather fronts move, or how prices relate to costs - and use this structure to contrain your model. If you do this well, then this may be enough to identify causal parameters! Or ranges thereof. Here is a technical treatment of such issues: http://fmwww.bc.edu/EC-P/wp957.pdf http://fmwww.bc.edu/EC-P/wp957.pdf Another way is to find some measure of interest that you can derive without a complete model. For example, you may be able to "identify" so-called average marginal/treatment effects of some input variable without a parametric model. Then, as long as the changes in your inputs are "exogenous" in some sense, you can get a causal parameter. In experiments, the differences in inputs are randomized. What remains as group differences between placebo and treatment are the causal effects. In observational data, you may look for "natural experiments". Here, some exogenous variation can be found that identifies a similar treatment effect between individuals that are affected, and those that are not. Depending on your issue at hand, further techniques may be necessary to identify causal effects. Let's make an example: You built your start up to a huge and successful company, made up of programming teams. As CEO, you finally think about realizing your dream: Firing every non-technical team lead (say, everyone with an MBA). What effect would this have on productivity in the short term? Well, you have lots of data about your teams, and have certainly fired a lot of people, so you build a model. That model tells you, that firing such team leads is super good for productivity. However, after you do the firing, you find that productivity drops. What happened? Your model did not identify a causal effect. In this case, the firings you have observed probably occurred because the team lead was simply a bad manager. However, that data does not identify the "casual effect" of firing team leads. The "intervention" or the "treatment" you observed was not "exogenous". Okay, let's do more science here. Let's say you try to find out the effect of technical ability of the team lead on team productivity. You run your fancy ML models, and again conclude that more technical ability of the manager makes for a better team. But when you implement your effective training measures (or hiring measures, for that matter), the benefit is less than you expect. Again, what happened? Well, you might have a selection effect. Technically competent team leads may select into productive teams. Or, there is endogenity: productive teams mean that the team lead learns technical stuff, instead of managing chaos. The effect is actually reversed: good teams make leaders more competent. Your estimation - your model - overestimates the causal effect. It is not sufficient to create a counterfactual, simply because you did not find the real causal chain. Some key words to look into that are probably simpler than Pearl's full framework (and more practical, as they are applied every day): Natural Experiments, Instrumental Variables, Difference-in-Difference, Synthetic Controls, Matching, Heckman Selection If you are a person into technical papers, here's an approach that combines ML with causal analysis https://economics.mit.edu/files/12538 https://economics.mit.edu/files/12538

GistNoesis 6y ago

Formally known as "do-calculus" which helps solve the correlation is not causation issue which usually affect statistical methods (like ML).

JulianMorrison 6y ago

TBH the idea that humans learn anything on a blank slate pure learning architecture is messed up. Our brains are literally evolved to interpret causality into the world. It isn't randomly just "learned" any more than walking is.

andyxor 6y ago
exactly, besides ignoring the innate structures, heuristics and biases hardcoded via evolution, the whole notion of "learning" became highly intertwined with reinforcement kind of learning, i.e trial & error, stimulus and response behaviorist terms popularized by Pavlov and Skinner a century ago, which is just one type in a large repertoire of adaptation mechanisms. Memory in these models is used as afterfact, or some side utility for complex iterative routines based on calculus of function optimization. While in living organisms memory and its "hardcoded" shortcuts allow to cut through the search space quickly as in a large database index. Speaking in database terms we have something like "materialized views" on acquired and genetically inherited knowledge, built from compressed and hierarchically organized sensory data and prior related actions and associations, including causal links. Causality is just a way to associate items in the memory graph. Error correction doesn't play as much role in storing and retrieving information and pattern recognition, as current machine learning models may lead you to believe. Instead, something akin to self-organized clustering is going on, with new info embedded in the existing "concept" graph via associations and generalizations, through simple LINK and JOIN mechanisms on massive scale.[1] The formation of this graph in long term memory is tightly coupled with sleep cycles and memory consolidation, while short term memory serves as a kind of cache. Knowledge is organized hierarchically starting from principal components [2] of sensory data from e.g. visual receptive fields, and increasing in level of abstraction via "chunking", connecting objects A and B to form a new object C via JOIN mechanism, or associating objects A and B via LINK mechanism. Both LINK and JOIN outputs are "persisted" to memory via Hebbian plasticity. All knowledge including causal links are expressed via this simple mechanism. Generating a prediction given a new sensory signal is just LINKing the signal with existing cluster by similarity. Navigation in this abstract space is facilitated via coordinate system similar or perhaps identical to the role hippocampal place & grid cells play in spatial navigation. Similarity between objects is determined as similarity between their "embeddings" in this abstract concept space. It's possible that innate structures are genetically pre-wired in this graph which represent high level "schemas", such as innate language grammar which distinguishes e.g. verb from noun, visual object grammar which distinguishes "up" from "down", etc. It is also possible these are embodied, i.e. connected to some representation of motor and sensory embeddings. And serve to bootstrap the graph structure for subsequent knowledge acquisition. I.e. no blank slate. The information is passed, stored and retrieved via several (analogue) means both in point-to-point and broadcast communication, with electromagnetic oscillations playing primary role in synchronization in neural assemblies, facilitating e.g. speech segmentation (or boundary detection in general), and coupling an input signal "embedding" to existing knowledge embeddings in short term memory; while neural plasticity/LTP/STDP as storage mechanisms on single neuron level. [1] See Leslie Valiant "neuroidal" model and his book https://www.amazon.com/Circuits-Mind-Leslie-G-Valiant/dp/0195126688 https://www.amazon.com/Circuits-Mind-Leslie-G-Valiant/dp/019... [2] See Oja Rule http://www.scholarpedia.org/article/Oja_learning_rule http://www.scholarpedia.org/article/Oja_learning_rule and Olshausen & Field classic work on sparse coding http://www.scholarpedia.org/article/Sparse_coding http://www.scholarpedia.org/article/Sparse_coding

Krabbyos 6y ago

citation for the two kitten experiment: Held, R., & Hein, A. (1963). Movement-produced stimulation in the development of visually guided behavior. Journal of Comparative and Physiological Psychology, 56(5), 872–876. https://doi.org/10.1037/h0040546 https://doi.org/10.1037/h0040546

_0ffh 6y ago

Another thing which is often forgotten, we have evolutionarily adapted biases built into our learning machinery. That helps us to learn some things that tended to be essential for survival (much) more quickly, but it can also hinder us in learning some other things that we are not adapted for.

Der_Einzige 6y ago

I disagree here. We DO "learn them at a very early age, without being explicitly instructed by anyone and just by observing the world" the majority of information that a human ingests doesn't include direct actions upon the world (in any more than a philosophical sense anyway). We may feel as though actions are more memorable, but I claim that we observe than take actions, not the other way around. We learn things all the time that are not "consciously learned". Addition (e.g. 2 apples are more than one apple) is learned by most humans in an unsupervised manner far before labels are added later. I claim that early childhood development and thus much of our "foundation" is primarily rooted in having no explicit information AND not enough power to directly label data ourselves (through experiments, play, etc) This is important because "learning without explicit instructions" in ML speak is Unsupervised learning (clustering and dimensionality reduction). There are no labels except the ones that you decide upon yourself (cluster membership). Unsupervised Learning is still far in its infancy in effectiveness compared to supervised systems, and it's no surprise that its algorithms are generally extremely easy to implement from scratch (e.g. K-means or DBSCAN) compared to relatively difficult work like automatic-differentiation in neural networks. Learning by reading information in a book or by direct didactic teaching would be supervised learning. Learning through a dialectical format would be reinforcement learning . Self-supervised learning would be equivalent to autodidactic learning and the creative act upon the world. (maybe the distinction between self-supervised and reinforcement is arbitrary) The point is that we want to learn as much as we can given the information available to us. We should not rule out the role that the biological analogue to unsupervised learning plays in human development. It is for all of these reasons that I become far more excited when a new clustering or dimensionality reduction algorithm comes out than I am when a new neural network architecture becomes state of the art.

shkkmo 6y ago
Children do not learn completely "unsupervised" and recieve frequent feedback from the agents. I would argue that a significant amounts of childhood development, (especially around "labels"), is due to our hardwired attachment to human faces. I have always felt that the signifance of the "social software of human culture" in our general intelligence and learning capacity was underestimated by the AGI community. So personally, I see more potential in communities of learning agents than any developments in the underpinnings.

jfengel 6y ago

I suspect that we have some of it built in, even before we observe or act. Something in us starts us on the road of the hypothesis that time is a thing we can reason about. The illusion of causality is incredibly strong -- so much so that it's really hard to get people to give it up, even when faced with paradoxes (First Mover, quantum mechanics, multiple causation, etc.) We don't come into the world with a fully formed sense of causality, and it appears over the course of months. But at least some of it may be wired into the hardware, like the language instinct. It's just that the wiring isn't done the day we leave the womb, and is influenced by what comes after.

t0mas88 6y ago
Indeed, but it comes really early in development. A 2 months old baby understands that if you smile at other people they will interact with you. And they'll try to use the to get more interaction / playing (in a very simple way) which also makes sense evolutionary because more attention from adults means higher chance of survival.

danans 6y ago

> we learn these inferences by acting on the world and observing the results, and arguably they are simply not learnable by pure observation alone. Tangentially, wouldn't sampling based ML methods like particle filters / kalman filters or other randomized state space exploration algorithms be analogous to the person learning by acting on the world? In this case, the "action" would be bouncing the radar off the object being tracked. Of course these models are far more limited than a child in the way they can act on the world, and in the number of aspects of reality they can model. And furthermore, they have no concept of causality, and represent only the current state of knowledge they are modeling.

mcguire 6y ago

"...we do not learn these inferences "just by observing the world", we learn these inferences by acting on the world and observing the results..." The article: "“Machine learning often disregards information that animals use heavily: interventions in the world, domain shifts, temporal structure — by and large, we consider these factors a nuisance and try to engineer them away,” write the authors of the causal representation learning paper. “In accordance with this, the majority of current successes of machine learning boil down to large scale pattern recognition on suitably collected independent and identically distributed (i.i.d.) data.”" The key words are "interventions in the world." The article goes on to say, "“Generalizing well outside the i.i.d. setting requires learning not mere statistical associations between variables, but an underlying causal model,” the AI researchers write." The point being that, whether or not acting in the world is an essential condition for learning causality, current machine learning approaches are not even trying for causality.

6gvONxR4sf7o 6y ago

> we learn these inferences by acting on the world and observing the results, and arguably they are simply not learnable by pure observation alone. Also crucially, we learn these inferences by acting on the world and knowing something about why we acted. "I was playing with it" is a conditional independence statement that we use all the time while learning how things work, we just usually don't use the mathy language to describe it. We're running randomized controlled trials constantly, but implicitly. Coincidentally, it's a common anecdote that people who seem to learn things quickly and deeply have this curiosity and will play with a thing/twiddle the knobs while they're learning how it works. When you're playing a video game and someone says "hang on, let me figure out the controls for a sec" they're changing the conditional independence structure of their observations and running an RCT.

smusamashah 6y ago

We do learn these by recognizing what actions usually comes before this outcome. That's how we learn the causality. That's how we learn the wrong causality as well (required outcome after praying, that's how we created all Gods). We see what did we really do to get this. We often don't have any idea. And we put any label we find most appealing and plausible based on our previous assumptions. Machines can learn to do the same by going back one step by answering "What could have happened before" in terms of probability.