4 ms·

> ML was not necessary to recognize the yield curve inversion as a strongly predictive signal correlating to subsequent contraction. > An NN can certainly lear

by zwaps 7y ago

> ML was not necessary to recognize the yield curve inversion as a strongly predictive signal correlating to subsequent contraction. > An NN can certainly learn to predict according to the presence or magnitude of a yield curve inversion and which combinations of other features. > - [ ] Exercise: Learning this and other predictive signals by cherry-picking data and hand-optimizing features may be an extremely appropriate exercise. If the financial crisis has not yet occurred, how will the NN learn a relationship that does not exist in the data? The exercise of cherry picking data and hand-optimizing is equivalent to applying theory to your statistical problem. It is what is required if you lack data points - using ML or otherwise. Nevertheless, we (as in humans) are bad at it. Speaking of the financial crisis. It was not AI's that picked up on it, it was some guys applying sophisticated and deep understanding of causal relationships. And that so few people did this, shows how bad we humans are at doing this implicitly and automatically by just looking at data! > Maybe we're not in agreement about whether AI and ML can do causal inference just as well if not better than humans manipulating symbols with human cognition and physical world intuition. The time is nigh! In general, while skepticism and caution are appropriate, many fields suffer from a degree of hubris which prevents them from truly embracing stronger AI in their problem domain. (A human person cannot mutate symbol trees and validate with shuffled and split test data all night long) ML and AI certainly can do causal inference. But then you have to do causal inference. Again, prediction on historical data is not equivalent to causal analysis, and neither is backtesting or validation. At the end of the day, AI and ML improves on predictions, but the distinction of causal analysis is a qualitative one. > I read this as "must be biased by the literature and willing to disregard an unacceptable error term"; but also caution against rationalizing blind findings which can easily be rationalized as logical due to any number of cognitive biases. No. My point is that for causal analysis, you have to leverage assumptions that are beyond your data set. Where these come from is besides the point. You will always employ a theory, implicitly or explicitly. The major issue is not the we use theories, but rather that we might do it implicitly, hiding the assumptions about the DGP that allows causal inference. This is where humans are bad. Theories are just theories. With precise assumptions giving us causal identification, we are in a good position to argue where we stand. If we just run algorithms without really understand what is going on, we are just repeating the mistakes from the last forty years! > If you're suggesting that the information theory that underlies AI and ML is insufficient to learn what we humans have learned in a few hundred years of observing and attempting to optimize, I must disagree (regardless of the hardness or softness of the given complex field). Beyond a few combinations/scenarios, our puny little brains are no match for our department's new willing AI scientist. All the information theory I have seen in any of the Machine Learning textbooks I have picked up is methodologically equivalent to statistics. In particular, the standard textbooks (Elements, Murhpy etc.) treatment of information theory would only allow causal identification under the exact same conditions that the statistics literature treats. I fail to see the difference, or what AI in particular adds. The issue of causal inference is a "hot topic" in many fields, including AI, but the underlying philosophical issues are not exactly new. This includes information theory. You seem to think that ML has somehow solved this problem. From my reading of these books, I certainly disagree. Causal inference is certainly POSSIBLE - just as in statistics, but ML doesn't give it to you for free! In particular, note the following issue: To show causal identification, you need to make assumptions on your DGP (exogenous variation, timing, graphical causal relations ... whatever). Even if these assumptions are very implicit, they do exist. Just by looking at data, and running a model, you do not get causal inference. It can not be done "within" the system/model. If you bake these things into your AI, then it, too makes these assumptions. There really is no difference. For example, I could imagine an AI that can identify likely exogenous variations in the data and use them to predict counterfactuals. That's probably not too far off, if it doesn't exist already. But, this is still based on the assumption that these variations are, indeed exogenous, which can never be proven within the DGP. In contrast, I find that most "AI scientists" care very much about prediction, and very little about causal inference. I don't mean this subfield doesn't exist. But it is a subfield. In contrast, for many non-AI scientists, causal inference IS the fundamental question, and prediction is only an afterthought. ML in practice involved doing correct experiments (AB testing), at best. It will sooner or later also adopt all other causal inference techniques. But, my point stands, I have yet to see what ML adds. Enlighten me! AI, ML and stats will merge, if they haven't already. The distinction will disappear. I believe the issues will not. I employ a lot of AI/ML techniques in my scientific work. Never have they solved the underlying issue of causal inference for me!

westurner 7y ago
A causal model is a predictive model. We must validate the error of a causal model. Why are theoretic models hand-wavy? "That's just because noise, the model is correct." No, such a model is insufficient to predict changes in dependent variables when in the presence of noise; which is always the case. How does validating a causal model differ from validating a predictive model with historical and future data? Yield-curve inversion as a signal can be learned by human and artificial NNs. Period. There are a few false positives in historical data: indeed, describe the variance due to "noise" by searching for additional causal and correlative relations in additional datasets. I searched for "python causal inference" and found a few resources on the first page of search results: https://www.google.com/search?q=python+causal+inference https://www.google.com/search?q=python+causal+inference CausalInference: https://pypi.org/project/CausalInference/ https://pypi.org/project/CausalInference/ DoWhy: https://github.com/microsoft/dowhy https://github.com/microsoft/dowhy CausalImpact (Python port of the R package): https://github.com/dafiti/causalimpact https://github.com/dafiti/causalimpact "What is the best Python package for causal inference?" https://www.quora.com/What-is-the-best-Python-package-for-causal-inference https://www.quora.com/What-is-the-best-Python-package-for-ca... Search: graphical model "information theory" [causal] https://www.google.com/search?q=graphical+model+%22information+theory%22 https://www.google.com/search?q=graphical+model+%22informati... Search: opencog causal inference https://www.google.com/search?q=opencog+causal+inference https://www.google.com/search?q=opencog+causal+inference (MOSES, PLN,) If you were to write a pseudocode algorithm for an econometric researcher's process of causal inference (and also their cognitive processes (as executed in a NN with a topology)), how would that read? (Edit) Something about the sufficiency of RL (Reinforcement Learning) for controlling cybernetic systems. https://en.wikipedia.org/wiki/Cybernetics https://en.wikipedia.org/wiki/Cybernetics
- zwaps 7y ago
  I am at loss at what you want to say to me, but let me reiterate: Any learning model by itself is a statistical model. Statistical models are never automatically causal models, albeit causal models are statistical models. Several causal models can be observationally equivalent to a single statistical model, but the substantive (inferential) implications on doing "an intervention" on the DGP differ. It is therefore not enough to validate and run on a model on data. Several causal models WILL validate on the same data, but their implications are drastically different. The data ALONE provides you no way to differentiate (we say, identify) the correct causal model without any further restrictions. By extension, it is impossible for any ML mechanism to predict unobserved interventions without being a causal model. ML and AI models CAN be causal models, which is the case if they are based on further assumptions about the DGP. For example, they may be graphical models, SCM/SEM etc. These restrictions can be derived algorithmically, based on all sorts of data, tuning, coding and whatever. It really doesn't change the distinction between causal and statistical analysis. The way these models become causal is based on assumptions that constitute a theory in the scientific sense. These theories can then of course also be validated. But this is not based on learning from historical data alone. You always have to impose sufficient restrictions on your model (e.g. the DGP) to make such causal inference. This is not new, but for your benefit, I basically transferred the above from an AI/ML book on causal analysis. AI/ML can do causal analysis, because its statistics. AI/ML are not separate from these issues, do not solve these issues ex-ante, are not "better" than other techniques except on the dimensions that they are better as statistical techniques, AND, most importantly, causal application necessarily implies a theory. Whether this is implicit or explicit is up to the researcher, but there are dangers associated with implicit causal reasoning. And as Pearl wrote (who is not a fan of econometrics by any means!), the issue of causal inference was FIRST raised by econometricians BASED on combining the structure of economic models with statistical inference. In the 1940's. I mean I get the appeal to trash talk social sciences, but when it comes to causal inference, you probably picked exactly the wrong one. You are free to disregard economic theory. But you can not claim to do causal analysis without any theory. Doing so implicitly is dangerous. Furthermore, you are wrong in the sense that economic theory has put causal inference issues at the forefront of econometric research, and is therefore good for science even if you dislike those theories.

by zwaps 7y ago

westurner 7y ago

A causal model is a predictive model. We must validate the error of a causal model. Why are theoretic models hand-wavy? "That's just because noise, the model is correct." No, such a model is insufficient to predict changes in dependent variables when in the presence of noise; which is always the case. How does validating a causal model differ from validating a predictive model with historical and future data? Yield-curve inversion as a signal can be learned by human and artificial NNs. Period. There are a few false positives in historical data: indeed, describe the variance due to "noise" by searching for additional causal and correlative relations in additional datasets. I searched for "python causal inference" and found a few resources on the first page of search results: https://www.google.com/search?q=python+causal+inference https://www.google.com/search?q=python+causal+inference CausalInference: https://pypi.org/project/CausalInference/ https://pypi.org/project/CausalInference/ DoWhy: https://github.com/microsoft/dowhy https://github.com/microsoft/dowhy CausalImpact (Python port of the R package): https://github.com/dafiti/causalimpact https://github.com/dafiti/causalimpact "What is the best Python package for causal inference?" https://www.quora.com/What-is-the-best-Python-package-for-causal-inference https://www.quora.com/What-is-the-best-Python-package-for-ca... Search: graphical model "information theory" [causal] https://www.google.com/search?q=graphical+model+%22information+theory%22 https://www.google.com/search?q=graphical+model+%22informati... Search: opencog causal inference https://www.google.com/search?q=opencog+causal+inference https://www.google.com/search?q=opencog+causal+inference (MOSES, PLN,) If you were to write a pseudocode algorithm for an econometric researcher's process of causal inference (and also their cognitive processes (as executed in a NN with a topology)), how would that read? (Edit) Something about the sufficiency of RL (Reinforcement Learning) for controlling cybernetic systems. https://en.wikipedia.org/wiki/Cybernetics https://en.wikipedia.org/wiki/Cybernetics

zwaps 7y ago
I am at loss at what you want to say to me, but let me reiterate: Any learning model by itself is a statistical model. Statistical models are never automatically causal models, albeit causal models are statistical models. Several causal models can be observationally equivalent to a single statistical model, but the substantive (inferential) implications on doing "an intervention" on the DGP differ. It is therefore not enough to validate and run on a model on data. Several causal models WILL validate on the same data, but their implications are drastically different. The data ALONE provides you no way to differentiate (we say, identify) the correct causal model without any further restrictions. By extension, it is impossible for any ML mechanism to predict unobserved interventions without being a causal model. ML and AI models CAN be causal models, which is the case if they are based on further assumptions about the DGP. For example, they may be graphical models, SCM/SEM etc. These restrictions can be derived algorithmically, based on all sorts of data, tuning, coding and whatever. It really doesn't change the distinction between causal and statistical analysis. The way these models become causal is based on assumptions that constitute a theory in the scientific sense. These theories can then of course also be validated. But this is not based on learning from historical data alone. You always have to impose sufficient restrictions on your model (e.g. the DGP) to make such causal inference. This is not new, but for your benefit, I basically transferred the above from an AI/ML book on causal analysis. AI/ML can do causal analysis, because its statistics. AI/ML are not separate from these issues, do not solve these issues ex-ante, are not "better" than other techniques except on the dimensions that they are better as statistical techniques, AND, most importantly, causal application necessarily implies a theory. Whether this is implicit or explicit is up to the researcher, but there are dangers associated with implicit causal reasoning. And as Pearl wrote (who is not a fan of econometrics by any means!), the issue of causal inference was FIRST raised by econometricians BASED on combining the structure of economic models with statistical inference. In the 1940's. I mean I get the appeal to trash talk social sciences, but when it comes to causal inference, you probably picked exactly the wrong one. You are free to disregard economic theory. But you can not claim to do causal analysis without any theory. Doing so implicitly is dangerous. Furthermore, you are wrong in the sense that economic theory has put causal inference issues at the forefront of econometric research, and is therefore good for science even if you dislike those theories.