8 ms·

Depending on how complex a curve, and how many dimensions it is in, couldn't you argues that is essentially what our brains do as well? Not that I am defending

Depending on how complex a curve, and how many dimensions it is in, couldn't you argues that is essentially what our brains do as well? Not that I am defending the massive hype field that is ML today, but curve fitting is a form of intelligence.

crazygringo 7y ago
I was say fundamentally no -- and Douglas Hofstadter is probably the foremost spokesperson against intelligence being curve fitting. General intelligence is primarily about developing useful conceptual categories (not mapping to existing ones) and drawing cause-and-effect inferences that assist us in achieving goals. Curve fitting is just another name for pattern recognition, mapping to previously defined categories. I would personally argue there's no intelligence there whatsoever. Intelligence can't exist without a foundation of pattern recognition, but it isn't the same thing. Intelligence is fundamentally goal-directed and able to reason, while curve-fitting is fundamentally not. (There is also unsupervised learning in deep learning, which doesn't use previously defined categories, but since it is similarly non-goal-directed, I would still argue that this is merely dimension reduction as opposed to intelligence -- useful for sure, but not the same.)
- codingslave 7y ago
  We dont know anything about AGI, and we arent sure how it would work. Its impossible to make assertions like this
  - quikoa 7y ago
    Yea I think we just don't know enough. Even if curve fitting alone wouldn't be sufficient it may very well play a crucial role in the agi algorithm.
  - deleted 7y ago
    [deleted]
  - slx26 7y ago
    we don't know everything, but we definitely know something about current ML, and we can indeed speculate and make some educated guesses about AGI. for a random comment on the internet, that contained some interesting information. of course, an actual implementation of AGI might be anything, but the comment was broader than that
  - chongli 7y ago
    We know a great deal more about biological general intelligence than we do about AGI. Many animals create their own goals and work to achieve them. We learn what works and what does not, and adapt our strategies to compensate. Humans do this (obviously) but a lot of other intelligent animals can do it as well. One of my favourite examples is the New Caledonian crows who have learned to use traffic to crack nuts [1]. Here, a crow had no pre-defined objective function apart from "eat food to stay alive" and has accomplished something remarkable. It found a food source that it had never had access to before, it developed a complex model of its urban environment, it combined its knowledge of the problem (the hard nut shell) with its knowledge of its environment (cars crush small objects), and it constructed a sophisticated for strategy for using cars to crack open the nuts and fetching the contents when the traffic lights indicated it was safe to do so. This is general intelligence! [1] https://www.youtube.com/watch?v=BGPGknpq3e0 https://www.youtube.com/watch?v=BGPGknpq3e0
- petra 7y ago
  Reinforcement learning is unsupervised and goal directed.
- zozbot234 7y ago
  It's not like there's a huge difference. Causal inference is a special case of curve-fitting, where one chooses what variables should enter into the fit according to a causal graph.
- gus_massa 7y ago
  > General intelligence is primarily about developing useful conceptual categories (not mapping to existing ones) … There are some algorithms like https://en.wikipedia.org/wiki/K-means_clustering https://en.wikipedia.org/wiki/K-means_clustering that get a set of data and try to create the categories to better classify them. There are many algorithm and the results don't agree all the time. But this is an open ended task, like the classification of biological species in animals. (Plants are more difficult, and bacterias even more.)
- deleted 7y ago
  [deleted]
- bobcostas55 7y ago
  By that definition the vast majority of humans do not possess general intelligence.
- jimduk 7y ago
  Computer Vision and Pattern Recognition. Was a good name, still will be. Great progress has been made, but it's not AGI as anyone who goes to CVPR will tell you.
ur-whale 7y ago
First, no AI that I know of has at its disposal a full blown model of the world it operates in, whereas most human brains do, and even if the model is imperfect, it is capable of producing fairly accurate simulations (what-if scenarios). Second, deep learning model, however much we'd like to think they do, aren't capable of doing proper causal inference in a general setting (that is, within the confines of the model) and are therefore far from capable of doing what humans do and will remain so limited for a long time to come. AGI will require the curve-fitting of deep learning, a general model of the world, the causal inference capabilities of something like AlphaGo, but in a general setting, not the super limited world AlphaGo operates in. So no, AGI will require much more than just curve-fitting abilities.
- elcomet 7y ago
  But all this is curve-fitting in a more general sense - fitting the curve of life, gene reproduction. So it is still curve fitting, it is just that a pottential AGI is certainly not in the hypothesis space of current deep learning models, and those cannot reach AGI by curve-fitting.
- huffmsa 7y ago
  "casual inference in a general setting" is just your brain running an input through it's existing thought and decision processes with a low threshold for a passing answer. So an ML model running an input through a collection of other models to see if it gets a reasonable answer.
- longtom 7y ago
  It's all pretty meaningless semantics and guesswork. Curve fitting means adaptive computation in networks of fairly simple units that allow for fairly general computation, i.e. traversing program space to find a good solution, or equivalently, intelligence is about evolving/searching a program that solves a wide array of tasks. It is about finding programs that maps from sensory space to the space of action sequences, maximizing reward. But you need the right prior structure such that learning and producing action sequences is efficient or even feasible/reachable. You can see any additional program structure that aids e.g. generation and recall of memories and planning (production of output targeted at solving a goal) as prior structure that limits and defines the searched program space. You can even regard a planning module as part of the curve fitting as it simply concerns the last step of producing the output. Therefore, intelligence is "curve fitting". So the actual question is: How much additional structure over just a large number of simple repeated units is necessary? Nobody knows. Possibly not much. Possibly quite a bit.
yk 7y ago
That is basically what Chalmers argues in "Facing up the problem of consciousness." Basically, by the general approximation theorem, it is possible to find a neural network that acts externally precisely as you would, up to an arbitrarily small epsilon. However, one wonders if such a thing would be consciousness and if so, where does the consciousness sit, in the matrix multiplication or the graphics card. So, you can argue something like our brains are nothing but curve fitting machine with enough parameters, but then you are probably forced to argue that consciousness is very closely related to computation, to the point were a coin flip, or a hello world program has some sliver of consciousness. There are of course two possible ways around that, either one can argue for p-zombies, that is intelligent but not conscious beings, which then seems to require a super natural explanation for consciousness. Or you can argue that the brain is different, and that this gives rise to consciousness, and to general intelligence, which is the explanation that at least corresponds most closely to my subjective experience (but that is precisely what a soulless machine would write, isn't it?)
- joycian 7y ago
  Why is it weird that simple programs have a sliver of consciousness? That seems like a nice practical, non-magic conclusion?
  - yk 7y ago
    Actually I like that solution, but try to show it to any degree of scientific standard. You need to have a good working definition of consciousness, then you need some experimental procedure, etc. etc.
- missosoup 7y ago
  There's no need for a supernatural explanation of consciousness. We are all p-zombies. Problem solved. The longer we refuse to acknowledge that consciousness is nothing special, the longer it will take to tackle this topic. The only reason we cling to the idea that our minds are somehow special compared to other animals of various complexity is because we refuse to acknowledge that consciousness might exist in something we can't communicate with and that consciousness is a sliding scale rather than a binary property. Ascribing consciousness only to ourselves is hubris. If one spends a bit of time observing humans, they will inevitably realise that some humans are more 'conscious' than others also. tl;dr: we're all p-zombies. The fact that we think that each one of us isn't individually doesn't detract from that. Just as natural sciences left less and less hiding places for god to exist, ML is leaving less and less hiding places for this borderline magical version of human-unique consciousness to exist. Answering this question in any more detail requires a much more rigorous definition of consciousness which is a big can of worms in itself.
Isinlor 7y ago
If someone wants to go as far as to claim that any computation is just curve-fitting then your statement is equivalent to Church–Turing thesis. There are no formal arguments against Church–Turing thesis. From that perspective intelligence is indeed just a curve fitting. https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis I really enjoyed the "The Measure of Intelligence" by François Chollet. https://arxiv.org/abs/1911.01547 https://arxiv.org/abs/1911.01547 "We note that in practice, the contemporary AI community still gravitates towards benchmarking intelligence by comparing the skill exhibited by AIs and humans at specific tasks such as board games and video games. We argue that solely measuring skill at any given task falls short of measuring intelligence, because skill is heavily modulated by prior knowledge and experience: unlimited priors or unlimited training data allow experimenters to "buy" arbitrary levels of skills for a system, in a way that masks the system's own generalization power." He argues that we should move towards evaluating "Intelligence as skill-acquisition efficiency". I agree with him. We should move away from benchmarks that involve training and evaluating algorithms on the same datasets. This is indeed more or less "curve fitting". Instead we should focus on benchmarking how efficient algorithms are at solving tasks involving completely new datasets, preferably even unknown to the developers. For example, language model GPT-2 was trained to predict next word given some previous words. After that training GPT-2 was able to do things that were unrelated like question answering, translating etc. GPT-2 is of course doing that very badly, and requires GB of training data, but it is a step towards skill-acquisition efficiency and away from what everyone sees as curve-fitting. We should benchmark models so that we select for these that are able to do solve tasks they were not build to solve.
- sgt101 7y ago
  >There are no formal arguments against Church–Turing thesis. do you mean "all computation that can be done, can be done using a Turing machine"? Or do you mean "no one has proven Church Turing wrong?" If it's the second - yes, that's so. But so what? If it's the first then many people in Quantum Computing community will be quite upset, P=BQP? You have proof?
  - voidmain 7y ago
jerf 7y ago
I am not accusing you of doing this in a bad way, but it kinda begs the question to argue that all intelligence is curve fitting, because if it is to be looked at that way, then we don't know the characteristics of the hyperspace on which it can be defined as "just" curve fitting, which is an important aspect of being able to carefully say that it is "just" curve fitting. While there almost certainly is such a hyperspace, just by virtue of the fact we're leaving ourselves so many degrees of freedom in our unspecified speculations here that it can't help but cover everything we ever do, we can't be confident that if we did know everything involved that it would be anywhere near the best representation, which is the only standard we can have in terms of whether or not something is "really" curve fitting. It isn't that hard to imagine that while an n-dimensional space can be defined upon which our intelligence is "curve fitting" that there is some better computational paradigm that would both describe it in fewer free parameters (or, in this case, a big-O different number of free parameters, probably) and also be easier to work with computationally, in which case we would have a solid ground to stand on to say that, no, it isn't really just "curve fitting". Or to bring it down to Earth in another way, consider just the act of writing a program in the modern world. If you work really, really hard, you can define a space in which our act of programming is just "curve fitting"... but it's far from obvious that that is even remotely a sensible way to look at the world. (See "differentiable programming" for the best counterpoint I know to that: https://en.wikipedia.org/wiki/Differentiable_programming https://en.wikipedia.org/wiki/Differentiable_programming but it's a very small niche right now.) When I'm debugging a program there is almost never any utility at all in trying to think about it as a "curve" and trying to get it closer to a the "correct" curve. A Turing-complete-complex space can be described as curves, but those curves are just awfully complicated and I don't see how it would be a help. My personal suspicion is that while our cognition involves rather less of this "Turing complete" thinking than we'd like to fancy ourselves using, we do irreducibly use elements of it [1], and as long as our best AI models are incapable of representing Turing-complete computations there is simply no chance of them being the answer to true human-scale cognition. (We do have models that can do it, e.g., evolutionary computation, but we lack any sensible idea of how to "update" such models like a neural net. Neural nets themselves in the simplest case aren't Turing complete, and none of the hybrid models seem to get there to me either, though I welcome correction on that point.) [1]: Evidence: I don't think we could program Turing-complete machines if we were incapable of thinking that way ourselves. We aren't necessarily great at it, our engineering techniques are deeply characterized by the fact we can't really manipulate very many things at once in this manner and we have no choice but to break things up into very small modules and for us to combine them in a way that means that at any given time we have only a very small number of things to keep track of locally, but we are still doing non-trivially more than zero of the Turing-complete style of thinking. It isn't a hard guess from there to think that even if we aren't all that great at a full mathematical manifestation of this style of thinking, we may indeed be doing something somewhere between what our current neural nets do and this full TC-style thinking at a larger scale, and the inability to capture this in our neural nets is a currently-fatal-flaw.
bitxbit 7y ago
It’s just fancy pattern matching. Multi-stage heuristics. Where it fails is data limitations. This is different from someone coming up with new insights based on different combinations of data inputs and better yet entirely new data/metrics.
zby 7y ago
What our brains do is curve fitting plus experiments. Causality is learned through experiments - and as children we do a lot of experiments (with moving our limbs etc). With just observations you can have only correlations. For example you can correlate smoke with fire, but only through experiment you can learn that it is fire that causes smoke not the other way around.
- manx 7y ago
  This is exactly what I thought, but was not able to put into words. Thank you.
- mr_toad 7y ago
  Just include time in the model and use lagged features. It’s used all the time in automated time series forecasting.
  - tomrod 7y ago
    This can tease out transfer entropy but still isn't identifying causal factors. A common issue you find would be confounding. Then, because you haven't identified the latent connection, you may try to increase level A to have effect on output B, and be disappointed. This is basically Judea Pearls Book of Why's main hypothesis, that E(Y|X) != E(Y|do(X)), where do(X) is when we modify X somehow.

crazygringo 7y ago

I was say fundamentally no -- and Douglas Hofstadter is probably the foremost spokesperson against intelligence being curve fitting. General intelligence is primarily about developing useful conceptual categories (not mapping to existing ones) and drawing cause-and-effect inferences that assist us in achieving goals. Curve fitting is just another name for pattern recognition, mapping to previously defined categories. I would personally argue there's no intelligence there whatsoever. Intelligence can't exist without a foundation of pattern recognition, but it isn't the same thing. Intelligence is fundamentally goal-directed and able to reason, while curve-fitting is fundamentally not. (There is also unsupervised learning in deep learning, which doesn't use previously defined categories, but since it is similarly non-goal-directed, I would still argue that this is merely dimension reduction as opposed to intelligence -- useful for sure, but not the same.)

codingslave 7y ago
We dont know anything about AGI, and we arent sure how it would work. Its impossible to make assertions like this
- quikoa 7y ago
  Yea I think we just don't know enough. Even if curve fitting alone wouldn't be sufficient it may very well play a crucial role in the agi algorithm.
- deleted 7y ago
  [deleted]
- slx26 7y ago
  we don't know everything, but we definitely know something about current ML, and we can indeed speculate and make some educated guesses about AGI. for a random comment on the internet, that contained some interesting information. of course, an actual implementation of AGI might be anything, but the comment was broader than that
- chongli 7y ago
  We know a great deal more about biological general intelligence than we do about AGI. Many animals create their own goals and work to achieve them. We learn what works and what does not, and adapt our strategies to compensate. Humans do this (obviously) but a lot of other intelligent animals can do it as well. One of my favourite examples is the New Caledonian crows who have learned to use traffic to crack nuts [1]. Here, a crow had no pre-defined objective function apart from "eat food to stay alive" and has accomplished something remarkable. It found a food source that it had never had access to before, it developed a complex model of its urban environment, it combined its knowledge of the problem (the hard nut shell) with its knowledge of its environment (cars crush small objects), and it constructed a sophisticated for strategy for using cars to crack open the nuts and fetching the contents when the traffic lights indicated it was safe to do so. This is general intelligence! [1] https://www.youtube.com/watch?v=BGPGknpq3e0 https://www.youtube.com/watch?v=BGPGknpq3e0
petra 7y ago
Reinforcement learning is unsupervised and goal directed.
zozbot234 7y ago
It's not like there's a huge difference. Causal inference is a special case of curve-fitting, where one chooses what variables should enter into the fit according to a causal graph.
gus_massa 7y ago
> General intelligence is primarily about developing useful conceptual categories (not mapping to existing ones) … There are some algorithms like https://en.wikipedia.org/wiki/K-means_clustering https://en.wikipedia.org/wiki/K-means_clustering that get a set of data and try to create the categories to better classify them. There are many algorithm and the results don't agree all the time. But this is an open ended task, like the classification of biological species in animals. (Plants are more difficult, and bacterias even more.)
deleted 7y ago
[deleted]
bobcostas55 7y ago
By that definition the vast majority of humans do not possess general intelligence.
jimduk 7y ago
Computer Vision and Pattern Recognition. Was a good name, still will be. Great progress has been made, but it's not AGI as anyone who goes to CVPR will tell you.

ur-whale 7y ago

First, no AI that I know of has at its disposal a full blown model of the world it operates in, whereas most human brains do, and even if the model is imperfect, it is capable of producing fairly accurate simulations (what-if scenarios). Second, deep learning model, however much we'd like to think they do, aren't capable of doing proper causal inference in a general setting (that is, within the confines of the model) and are therefore far from capable of doing what humans do and will remain so limited for a long time to come. AGI will require the curve-fitting of deep learning, a general model of the world, the causal inference capabilities of something like AlphaGo, but in a general setting, not the super limited world AlphaGo operates in. So no, AGI will require much more than just curve-fitting abilities.

elcomet 7y ago
But all this is curve-fitting in a more general sense - fitting the curve of life, gene reproduction. So it is still curve fitting, it is just that a pottential AGI is certainly not in the hypothesis space of current deep learning models, and those cannot reach AGI by curve-fitting.
huffmsa 7y ago
"casual inference in a general setting" is just your brain running an input through it's existing thought and decision processes with a low threshold for a passing answer. So an ML model running an input through a collection of other models to see if it gets a reasonable answer.
longtom 7y ago
It's all pretty meaningless semantics and guesswork. Curve fitting means adaptive computation in networks of fairly simple units that allow for fairly general computation, i.e. traversing program space to find a good solution, or equivalently, intelligence is about evolving/searching a program that solves a wide array of tasks. It is about finding programs that maps from sensory space to the space of action sequences, maximizing reward. But you need the right prior structure such that learning and producing action sequences is efficient or even feasible/reachable. You can see any additional program structure that aids e.g. generation and recall of memories and planning (production of output targeted at solving a goal) as prior structure that limits and defines the searched program space. You can even regard a planning module as part of the curve fitting as it simply concerns the last step of producing the output. Therefore, intelligence is "curve fitting". So the actual question is: How much additional structure over just a large number of simple repeated units is necessary? Nobody knows. Possibly not much. Possibly quite a bit.

yk 7y ago

That is basically what Chalmers argues in "Facing up the problem of consciousness." Basically, by the general approximation theorem, it is possible to find a neural network that acts externally precisely as you would, up to an arbitrarily small epsilon. However, one wonders if such a thing would be consciousness and if so, where does the consciousness sit, in the matrix multiplication or the graphics card. So, you can argue something like our brains are nothing but curve fitting machine with enough parameters, but then you are probably forced to argue that consciousness is very closely related to computation, to the point were a coin flip, or a hello world program has some sliver of consciousness. There are of course two possible ways around that, either one can argue for p-zombies, that is intelligent but not conscious beings, which then seems to require a super natural explanation for consciousness. Or you can argue that the brain is different, and that this gives rise to consciousness, and to general intelligence, which is the explanation that at least corresponds most closely to my subjective experience (but that is precisely what a soulless machine would write, isn't it?)

joycian 7y ago
Why is it weird that simple programs have a sliver of consciousness? That seems like a nice practical, non-magic conclusion?
- yk 7y ago
  Actually I like that solution, but try to show it to any degree of scientific standard. You need to have a good working definition of consciousness, then you need some experimental procedure, etc. etc.
missosoup 7y ago
There's no need for a supernatural explanation of consciousness. We are all p-zombies. Problem solved. The longer we refuse to acknowledge that consciousness is nothing special, the longer it will take to tackle this topic. The only reason we cling to the idea that our minds are somehow special compared to other animals of various complexity is because we refuse to acknowledge that consciousness might exist in something we can't communicate with and that consciousness is a sliding scale rather than a binary property. Ascribing consciousness only to ourselves is hubris. If one spends a bit of time observing humans, they will inevitably realise that some humans are more 'conscious' than others also. tl;dr: we're all p-zombies. The fact that we think that each one of us isn't individually doesn't detract from that. Just as natural sciences left less and less hiding places for god to exist, ML is leaving less and less hiding places for this borderline magical version of human-unique consciousness to exist. Answering this question in any more detail requires a much more rigorous definition of consciousness which is a big can of worms in itself.

Isinlor 7y ago

If someone wants to go as far as to claim that any computation is just curve-fitting then your statement is equivalent to Church–Turing thesis. There are no formal arguments against Church–Turing thesis. From that perspective intelligence is indeed just a curve fitting. https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis I really enjoyed the "The Measure of Intelligence" by François Chollet. https://arxiv.org/abs/1911.01547 https://arxiv.org/abs/1911.01547 "We note that in practice, the contemporary AI community still gravitates towards benchmarking intelligence by comparing the skill exhibited by AIs and humans at specific tasks such as board games and video games. We argue that solely measuring skill at any given task falls short of measuring intelligence, because skill is heavily modulated by prior knowledge and experience: unlimited priors or unlimited training data allow experimenters to "buy" arbitrary levels of skills for a system, in a way that masks the system's own generalization power." He argues that we should move towards evaluating "Intelligence as skill-acquisition efficiency". I agree with him. We should move away from benchmarks that involve training and evaluating algorithms on the same datasets. This is indeed more or less "curve fitting". Instead we should focus on benchmarking how efficient algorithms are at solving tasks involving completely new datasets, preferably even unknown to the developers. For example, language model GPT-2 was trained to predict next word given some previous words. After that training GPT-2 was able to do things that were unrelated like question answering, translating etc. GPT-2 is of course doing that very badly, and requires GB of training data, but it is a step towards skill-acquisition efficiency and away from what everyone sees as curve-fitting. We should benchmark models so that we select for these that are able to do solve tasks they were not build to solve.

sgt101 7y ago
>There are no formal arguments against Church–Turing thesis. do you mean "all computation that can be done, can be done using a Turing machine"? Or do you mean "no one has proven Church Turing wrong?" If it's the second - yes, that's so. But so what? If it's the first then many people in Quantum Computing community will be quite upset, P=BQP? You have proof?
- voidmain 7y ago

jerf 7y ago

I am not accusing you of doing this in a bad way, but it kinda begs the question to argue that all intelligence is curve fitting, because if it is to be looked at that way, then we don't know the characteristics of the hyperspace on which it can be defined as "just" curve fitting, which is an important aspect of being able to carefully say that it is "just" curve fitting. While there almost certainly is such a hyperspace, just by virtue of the fact we're leaving ourselves so many degrees of freedom in our unspecified speculations here that it can't help but cover everything we ever do, we can't be confident that if we did know everything involved that it would be anywhere near the best representation, which is the only standard we can have in terms of whether or not something is "really" curve fitting. It isn't that hard to imagine that while an n-dimensional space can be defined upon which our intelligence is "curve fitting" that there is some better computational paradigm that would both describe it in fewer free parameters (or, in this case, a big-O different number of free parameters, probably) and also be easier to work with computationally, in which case we would have a solid ground to stand on to say that, no, it isn't really just "curve fitting". Or to bring it down to Earth in another way, consider just the act of writing a program in the modern world. If you work really, really hard, you can define a space in which our act of programming is just "curve fitting"... but it's far from obvious that that is even remotely a sensible way to look at the world. (See "differentiable programming" for the best counterpoint I know to that: https://en.wikipedia.org/wiki/Differentiable_programming https://en.wikipedia.org/wiki/Differentiable_programming but it's a very small niche right now.) When I'm debugging a program there is almost never any utility at all in trying to think about it as a "curve" and trying to get it closer to a the "correct" curve. A Turing-complete-complex space can be described as curves, but those curves are just awfully complicated and I don't see how it would be a help. My personal suspicion is that while our cognition involves rather less of this "Turing complete" thinking than we'd like to fancy ourselves using, we do irreducibly use elements of it [1], and as long as our best AI models are incapable of representing Turing-complete computations there is simply no chance of them being the answer to true human-scale cognition. (We do have models that can do it, e.g., evolutionary computation, but we lack any sensible idea of how to "update" such models like a neural net. Neural nets themselves in the simplest case aren't Turing complete, and none of the hybrid models seem to get there to me either, though I welcome correction on that point.) [1]: Evidence: I don't think we could program Turing-complete machines if we were incapable of thinking that way ourselves. We aren't necessarily great at it, our engineering techniques are deeply characterized by the fact we can't really manipulate very many things at once in this manner and we have no choice but to break things up into very small modules and for us to combine them in a way that means that at any given time we have only a very small number of things to keep track of locally, but we are still doing non-trivially more than zero of the Turing-complete style of thinking. It isn't a hard guess from there to think that even if we aren't all that great at a full mathematical manifestation of this style of thinking, we may indeed be doing something somewhere between what our current neural nets do and this full TC-style thinking at a larger scale, and the inability to capture this in our neural nets is a currently-fatal-flaw.

bitxbit 7y ago

It’s just fancy pattern matching. Multi-stage heuristics. Where it fails is data limitations. This is different from someone coming up with new insights based on different combinations of data inputs and better yet entirely new data/metrics.

zby 7y ago

What our brains do is curve fitting plus experiments. Causality is learned through experiments - and as children we do a lot of experiments (with moving our limbs etc). With just observations you can have only correlations. For example you can correlate smoke with fire, but only through experiment you can learn that it is fire that causes smoke not the other way around.

manx 7y ago
This is exactly what I thought, but was not able to put into words. Thank you.
mr_toad 7y ago
Just include time in the model and use lagged features. It’s used all the time in automated time series forecasting.
- tomrod 7y ago
  This can tease out transfer entropy but still isn't identifying causal factors. A common issue you find would be confounding. Then, because you haven't identified the latent connection, you may try to increase level A to have effect on output B, and be disappointed. This is basically Judea Pearls Book of Why's main hypothesis, that E(Y|X) != E(Y|do(X)), where do(X) is when we modify X somehow.