25 ms·

AI is mostly about curve fitting (2018)

• galangalalgol 7y ago
  Depending on how complex a curve, and how many dimensions it is in, couldn't you argues that is essentially what our brains do as well? Not that I am defending the massive hype field that is ML today, but curve fitting is a form of intelligence.
  • crazygringo 7y ago
    I was say fundamentally no -- and Douglas Hofstadter is probably the foremost spokesperson against intelligence being curve fitting. General intelligence is primarily about developing useful conceptual categories (not mapping to existing ones) and drawing cause-and-effect inferences that assist us in achieving goals. Curve fitting is just another name for pattern recognition, mapping to previously defined categories. I would personally argue there's no intelligence there whatsoever. Intelligence can't exist without a foundation of pattern recognition, but it isn't the same thing. Intelligence is fundamentally goal-directed and able to reason, while curve-fitting is fundamentally not. (There is also unsupervised learning in deep learning, which doesn't use previously defined categories, but since it is similarly non-goal-directed, I would still argue that this is merely dimension reduction as opposed to intelligence -- useful for sure, but not the same.)
    • codingslave 7y ago
      We dont know anything about AGI, and we arent sure how it would work. Its impossible to make assertions like this
      • quikoa 7y ago
        Yea I think we just don't know enough. Even if curve fitting alone wouldn't be sufficient it may very well play a crucial role in the agi algorithm.
      • deleted 7y ago
        [deleted]
      • slx26 7y ago
        we don't know everything, but we definitely know something about current ML, and we can indeed speculate and make some educated guesses about AGI. for a random comment on the internet, that contained some interesting information. of course, an actual implementation of AGI might be anything, but the comment was broader than that
      • chongli 7y ago
        We know a great deal more about biological general intelligence than we do about AGI. Many animals create their own goals and work to achieve them. We learn what works and what does not, and adapt our strategies to compensate. Humans do this (obviously) but a lot of other intelligent animals can do it as well. One of my favourite examples is the New Caledonian crows who have learned to use traffic to crack nuts [1]. Here, a crow had no pre-defined objective function apart from "eat food to stay alive" and has accomplished something remarkable. It found a food source that it had never had access to before, it developed a complex model of its urban environment, it combined its knowledge of the problem (the hard nut shell) with its knowledge of its environment (cars crush small objects), and it constructed a sophisticated for strategy for using cars to crack open the nuts and fetching the contents when the traffic lights indicated it was safe to do so. This is general intelligence! [1] https://www.youtube.com/watch?v=BGPGknpq3e0 https://www.youtube.com/watch?v=BGPGknpq3e0
    • petra 7y ago
      Reinforcement learning is unsupervised and goal directed.
    • zozbot234 7y ago
      It's not like there's a huge difference. Causal inference is a special case of curve-fitting, where one chooses what variables should enter into the fit according to a causal graph.
    • gus_massa 7y ago
      > General intelligence is primarily about developing useful conceptual categories (not mapping to existing ones) … There are some algorithms like https://en.wikipedia.org/wiki/K-means_clustering https://en.wikipedia.org/wiki/K-means_clustering that get a set of data and try to create the categories to better classify them. There are many algorithm and the results don't agree all the time. But this is an open ended task, like the classification of biological species in animals. (Plants are more difficult, and bacterias even more.)
    • deleted 7y ago
      [deleted]
    • bobcostas55 7y ago
      By that definition the vast majority of humans do not possess general intelligence.
    • jimduk 7y ago
      Computer Vision and Pattern Recognition. Was a good name, still will be. Great progress has been made, but it's not AGI as anyone who goes to CVPR will tell you.
  • ur-whale 7y ago
    First, no AI that I know of has at its disposal a full blown model of the world it operates in, whereas most human brains do, and even if the model is imperfect, it is capable of producing fairly accurate simulations (what-if scenarios). Second, deep learning model, however much we'd like to think they do, aren't capable of doing proper causal inference in a general setting (that is, within the confines of the model) and are therefore far from capable of doing what humans do and will remain so limited for a long time to come. AGI will require the curve-fitting of deep learning, a general model of the world, the causal inference capabilities of something like AlphaGo, but in a general setting, not the super limited world AlphaGo operates in. So no, AGI will require much more than just curve-fitting abilities.
    • elcomet 7y ago
      But all this is curve-fitting in a more general sense - fitting the curve of life, gene reproduction. So it is still curve fitting, it is just that a pottential AGI is certainly not in the hypothesis space of current deep learning models, and those cannot reach AGI by curve-fitting.
    • huffmsa 7y ago
      "casual inference in a general setting" is just your brain running an input through it's existing thought and decision processes with a low threshold for a passing answer. So an ML model running an input through a collection of other models to see if it gets a reasonable answer.
    • longtom 7y ago
      It's all pretty meaningless semantics and guesswork. Curve fitting means adaptive computation in networks of fairly simple units that allow for fairly general computation, i.e. traversing program space to find a good solution, or equivalently, intelligence is about evolving/searching a program that solves a wide array of tasks. It is about finding programs that maps from sensory space to the space of action sequences, maximizing reward. But you need the right prior structure such that learning and producing action sequences is efficient or even feasible/reachable. You can see any additional program structure that aids e.g. generation and recall of memories and planning (production of output targeted at solving a goal) as prior structure that limits and defines the searched program space. You can even regard a planning module as part of the curve fitting as it simply concerns the last step of producing the output. Therefore, intelligence is "curve fitting". So the actual question is: How much additional structure over just a large number of simple repeated units is necessary? Nobody knows. Possibly not much. Possibly quite a bit.
  • yk 7y ago
    That is basically what Chalmers argues in "Facing up the problem of consciousness." Basically, by the general approximation theorem, it is possible to find a neural network that acts externally precisely as you would, up to an arbitrarily small epsilon. However, one wonders if such a thing would be consciousness and if so, where does the consciousness sit, in the matrix multiplication or the graphics card. So, you can argue something like our brains are nothing but curve fitting machine with enough parameters, but then you are probably forced to argue that consciousness is very closely related to computation, to the point were a coin flip, or a hello world program has some sliver of consciousness. There are of course two possible ways around that, either one can argue for p-zombies, that is intelligent but not conscious beings, which then seems to require a super natural explanation for consciousness. Or you can argue that the brain is different, and that this gives rise to consciousness, and to general intelligence, which is the explanation that at least corresponds most closely to my subjective experience (but that is precisely what a soulless machine would write, isn't it?)
    • joycian 7y ago
      Why is it weird that simple programs have a sliver of consciousness? That seems like a nice practical, non-magic conclusion?
      • yk 7y ago
        Actually I like that solution, but try to show it to any degree of scientific standard. You need to have a good working definition of consciousness, then you need some experimental procedure, etc. etc.
    • missosoup 7y ago
      There's no need for a supernatural explanation of consciousness. We are all p-zombies. Problem solved. The longer we refuse to acknowledge that consciousness is nothing special, the longer it will take to tackle this topic. The only reason we cling to the idea that our minds are somehow special compared to other animals of various complexity is because we refuse to acknowledge that consciousness might exist in something we can't communicate with and that consciousness is a sliding scale rather than a binary property. Ascribing consciousness only to ourselves is hubris. If one spends a bit of time observing humans, they will inevitably realise that some humans are more 'conscious' than others also. tl;dr: we're all p-zombies. The fact that we think that each one of us isn't individually doesn't detract from that. Just as natural sciences left less and less hiding places for god to exist, ML is leaving less and less hiding places for this borderline magical version of human-unique consciousness to exist. Answering this question in any more detail requires a much more rigorous definition of consciousness which is a big can of worms in itself.
  • Isinlor 7y ago
    If someone wants to go as far as to claim that any computation is just curve-fitting then your statement is equivalent to Church–Turing thesis. There are no formal arguments against Church–Turing thesis. From that perspective intelligence is indeed just a curve fitting. https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis I really enjoyed the "The Measure of Intelligence" by François Chollet. https://arxiv.org/abs/1911.01547 https://arxiv.org/abs/1911.01547 "We note that in practice, the contemporary AI community still gravitates towards benchmarking intelligence by comparing the skill exhibited by AIs and humans at specific tasks such as board games and video games. We argue that solely measuring skill at any given task falls short of measuring intelligence, because skill is heavily modulated by prior knowledge and experience: unlimited priors or unlimited training data allow experimenters to "buy" arbitrary levels of skills for a system, in a way that masks the system's own generalization power." He argues that we should move towards evaluating "Intelligence as skill-acquisition efficiency". I agree with him. We should move away from benchmarks that involve training and evaluating algorithms on the same datasets. This is indeed more or less "curve fitting". Instead we should focus on benchmarking how efficient algorithms are at solving tasks involving completely new datasets, preferably even unknown to the developers. For example, language model GPT-2 was trained to predict next word given some previous words. After that training GPT-2 was able to do things that were unrelated like question answering, translating etc. GPT-2 is of course doing that very badly, and requires GB of training data, but it is a step towards skill-acquisition efficiency and away from what everyone sees as curve-fitting. We should benchmark models so that we select for these that are able to do solve tasks they were not build to solve.
    • sgt101 7y ago
      >There are no formal arguments against Church–Turing thesis. do you mean "all computation that can be done, can be done using a Turing machine"? Or do you mean "no one has proven Church Turing wrong?" If it's the second - yes, that's so. But so what? If it's the first then many people in Quantum Computing community will be quite upset, P=BQP? You have proof?
      • voidmain 7y ago
  • jerf 7y ago
    I am not accusing you of doing this in a bad way, but it kinda begs the question to argue that all intelligence is curve fitting, because if it is to be looked at that way, then we don't know the characteristics of the hyperspace on which it can be defined as "just" curve fitting, which is an important aspect of being able to carefully say that it is "just" curve fitting. While there almost certainly is such a hyperspace, just by virtue of the fact we're leaving ourselves so many degrees of freedom in our unspecified speculations here that it can't help but cover everything we ever do, we can't be confident that if we did know everything involved that it would be anywhere near the best representation, which is the only standard we can have in terms of whether or not something is "really" curve fitting. It isn't that hard to imagine that while an n-dimensional space can be defined upon which our intelligence is "curve fitting" that there is some better computational paradigm that would both describe it in fewer free parameters (or, in this case, a big-O different number of free parameters, probably) and also be easier to work with computationally, in which case we would have a solid ground to stand on to say that, no, it isn't really just "curve fitting". Or to bring it down to Earth in another way, consider just the act of writing a program in the modern world. If you work really, really hard, you can define a space in which our act of programming is just "curve fitting"... but it's far from obvious that that is even remotely a sensible way to look at the world. (See "differentiable programming" for the best counterpoint I know to that: https://en.wikipedia.org/wiki/Differentiable_programming https://en.wikipedia.org/wiki/Differentiable_programming but it's a very small niche right now.) When I'm debugging a program there is almost never any utility at all in trying to think about it as a "curve" and trying to get it closer to a the "correct" curve. A Turing-complete-complex space can be described as curves, but those curves are just awfully complicated and I don't see how it would be a help. My personal suspicion is that while our cognition involves rather less of this "Turing complete" thinking than we'd like to fancy ourselves using, we do irreducibly use elements of it [1], and as long as our best AI models are incapable of representing Turing-complete computations there is simply no chance of them being the answer to true human-scale cognition. (We do have models that can do it, e.g., evolutionary computation, but we lack any sensible idea of how to "update" such models like a neural net. Neural nets themselves in the simplest case aren't Turing complete, and none of the hybrid models seem to get there to me either, though I welcome correction on that point.) [1]: Evidence: I don't think we could program Turing-complete machines if we were incapable of thinking that way ourselves. We aren't necessarily great at it, our engineering techniques are deeply characterized by the fact we can't really manipulate very many things at once in this manner and we have no choice but to break things up into very small modules and for us to combine them in a way that means that at any given time we have only a very small number of things to keep track of locally, but we are still doing non-trivially more than zero of the Turing-complete style of thinking. It isn't a hard guess from there to think that even if we aren't all that great at a full mathematical manifestation of this style of thinking, we may indeed be doing something somewhere between what our current neural nets do and this full TC-style thinking at a larger scale, and the inability to capture this in our neural nets is a currently-fatal-flaw.
  • bitxbit 7y ago
    It’s just fancy pattern matching. Multi-stage heuristics. Where it fails is data limitations. This is different from someone coming up with new insights based on different combinations of data inputs and better yet entirely new data/metrics.
  • zby 7y ago
    What our brains do is curve fitting plus experiments. Causality is learned through experiments - and as children we do a lot of experiments (with moving our limbs etc). With just observations you can have only correlations. For example you can correlate smoke with fire, but only through experiment you can learn that it is fire that causes smoke not the other way around.
    • manx 7y ago
      This is exactly what I thought, but was not able to put into words. Thank you.
    • mr_toad 7y ago
      Just include time in the model and use lagged features. It’s used all the time in automated time series forecasting.
      • tomrod 7y ago
        This can tease out transfer entropy but still isn't identifying causal factors. A common issue you find would be confounding. Then, because you haven't identified the latent connection, you may try to increase level A to have effect on output B, and be disappointed. This is basically Judea Pearls Book of Why's main hypothesis, that E(Y|X) != E(Y|do(X)), where do(X) is when we modify X somehow.
• deleted 7y ago
  [deleted]
• psv1 7y ago
  I tend to avoid using the term AI at all and I mostly agree with the article, but I still don't see this as any kind of drawback. Computers are great at curve-fitting and there are many good use-cases for these tasks. We just need to be clear that we are very far from the sci-fi vision of a single computer which can understand, learn and know everything. Which is absolutely fine.
• abeppu 7y ago
  I suppose there are lots of papers and results that fall into the "only curve fitting" bucket, but there are many exciting results in recent years that have been curve fitting + X, where ANNs formed part of a system with other components. While none of these approach the level of something you could have a meaningful conversation with (which was an ambitious bar introduced in the article) some do kind of consider multiple actions, work through consequences of each, and pick a single action. This looks a lot like a crude form of reasoning about causality and hypotheticals, though not quite along the lines Pearl would like to see. But they typically do that in the context of a specific task, with an enumerable set of available actions.
• finnmagic 7y ago
  shhh, let's keep it that way
  • quikoa 7y ago
    Why? Afraid of Skynet taking over?
    • le3dh4x0r 7y ago
      If you mean with Skynet an army of limited chatbots, then yes. We are all scared
      • quikoa 7y ago
        You joke but it could ruin plenty of forums!
• dr_dshiv 7y ago
  So, one view of AI is that is is machine learning. Another view is that it consists of automated process. So, if expert systems count as AI, or if production rule systems count as AI, then pretty much any handcrafted if-then statement counts as AI. And, so too might any automated process.. This is has the advantage of recognizing how human intelligence can be automated and aggregated into system processes. And, the disadvantage that the boundaries of the concept start exploding. I like cybernetics for providing a clear model of what constitutes intelligence -- a feedback loop between perception and action that achieves goals or lowers local entropy. And, cybernetic systems can be artificial, natural or a mix
  • wayoutthere 7y ago
    I too hold the view that artificial intelligence is not reliant on computers. Sufficiently complex business logic is impossible for a single person -- even the CEO -- to have an end-to-end view (much less have a significant influence on). The emergent behavior of large companies fits a definition of an AI today; human "decision makers" are increasingly just reviewing and approving the output of algorithms. I think the end state for large corporations will be to automate away so much of the human input that they end up looking like what we think of as "AI" in a broader cultural context. But we already live in a world controlled by AI, and we have for 3+ decades.
    • gus_massa 7y ago
      I agree. Just note that it is related to https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room
    • dr_dshiv 7y ago
      Yeah, I mean, it's a slippery slope, but one worth sliding down. Is autopilot AI? If so, it was invented in 1914. Are speed governors AI? That was James Watt. Are if-then statements AI? That's the basis of human laws, dating back thousands of years. Are corporations AI, or rather superintelligences, because they are groups of people bound by bylaws? If so, the whole damn world is AI through and through and machine learning is really just the tip of the iceberg.
• tomthe 7y ago
  Well, most of science is also just curve fitting.
  • scottlocklin 7y ago
    Pretty sure Maxwells Equations and the laws of thermodynamics didn't come from curve fitting! Experimentalists may have verified these ideas using some kind of curve fitting, but thinking in abstractions, aka, "a ball rolled out in the street, maybe a child will follow," is one of the things curve fitting can't do.
    • sgt101 7y ago
      As Uber found out.
    • gus_massa 7y ago
      Most of the terms in Maxwell Equations were added one at a time to fit some restrictive set of experiments. https://en.wikipedia.org/wiki/Maxwell%27s_equations#Conceptual_descriptions https://en.wikipedia.org/wiki/Maxwell%27s_equations#Conceptu... The last term was a leap of faith. The Plank Equation for the black body https://en.wikipedia.org/wiki/Planck%27s_law https://en.wikipedia.org/wiki/Planck%27s_law was the most successful curve fitting in history. They need like 30 years to discover quantum mechanics and understand the details.
• huffmsa 7y ago
  I'd say this is outdated. Since its publication, transfer learning has really started to pick up speed. Taking a model trained in one subject, and reapplying it to build out new vertical is pretty much how children learn new things. We'll have models using models to train new models sooner rather than later. If a program can create it's own curve functions and decide which one to use to evaluate a problem, is it that much different than a brain?
  • jgalt212 7y ago
    Isn't transfer learning just a faster way to fit a curve by starting with a curve that's partially fit instead of a random vector. So under this interpretation, transfer learning is also "curve fitting."
    • huffmsa 7y ago
      It is. But progress has been made taking models fit on pretty different topics and using them for something tangentially related. And that's exactly what humans do. You don't start from zero when you learn to paint if you already know how to draw. You take another model of behavior and use it as a baseline to develop your painting behavior. The fundamental thing that you can do that the computer can't right now is decide when you need a new model, when to use an existing one and when to transfer off of something.
• bitL 7y ago
  ML is about function approximation where function can be anything and mostly something that is practically intractable by usual analytical or numerical methods, such as observing real-time processes with millions of variables. In math, via a reduction, human life is just a function of time with many inputs and many outputs. I.e. you can do curve fitting there as well...
• streetcat1 7y ago
  So RL is true AI. Alpha GO did make moves inconceivable to the best human mind on the subject. There was no curve to fit into.
  • bitL 7y ago
    The learning from game itself was curve fitting, the Deep in Deep Reinforcement Learning usually means some difficult function is replaced by a deep neural network, approximating optimal values (for moves) trained on gameplay samples, usually in sense of rewards/punishments for reaching certain states; in games they could rank e.g. good/bad moves, winning states, losing states etc.
    • streetcat1 7y ago
      Right. But the curve itself, was invented by the machine.
      • mkl 7y ago
        I think the curve is defined by the rules of the game, and the machine learned some details of it that humans hadn't figured out yet.
  • arkano 7y ago
    It's curve-fitting with a few extra steps. You can do a lot with curve-fitting though.
• deleted 7y ago
  [deleted]
• le3dh4x0r 7y ago
  "AI is about X, not Y." Maybe we all should consider calling it what it is and not push marketing buzz words around.
  • m_mueller 7y ago
    I call it pattern matching.
• cjauvin 7y ago
  To me, this debate is akin to wondering whether the essence of computing resides in the lower-level, Turing-like operations of a microprocessor or rather in the higher-level constructs and abstractions that we're able to build way "above" them. Whatever it is, intelligence, whether implemented artificially or embodied in the substrate of a living being, is likely built on a ladder of subsystems interoperating at different levels of abstraction.
  • visarga 7y ago
    It's like a form of incredulity that a higher level can exist on top of a lower level. It's all 0's and 1's, so it can't be anything more.
• ipsa 7y ago
  "Machine learning" used to be a safe haven. You could flee there to escape the Terminators and brain-on-a-chip graphics. Business PR deliberately killed that. They wanted their ML algorithms to be refered to as AI, so they could fully ride the hype train. AI used to be a tight quirky community. Having the brain as inspiration led to all sorts of anthropomorphizing. This was ok. Researchers understood what was meant with "learning", "intelligence", "to perceive" in the context of AI. Nowadays, it is almost irresponsible to do this, not because you'll confuse your co-researchers, but because popular tech articles will write about chatbots inventing their own language and having to be shutdown. Still, as a business research lab, it is good to get your name out there, so all the wrong incentives are there: Careful researchers avoid anthropomorphizing, and lose their source of inspiration -- you can not be careful with difficult unsolved problems, you need to be a little crazy and "out there". Meanwhile, profit-seeking business engineers and their PR departments, obfuscate their progress and basic techniques, all to get that juicy article with "an AI taught itself to X and you won't believe what happened next". The researchers actually busy solving the hard problems of vision, natural language understanding, and common sense, do not have time to write books about how AI is not yet general. Nobody from the research community ever claimed that, nobody came forward to claim they've solved these decade-old problems. It is people selling books railing against the popular reporting of AI. Boring, self-serving, and predictive, and you do not need to fit a curve to see that. All this quarreling about definitions and Venn diagrams and well-known limitations is dust in the wind. Go figure out what to call it on your Powerpoint presentation by yourself, and quit bothering the community.
  • account73466 7y ago
    Very well said. Also, curve fitting is not a corner case. Most relevant and intelligent things we care about can be solved with "just" curve fitting + extrapolation.
    • nextos 7y ago
      I think curve fitting is an important component of future AGI. But it definitely needs causal reasoning baked in, which leads to better models with less data [1,2]. My intuition is that there's a lot of important work to be done using logical representations of models and transforming them back and forth using well understood semantics operators. Deep functions will be part of said models, but the whole model does not necessarily need to be deep. We can already see hints of the field going in this direction in deep generative models [3]. [1] http://web.stanford.edu/class/psych209/Readings/LakeEtAlBBS.pdf http://web.stanford.edu/class/psych209/Readings/LakeEtAlBBS.... [2] https://probmods.org/ https://probmods.org/ [3] http://pyro.ai/examples/ http://pyro.ai/examples/
• yzh 7y ago
  Related: https://news.ycombinator.com/item?id=21459220 https://news.ycombinator.com/item?id=21459220
• jorblumesea 7y ago
  Curve fitting is a useful tool, even in non ML contexts like a simple linear regression. There's the hype, which will eventually die, and then there's the business/engineering aspect, which will likely stick around.
• jonplackett 7y ago
  Is it feasible to create AI on top of a bunch of other AI? Current AIs seem more like specific, fairly simple, brain regions. Maybe we need a level up.
  • Lambdanaut 7y ago
    This is kinda just what libraries like pytorch and tensorflow make easy. Architecting neural networks at a higher level and gluing them together.
    • jonplackett 7y ago
      I guess what I mean is at a much larger scale. Like 1000s of networks or maybe more, like I presume you get in the brain. Is this already what’s happening currently?
      • sgt101 7y ago
        Well - it's what Minsky advocated... https://en.wikipedia.org/wiki/Society_of_Mind https://en.wikipedia.org/wiki/Society_of_Mind But, I think that its a bit "hit and hope" - as an idea it doesn't really address any questions unless/until it is realised and works.
      • mr_toad 7y ago
        On today’s machines you’ll run out of computing power (and money) long before you reached the scale of the human brain.
  • toolslive 7y ago
    Yes, it's done all the time. Suppose you have a model A that predicts some value for input x and you also have model B for the same problem, and they both work, but are not optimal for all cases, so they do the following: C(x) = alfa * A(x) + (1-alfa) * B(x) tada. nil nove sub sole
• umarniz 7y ago
  I think intelligence has for some time already been boiled down to curve fitting, even for humans. Our current accepted definition of intelligence in schools is to get a score that is higher than the average to be considered sufficiently intelligent to proceed to the next grade. I feel anything that we develop for AI would fundamentally always be inspired by our own experiences and hence curve fitting is something we understand to be the best metric to optimize for.
  • npo9 7y ago
    > Our current accepted definition of intelligence in schools is to get a score that is higher than the average I don’t even think educators mistake that for intelligence. It’s, at best, a proxy for intelligence mixed with other factors.
  • lasagnaphil 7y ago
    I agree. In our current neoliberal era, the ones in power have already been replacing various systems in our society with algorithms and computers, and formulating everything into an optimization problem. As a result of this, the reverse has happened: the systems are now shaping humanity into something that could be optimized. We have been trying to manage governments, public services, and education the same as corporations, creating numerical targets for institutions to optimize for. Education itself was formulated as an optimization problem about how to create more jobs. Public services like healthcare were privatized and became a target for profit optimization. Half of the stock market is controlled by High-Frequency Trading supercomputers, which will do virtually anything to gain an upper-hand in profits. Those methods were all inherited from the management styles of corporations that began in the neoliberal era. As fundamental parts of our society are replaced by those systems, the society now curve-fits the systems rather than the systems curve-fitting the society. We now hyperoptimize ourselves to fit in this neoliberal landscape; our time is told as something to be optimized between work, socializing, exercising, and self-improving, with no space for "actual free time" of our own. We go to college not to learn but to pass exams and get ourselves a good job that can sustain us. And the faults of our systems are now blamed to be individual problems: "You didn't optimize towards the current trends of the job market, it's your fault." And from the view of the corporations, we are just AI agents waiting to be optimized for cash, and we're now becoming one through fitting our bodies and minds to the social media that tries to maximize engagement and ad revenue no matter the real societal cost. Now, the real problem of AI compared to the algorithms of the past is its data-driven nature: it can only learn from what data you give it for training. We can only accumulate data from the past and never from the future, so the AI systems will just keep repeating the past, no matter what unseen change will come. We will lose the ability to imagine new political, economic alternatives, we will just be feeding ourselves the status quo, and societal advancement will stagnate at the hands of automated systems. The cancellation of the future: this is what I'm ultimately afraid of.
• Lambdanaut 7y ago
  I'm pretty sure that I'm just a curve trying to fit what it means to survive as a human.
  • virmundi 7y ago
    The difference is that you have the metacognition. You can think about your fitting. It’s possible to say that such a process is just another set of parameters, but they impact retro actively. At that point the metaphor of curve fitting might break down.
• queensnake 7y ago
  Out of date. Reinforcement Learning makes truly ‘smart’ actions, courses of action. Calling something ‘curve fitting’ pooh poohs it (especially ‘just’ curve fitting) - calling it ‘distilling’ gives it the respect that it deserves.
• H8crilA 7y ago
  It's still not clear to me that natural intelligence is anything more than curve fitting.
  • trevyn 7y ago
    Indeed; I think things will get interesting once a good number of people start to realize and internalize this.
• deleted 7y ago
  [deleted]
• unityByFreedom 7y ago
  Great article. Why does AI hype persist? Simply because we're seeing moderate gains? Newsflash- We're likely to continue seeing gains from ML tech for decades. Basic techniques may become part of core CS. With all the private siloed data out there, ML will need to be applied uniquely over and over. In decades to come, if marketing continues as-is, are salesmen going to keep laying claim to the moon (AGI) the whole time? It's hard to believe we can remain on this tilt for so long.
• avmich 7y ago
  > Our machines are still incapable of independently coming up with a thought or hypothesis, testing it against others and accepting or rejecting its validity based on reasoning and experimentation, i.e. following the core principles of the scientific method. https://www.scientificamerican.com/article/robots-adam-and-eve-ai/ https://www.scientificamerican.com/article/robots-adam-and-e... A prime example of this is "Adam," an autonomous mini laboratory that uses computers, robotics and lab equipment to conduct scientific experiments, automatically generate hypotheses to explain the resulting data, test these hypotheses, and then interpret the results.
• atemerev 7y ago
  Well, 95% of human activity is about curve fitting, so what?
• buboard 7y ago
  Thanks for the link to the J.Pearl interview , it 's very interesting. There are many counterpoints that are not examined though: - There's nothing wrong with curve fitting per se. NNs fit hundreds of curves in parallel and many of them may contain cues about the causal structure of the data. - Deep learning has become part of reinforcement learning, which is trying to learn a causal structure. The primary determinant of causality is the temporal order of cause and effect. The question is , do humans use other hints apart from time for causal inference? - There is also not much evidence from neuroscience that wet brains are causality-inferrence machines, most of the evidence is that they are decision-making machines. Humans are also pretty bad at inferring causality when it's not obvious, but we re pretty good at associations/patterns. - Reasoning (conscious) is often considered to act on a meta-level, which observes the internal action of the human brain itself and vocalizes what it sees. What the brain sees at this level is not the external world, but the representation, and we don't have evidence there is a model of the world in there (except perhaps temporary maps of space that exist in hippocampus). Assuming this is true, it s not impossible that current methods can be extended so that self-explaining an NN ends up being causal reasoning the more important question is whether any of these methods can lead to the remarkable ability of brains to generate extremely intricate and improbable causal chains. can we get a CNN to start from a photo of maxwell's equations and output the theory of relativity? who knows
• Veedrac 7y ago
  The article is less awful than the title. In short, the thesis is that ML seems only able to learn associations, rather than stronger, causal models. It should be fairly obvious that ‘curve fitting’ is a misleading category—these models are clearly learning highly meaningful latent spaces that no prior approaches ever did. But I would agree that the actual high-level ability to make causal inferences seems to be lacking. Where I disagree with Pearl is simply with the idea that these stronger models won't emerge through future research. It's too early to say this, after barely a decade of large-scale AI research that has been undergoing continual rapid progress. Greater generality and more powerful models are some of the most well-established goals of the field.
  • manifestsilence 7y ago
    This is interesting, as this appears to be similar to the difference between Daniel Kahneman's System1 and System2 modes of human thought. ML is perhaps beginning to approach our subconscious, associative intelligence. This shows up in its excellence for things like image processing, which we do instantly and automatically. Perhaps growing the tech equivalent of a prefrontal cortex is what's going to be hard, or require a different approach.
    • _0ffh 7y ago
      We're getting there. I have the feeling more an more people subscribe to the hunch that combining symbolic (GOFAI) with subsymbolic (PDP/NN) techniques is the way forward. Research is going on, an example is the Neuro-Symbolic Concept Learner described in this [1] paper. [1] https://deepai.org/publication/the-neuro-symbolic-concept-learner-interpreting-scenes-words-and-sentences-from-natural-supervision https://deepai.org/publication/the-neuro-symbolic-concept-le...
    • Symmetry 7y ago
      Yeah, that seems to be part of the reason that AlphaZero, which has its System2-like alpha-beta search seems so much smarter than AlphaStar which plays like a brilliant somnambulist. Maybe if we had some way of abstracting out the things a machine learning system implicitly learns so we could deal with them in a more classical AI-like way?
• ryeguy_24 7y ago
  It was so refreshing to read the title of this article. This couldn’t be more true and timely. Everyone and their brother is talking about artificial intelligence and it’s frankly annoying at this point. These models are, simply put, just fancy interpolation/ extrapolation approaches.
  • lostmsu 7y ago
    And we are not?
    • ryeguy_24 7y ago
      Point taken. I don’t have a good response to that.
• madacol 7y ago
  "Humans today and tomorrow are mostly about curve fitting, not intelligence" That would still makes sense
• lordnacho 7y ago
  I actually started reading the Book of Why and I recommend it to the HN crowd. Pearl does a really good job of going through the history of causality, including the quite interesting story behind the now known to be wrong (or at least incomplete) claim that "correlation is not causality". He then places causality on a 3-rung scale. The bottom rung is association in data, which is where he says AI is stuck. Then there's intervention, "what if I do this thing?", and then there's counterfactuals, "what if I had done some other thing?" He then makes a case for what intelligence actually is, and unsurprisingly it's getting up those three rungs. The method revolves around directed graphs, which have certain unintuitive properties. For instance if A and B can both cause C, knowing that A is unlikely make B more likely, given that C happened. There's a few other stock situations in various graphs that he walks through as well. In the end the point seems to be that we could have a causal machine if we'd spend some more time on it. It would take data and try out some potential graphs, and some of the graphs would be ruled out by the data. And then some algo would tell you things like whether a randomized trial is necessary or even whether you'd need one (yes, this is another revelation). I think there's also an argument that this is how people actually think, which makes sense because the graphs are not terribly large and they need to fit in your meat hardware. I haven't finished it but I would guess that you could take it in another direction and say this is why some animals sorta have intelligence in that they learn patterns, but they don't know the higher rungs. Really interesting ideas, and at least clarifies what we mean by causality.
  • disqard 7y ago
    This is a great summary. Thanks for sharing it!
• scottlegrand2 7y ago
  When I was working on recommenders at Amazon. I didn't really perceive this as modeling the mind of some sort of Oracle. Instead I saw it as a change of basis from the space of previous purchases to the space of recent purchases. That approach led to an entire AI framework and it still makes Amazon quite a bit of money annually apparently if Jeff Wilke's 2019 Re:MARS speech is to be believed. I'd love to work on something more ambitious like AlphaGo or AlphaFold but those require tremendous resources and I'm really focused on bang for buck. But even then I'd see it as the marriage of classical search with modeling the probability of victory. If someone says that the AGI is almost upon us I pretty much bozo bit them no matter how prestigious or fancy they may be otherwise.
• wwarner 7y ago
  Some really insightful discussion of the possibilities and limits of ML can be found on Les Fridman's AI podcast. Especially good were interviews with Yann LeCunn, Jeff Hawkins, Elon Musk and Francois Chollet. One memorable quote from LeCun "There is no intelligence without learning." Here's a link to the LeCun interview, though many aothers are excellent. https://lexfridman.com/yann-lecun/ https://lexfridman.com/yann-lecun/
  • tomjakubowski 7y ago
    The interviews with researchers and engineers in the field seem like they'd be interesting, but I don't see it for Musk. What special insights does he offer into AI?
    • wwarner 7y ago
      Tesla is deeply invested in self driving technology. He discussed the importance of data in the project, and he believes that Tesla is collecting 20x more data than any other player. He also believes that driverless will be more than 100x safer than humans fairly soon.
• mark_l_watson 7y ago
  I use the term “AI” but couch it in the context that we are probably ten to a hundred years away from general artificial intelligence. I have a lower bar on my definition of general artificial intelligence as something that can be narrow but has a world view model for its activities and the environment and this model changes with experience, the AGI can explain what it is doing in its narrow field of expertise, and can creatively develop new techniques for solving problems in its narrow domain of expertise. I have been paid to work in the field of AI since 1982 so I have experienced AI winters. I almost hope we have another to act sort-of like simulated annealing to get us out of the highly effective local minimum of deep learning. I have been paid to work with deep learning for about the last five years (except I retired six months ago) and I love it, a big fan, but it won’t get us to where I want to be. Perhaps hybrid symbolic AI and deep learning? I don’t know.
• yters 7y ago
  What if human intelligence is not computable? Why does no researcher address the fundamental assumption of the field?
  • gus_massa 7y ago
    It is possible that human intelligence is not computable, but it looks veeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeery difficult to prove it. Nobody is sure. So the options are: 1) Try to prove that human intelligence is not computable. 2) Give up and go to another research field, like particle physics or artificial sweeteners or whatever. 3) Try to see how close to human intelligence can you get with the current technology. Some small areas may have a good approximation, like image classification, playing chess, … 4) Try to make a fully functional bug compatible human simulation. Most people is in 2 or 3.
    • yters 7y ago
      Why does it seem so difficult? For example, currently human intelligence accomplishes much more with much less resource usage (many many orders of magnitude) than any AI algorithm we've developed. Someone could look at the trend, and get an idea of the extent of improvement we'd have to reach parity with human intelligence, and see if it seems at all feasible. A simple quantitative analysis of what we know so far. At any rate, I'm unaware of any sort of significant effort at #1.
  • robotresearcher 7y ago
    This has been addressed by many authors since modern computation was formulated, for example by Turing himself. We learn about and discuss Newell and Simon's Physical Symbol System hypothesis in undergrad classes, ie. explicitly stating the underlying assumption. Once in a while someone will assert that computation is not sufficient, eg. Penrose, and generate discussion. In practice you can work on AI-the-engineering-discipline without taking a position on this. Perhaps this is why you feel that researchers don't talk about it. Disciplined scientists tend not to take public positions on things they don't know for sure.
    • 7y ago
• suyash 7y ago
  Lets repeat this and tell it to everyone : Deep Neural Nets is not AI, it's just one powerful idea amongst many other ideas within AI
• antichronology 7y ago
  Where is the proof that human "rational" thinking is anything but curve fitting? I'm not convinced
• cs702 7y ago
  This article conflates two separate, very different issues into one: * Issue #1: There is a tremendous amount of hype, noise, and snake oil surrounding the moniker "AI." Pretty much everyone agrees with this statement. (And anyone who doesn't agree with it... is probably selling snake oil.) * Issue #2: Is intelligence just a form of "curve fitting," i.e., is it just finding solutions to very complicated, high-dimensional problems with more and more computation via search and self-play? Note that DL via SGD is a form of learning via search, RL via state-action-reward mechanisms is a form of learning via search, and multi-model/multi-agent DL/RL are forms of learning via search with self-play.) There is sharp disagreement on the answer to this question #2. Is that really all there is to intelligence? The OP believes the answer to #2 is no: intelligence ought to be more than "curve fitting" via search and self-play. Others believe the answer to #2 is yes. For a typical example of their thinking, here's Rich Sutton, Distinguished Research Scientist at DeepMind: http://www.incompleteideas.net/IncIdeas/BitterLesson.html http://www.incompleteideas.net/IncIdeas/BitterLesson.html
  • TheRealPomax 7y ago
    If we're adding weight to Rich Sutton with that last sentence, can we also mention that Pearl is a highly decorated and distinguished researcher, author, and professor in the field? Because it seems odd to call him "OP" when his work has shaped the field of machine learning as much as it has to this very day.
    • cs702 7y ago
      Judea Pearl is NOT the author of the piece. The author is someone called Kurt Mako.
      • jtloong 7y ago
        True but most of the content of the article is based around Pearl's thoughts on the field.
        • cs702 7y ago
• tomerbd 7y ago
  also humans
• mycall 7y ago
  > Pearl contends that until algorithms and the machines controlled by them can reason about cause and effect, or at least conceptualize the difference, their utility and versatility will never approach that of humans. Is not cause and effect a temporal-based acyclic directed graph with mutating state (wrt CompSci). Anyways, I think ML needs to look closer at simple cases, like sea squirts which have 8617 neurons. Try to answer how and why do they work better than some of our algorithms.
• The_rationalist 7y ago
  https://why19.causalai.net https://why19.causalai.net
• verytrivial 7y ago
  And what if intelligence is actually just curve fitting? Maybe we (or AI) just need to find the right number of the right curves.
• d--b 7y ago
  Well, the point is to know if intelligence is not about curve fitting after all!
• anonu 7y ago
  In the financial markets, multi linear regression is a very popular tool for predicting the markets. That tool has been around for over a century and used extensively to make sense of financial data for decades now. Ive always eyed companies and startups the boast predictive AI for the financial markets with suspicion. I just always assumed whatever they were doing was a glorified regression model... But you have a lot of shameless people who will gladly slap some AI jargon in their business plan just to get eyeballs
• largespoon 7y ago
  If AI is mostly curve fitting, I'd like to know how curve fitting applies to AI and chess.
• fionic 7y ago
  I absolutely love this article
• fionic 7y ago
  Absolutely love this article. Couldn’t agree more.
• sjg007 7y ago
  Some DNNs compute posterior probabilities for classification. You could feed those inputs into a Bayesian network for decision making too but you could also train a NN to make those decisions. Still there may be times when a human derived controller is better with AI as sensory input than an AI controller itself.
• euske 7y ago
  I tend to think that ML is a kind of automated program generation that maximizes a certain metrics given by human. Some of them are based on real data but some aren't (like RL). So it's certainly more than a curve that needs to be fit. It's more like Turing machine fitting.
• peter_d_sherman 7y ago
  This is a potentially interesting identity for AI, that is, as a curve-fitting function... I postulate (but cannot at the present moment prove!) that if there is proof such that: a) AI fits curves and b) That if all polynomial numbers (NX^M .. AX^2 + BX^1 + CX^0) are in fact curves... then (perhaps) c) Perhaps there is a link between polynomial numbers (as curves) and AI... that is, perhaps all AI can be thought of as a function with a polynomial solution, such that F(X) = NX^M .. AX^2 + BX^1 + CX^0, with F(X) being the AI in question... I leave it to mathematicians, logicians, and people who do dimensional reductions/transformations -- to prove or disprove this...
• deleted 7y ago
  [deleted]
• steenreem 7y ago
  There's a lot of talk in this thread about what intelligence is. Personally I found the AIXI model from algorithmic information theory a very convincing definition: https://en.wikipedia.org/wiki/AIXI https://en.wikipedia.org/wiki/AIXI I'm surprised it's not more generally accepted as _the_ definition. Note that Markus Hutter the main author is currently working at DeepMind.