6 ms·

> It barely learns anything given the amount of computation effort an data that goes into it, but it just happens to be good enough to be practically preferable

by soared 8y ago

> It barely learns anything given the amount of computation effort an data that goes into it, but it just happens to be good enough to be practically preferable to old symbolic systems. I don’t follow this. Are you implying there haven’t been absolutely massive gains in computer vision, nlg, nlp, etc?

laretluval 8y ago
There’s going to be people plugging their ears and shouting “but it’s just nonlinear function approximation!” all the way into the singularity.
- ggm 8y ago
  Singularity be like fusion... Always far enough away to be hinted at from thin evidence
- perl4ever 8y ago
  Human intelligence involves choosing which context to use at any given point. So far, so called AI appears to take an external source of context for granted, which is why it seems to me fundamentally different from "real" intelligence, just like your eyes are different from your entire nervous system.
  - wetpaws 8y ago
    Every time there is some progress in AI people would yell "but this is not a real AI 'cause humans can..." ( play go/understand pictures/language/semantics/etc, goalpost is always in motion)
    - pdimitar 8y ago
      Stop playing the victim. Just make a driverless car that never makes mistakes already -- and I promise you the goalpost won't move for a while. Go and chess mean nothing out there. The world has a lot of very practical and concrete needs that this magical "AI" is nowhere near solving yet. I find the constant defensiveness of the ML proponents tiring and cringe-worthy. Somebody should remind you that what you are doing is not a backyard hobby. The people who fund you make a lot of promises and I don't think any of them have been delivered.
      - testrun 8y ago
        As if humans in cars does not make mistakes. Last time I checked humans have intelligence.
      - dang 8y ago
        This comment crosses into incivility. Please don't do that on HN, regardless of how right you think you are compared to someone else. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
        pdimitar 8y ago
        Understood and point taken. Thank you.
    - mtrimpe 8y ago
      AI carves away what it means to be human one trivial slice at a time. That's how it always has been and always will be until perhaps one day there is truly nothing 'truly human' left to slice off.
- goatlover 8y ago
  If it happens and doesn’t involve more than the current DL approach.
- jsinai 8y ago
  Classic regression is only intelligible because there were only a few parameters and people could use ANOVA to try and interpret them. IMO ANOVA is alchemy as well and most people trained to use it, don’t fully understand it. Should we leave decisions to those kinds of models? Not to mention that part of what makes NN such a step forward is precisely the high nonlinearity. When you have millions of parameters, the contribution of each is unimportant and so any kind of analog to ANOVA would be barking up the wrong tree. In a NN all those parameters work in concert to learn a decision boundary to separate data. It’s not intelligible at the minute level but at least we know what it’s doing in the end. The problem of course lies with the outliers and that’s not so much a problem with NN being a black box as it is a problem with the nature of large datasets and our own inability to rationalize each and every datapoint. I’m going to defend NN as the natural evolution of regression. It’s precisely their high nonlinearity which makes them better. The problem is not that they “are” alchemy but that we treat them “as” magic. Society as no place leaving important decisions to algorithms unintended, NN or not. If major insurance companies left their decisions to logistic regression (which was and is still the case), then would we be making the same arguments? Probably not because someone paid by the insurance company will pull out that other kind of alchemy called ANOVA ...
  - RA_Fisher 8y ago
    ANOVA is grounded by theory that has stood the test of time for decades. Statisticians that question ANOVA aren't a thing (despite such an accurate criticism being valuable career-wise to an academic). Gelman points out that multilevel models are easier to use than ANOVA and I tend to agree.
  - vharuck 8y ago
    Classic regression's potential for intelligibility relies on the modeler choosing terms which are based on observed relations and improve performance. For example, population forecasts can be modeled as the current population plus forecasted changes through immigration, emigration, birth, and death. Yes, somebody can produce "magical" regression models with terms divorced from reality. But unsupervised learning is never guaranteed to produce a reality-grounded model, no matter the user's skill. It goes through steps, tries different transformations, and chooses the algorithm which led to the best result. Logic and understanding played no part. That sounds like alchemy. It definitely works, but alchemy also stumbled onto theories later explained by chemistry. And, yes, anyone who doesn't consider the implications of the linear model behind ANOVA would also be a "magician."
turingspiritfly 8y ago
Those massive gains have yet to considered reliable enough to be considered trusthworthy. Would you consider them trusthworthy in court, where lives are at stake? Gains are nice but we are still so far from the essence of AI systems and considering how much resources we are pouring into learning, at this point all of them appear as nothing more than massive fat expensive toys
- laretluval 8y ago
  Humans are often not considered trustworthy and reliable in court.
  - EpicEng 8y ago
    What does that have to do with engineering?
    - laretluval 8y ago
      Trustworthiness in court was advanced as a metric for AI success at which current methods are failing. > Would you consider them trusthworthy in court, where lives are at stake?
- solotronics 8y ago
  I have the perspective of an informed layman as a programmer who hasn't messed with ML yet. Wouldn't the "safest" solution be a system with multiple algorithms and a consensus mechanism?
  - bumby 8y ago
    I believe some models do precisely that. Random forest ML as an example tallies "votes" on the outcome. I'm not sure how robustly multiple algorithms have been applied to this voting technique, but it would be an interesting read if anyone has information on it.
    - 8y ago
quanticle 8y ago
Are you implying there haven’t been absolutely massive gains in computer vision, nlg, nlp, etc? The implication is that the massive gains haven't been the result of any algorithmic breakthroughs, but rather have been due to the application of massive amounts of computational resources, which weren't available when current ML algorithms were invented. So far as I can tell, that's a true accusation. If you look at the papers coming out of Google and Facebook, they talk about throwing thousands of hours of specialized GPU (or even more specialized and expensive TPU) time at some of the problems. The advances have more to do with Moore's Law making brute force feasible than they have to do with algorithmic breakthroughs.
- EmbarrassedFuel 8y ago
  A small bone of contention: it’s not about Moore’s law per se, which has been ‘dead’ since about 2013, coincidentally when the deep learning revival started. It’s matrix multiplication ASIC development that is driving the progress. GPUs already existed when the idea to use them to make the feasible size of neural nets larger came about. For a long time the drive for the increase of GPU compute power was still gaming/commercial graphics houses. It’s really only in the last 1-2 years that we’ve seen highly specialised GPUs with features like tensor cores (or indeed google’s TPUs). Also, calling neural nets ‘brute force’ because they use a lot of computing power to train a model is slightly reductionist - a true brute force approach to image recognition, ie enumerating all possible combinations of, say, 200x200x256x3 pixels, would be completely absurd and probably exceed the computing power currently available on earth.
  - quanticle 8y ago
    I'm not saying that neural nets are brute force. I'm saying that there haven't been any algorithmic improvements in neural nets to make them more computationally feasible than they were when they were first invented. Instead, we have specialized hardware which can just do the necessary computation quickly enough to make neural nets feasible. It's not like neural nets are a new technology. They've been known since the '80s, at least. It's just that they were considered a dead end, because we didn't have the computational resources to run deep neural nets, nor did we have sufficient training data to make neural-net approaches feasible. Once those preconditions were met, neural nets took off in short order.

by soared 8y ago

laretluval 8y ago

There’s going to be people plugging their ears and shouting “but it’s just nonlinear function approximation!” all the way into the singularity.

ggm 8y ago
Singularity be like fusion... Always far enough away to be hinted at from thin evidence
perl4ever 8y ago
Human intelligence involves choosing which context to use at any given point. So far, so called AI appears to take an external source of context for granted, which is why it seems to me fundamentally different from "real" intelligence, just like your eyes are different from your entire nervous system.
- wetpaws 8y ago
  Every time there is some progress in AI people would yell "but this is not a real AI 'cause humans can..." ( play go/understand pictures/language/semantics/etc, goalpost is always in motion)
  - pdimitar 8y ago
    Stop playing the victim. Just make a driverless car that never makes mistakes already -- and I promise you the goalpost won't move for a while. Go and chess mean nothing out there. The world has a lot of very practical and concrete needs that this magical "AI" is nowhere near solving yet. I find the constant defensiveness of the ML proponents tiring and cringe-worthy. Somebody should remind you that what you are doing is not a backyard hobby. The people who fund you make a lot of promises and I don't think any of them have been delivered.
    - testrun 8y ago
      As if humans in cars does not make mistakes. Last time I checked humans have intelligence.
    - dang 8y ago
      This comment crosses into incivility. Please don't do that on HN, regardless of how right you think you are compared to someone else. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
      - pdimitar 8y ago
        Understood and point taken. Thank you.
  - mtrimpe 8y ago
    AI carves away what it means to be human one trivial slice at a time. That's how it always has been and always will be until perhaps one day there is truly nothing 'truly human' left to slice off.
goatlover 8y ago
If it happens and doesn’t involve more than the current DL approach.
jsinai 8y ago
Classic regression is only intelligible because there were only a few parameters and people could use ANOVA to try and interpret them. IMO ANOVA is alchemy as well and most people trained to use it, don’t fully understand it. Should we leave decisions to those kinds of models? Not to mention that part of what makes NN such a step forward is precisely the high nonlinearity. When you have millions of parameters, the contribution of each is unimportant and so any kind of analog to ANOVA would be barking up the wrong tree. In a NN all those parameters work in concert to learn a decision boundary to separate data. It’s not intelligible at the minute level but at least we know what it’s doing in the end. The problem of course lies with the outliers and that’s not so much a problem with NN being a black box as it is a problem with the nature of large datasets and our own inability to rationalize each and every datapoint. I’m going to defend NN as the natural evolution of regression. It’s precisely their high nonlinearity which makes them better. The problem is not that they “are” alchemy but that we treat them “as” magic. Society as no place leaving important decisions to algorithms unintended, NN or not. If major insurance companies left their decisions to logistic regression (which was and is still the case), then would we be making the same arguments? Probably not because someone paid by the insurance company will pull out that other kind of alchemy called ANOVA ...
- RA_Fisher 8y ago
  ANOVA is grounded by theory that has stood the test of time for decades. Statisticians that question ANOVA aren't a thing (despite such an accurate criticism being valuable career-wise to an academic). Gelman points out that multilevel models are easier to use than ANOVA and I tend to agree.
- vharuck 8y ago
  Classic regression's potential for intelligibility relies on the modeler choosing terms which are based on observed relations and improve performance. For example, population forecasts can be modeled as the current population plus forecasted changes through immigration, emigration, birth, and death. Yes, somebody can produce "magical" regression models with terms divorced from reality. But unsupervised learning is never guaranteed to produce a reality-grounded model, no matter the user's skill. It goes through steps, tries different transformations, and chooses the algorithm which led to the best result. Logic and understanding played no part. That sounds like alchemy. It definitely works, but alchemy also stumbled onto theories later explained by chemistry. And, yes, anyone who doesn't consider the implications of the linear model behind ANOVA would also be a "magician."

turingspiritfly 8y ago

Those massive gains have yet to considered reliable enough to be considered trusthworthy. Would you consider them trusthworthy in court, where lives are at stake? Gains are nice but we are still so far from the essence of AI systems and considering how much resources we are pouring into learning, at this point all of them appear as nothing more than massive fat expensive toys

laretluval 8y ago
Humans are often not considered trustworthy and reliable in court.
- EpicEng 8y ago
  What does that have to do with engineering?
  - laretluval 8y ago
    Trustworthiness in court was advanced as a metric for AI success at which current methods are failing. > Would you consider them trusthworthy in court, where lives are at stake?
solotronics 8y ago
I have the perspective of an informed layman as a programmer who hasn't messed with ML yet. Wouldn't the "safest" solution be a system with multiple algorithms and a consensus mechanism?
- bumby 8y ago
  I believe some models do precisely that. Random forest ML as an example tallies "votes" on the outcome. I'm not sure how robustly multiple algorithms have been applied to this voting technique, but it would be an interesting read if anyone has information on it.
  - 8y ago

quanticle 8y ago

Are you implying there haven’t been absolutely massive gains in computer vision, nlg, nlp, etc? The implication is that the massive gains haven't been the result of any algorithmic breakthroughs, but rather have been due to the application of massive amounts of computational resources, which weren't available when current ML algorithms were invented. So far as I can tell, that's a true accusation. If you look at the papers coming out of Google and Facebook, they talk about throwing thousands of hours of specialized GPU (or even more specialized and expensive TPU) time at some of the problems. The advances have more to do with Moore's Law making brute force feasible than they have to do with algorithmic breakthroughs.

EmbarrassedFuel 8y ago
A small bone of contention: it’s not about Moore’s law per se, which has been ‘dead’ since about 2013, coincidentally when the deep learning revival started. It’s matrix multiplication ASIC development that is driving the progress. GPUs already existed when the idea to use them to make the feasible size of neural nets larger came about. For a long time the drive for the increase of GPU compute power was still gaming/commercial graphics houses. It’s really only in the last 1-2 years that we’ve seen highly specialised GPUs with features like tensor cores (or indeed google’s TPUs). Also, calling neural nets ‘brute force’ because they use a lot of computing power to train a model is slightly reductionist - a true brute force approach to image recognition, ie enumerating all possible combinations of, say, 200x200x256x3 pixels, would be completely absurd and probably exceed the computing power currently available on earth.
- quanticle 8y ago
  I'm not saying that neural nets are brute force. I'm saying that there haven't been any algorithmic improvements in neural nets to make them more computationally feasible than they were when they were first invented. Instead, we have specialized hardware which can just do the necessary computation quickly enough to make neural nets feasible. It's not like neural nets are a new technology. They've been known since the '80s, at least. It's just that they were considered a dead end, because we didn't have the computational resources to run deep neural nets, nor did we have sufficient training data to make neural-net approaches feasible. Once those preconditions were met, neural nets took off in short order.